View
1.532
Download
4
Category
Preview:
DESCRIPTION
Presented by Mikhail Khludnev, eCommerce Search Platform, Grid Dynamics This talk describes our experience in eCommerce Search: challenges which we’ve faced and the chosen approaches. It’s not indented to be a full description of implementation, because too many details need to be touched. This talk is more like problem statement and general solutions description, which have a number of points for technical or even academic discussion. It’s focused on text search use-case, structures (or scoped) search is out of agenda as well as faceted navigation.
Citation preview
CONCEPT SEARCH FOR ECOMMERCE WITH SOLR
Mikhail KhludnevPrincipal Engineer,eCommerce Search Teammkhludnev@griddynamics.com
http://goo.gl/2OHk2
Agenda
● eCommerce Search is Special
● Subphrase Query Parser
● Staged Search
● Match Spotting
Part I eCommerce Search is Special
Boolean Retrieval
(+ -)
Vector Space Model
http://upload.wikimedia.org/wikipedia/commons/f/ff/Vector_space_
model.jpg
Does she search this way?
http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_county.jpg
+"michael kors" type:handbag
http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_county.jpg
michael kors hand bag
That's how she searches!
Plain Text Documents
Else Jeans Skinny Jeans, Colored Denim Dark Green-WashIn a dark green wash perfect for fall, these Else Jeans skinny jeans hit the colored-denim trend right on the mark! Lyocell cotton rayon polyester Lycra Machine washable Imported Low rise: approx. 8 inches Skinny fit Skinny leg Zipper fly with button closure 5-pocket style Dark green wash, colored denim Waistband with belt loops Inseam: approx. 30-1/2 inches
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Plain Text Documents
Else Jeans Skinny Jeans, Colored Denim Dark Green-WashIn a dark green wash perfect for fall, these Else Jeans skinny jeans hit the colored-denim trend right on the mark! Lyocell cotton rayon polyester Lycra Machine washable Imported Low rise: approx. 8 inches Skinny fit Skinny leg Zipper fly with button closure 5-pocket style Dark green wash, colored denim Waistband with belt loops Inseam: approx. 30-1/2 inches
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
approx velit
BRAND: Calvin KleinGENDER: Women'sTYPE: JeansFIT: At WaistOCCASION: CasualLEG: Classic StraightWEIGHT: Super Skinny COLOR: Light Wash
Product Catalog Document
Polysemy Problem
Polysemy Problem
COLOR:"Pink" TYPE:"Sweater"
BRAND: "Tomas Pink" TYPE:"Sweater"BRAND: "Pink Lotus" TYPE:"Sweater"
http://www.shopstyle.com/browse/sweaters?fts=pink+sweater&fl=b7&fl=b28462&fl=b1830&fl=b15382&fl=b12130&fl=b737
http://www.shopstyle.com/browse/sweaters?fts=pink+sweater&fl=b7&fl=b28462&fl=b1830&fl=b15382&fl=b12130&fl=b737
pink jeans by google
https://www.google.com/search?hl=ru&tbm=shop&q=pink+sweater&oq=pink+sweater&gs_l=products-cc.3..0l7j0i5l3.4658.7244.0.7699.12.11.0.1.1.0.298.1153.5j4j1.10.0...0.0...1ac.1.denzbOMlNDg#q=pink+sweater&hl=ru&tbm=shop&ei=oDmIUPXgAoqC4gSylYGACg&start=20&sa=N&num=20&bav=on.2,or.r_gc.r_pw.r_qf.&fp=7898c08770da3440&bpcl=35466521&biw=1280&bih=660
pink jeans by google
https://www.google.com/search?hl=ru&tbm=shop&q=pink+sweater&oq=pink+sweater&gs_l=products-cc.3..0l7j0i5l3.4658.7244.0.7699.12.11.0.1.1.0.298.1153.5j4j1.10.0...0.0...1ac.1.denzbOMlNDg#q=pink+sweater&hl=ru&tbm=shop&ei=oDmIUPXgAoqC4gSylYGACg&start=20&sa=N&num=20&bav=on.2,or.r_gc.r_pw.r_qf.&fp=7898c08770da3440&bpcl=35466521&biw=1280&bih=660
right pink jeans
right pink jeans
Polysemy Problem No.2
BRAND:"Pink Rose" TYPE:"Sweater"
COLOR:"Pink","Rose" TYPE:"Sweater"
sweater
pink rose jeans problem
pink rose sweater
precise pink rose jeans
pink rose sweater
Ranking Can't Help
Ranking Can't Help
Precision is a MUST!
http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_county.jpg
Think About the User
http://www.amazon.com/Silver-Jeans/e/2206256011
Wrap-Up
● Product Descriptions Differs to Plain Text
● No Query Syntax
● Polysemy Problem
● Ranking Can't Help When Precision is Low
eCommerce Search is Special!
Part II Subphrase Query Parser
Don't tokenize at index time
black and decker toaster oven"black"
"black and"
"black and decker"
... "and"
"and decker"
"and decker toaster"
... "decker"
"decker toaster"
"decker toaster oven"
"toaster oven"
black and decker toaster oven"black"
"black and"
"black and decker"
... "and"
"and decker"
"and decker toaster"
... "decker"
"decker toaster"
"decker toaster oven"
..."toaster oven"
ID: 123BRAND: "black and
decker"TYPE: "toaster
oven"COLOR: white
black and decker toaster oven"black"
"black and"
"black and decker"
... "and"
"and decker"
"and decker toaster"
... "decker"
"toaster""decker toaster oven"
... "oven"
ID: 456BRAND: "black and
decker"TYPE: "oven"COLOR: white
black and decker toaster oven"black"
"black and"
"black and decker"
... "and"
"and decker"
"and decker toaster"
... "decker"
"decker toaster"
"decker toaster oven"
..."toaster oven"
ID: 789BRAND: "decker
toaster"TYPE: "toaster
oven"COLOR: black
fee people sweater
Part III Staged Search
Precision and Recall
Expected
ExpectedActual
ExpectedActual
×× OK
True Positive
False Negative
False Positive
Precision =
ExpectedActual
Actual ∩ Expected
Actual
ExpectedActual
Recall =Actual ∩ Expected
Expected
Expected
Actual
100% Recall
ExpectedActual
Tuning Trade-Off
ExpectedActual
Tuning Trade-Off
ExpectedActual
Tuning Trade-Off
ExpectedActual
Tuning Trade-Off
defaultOperator
AND OR
numFound=100 numFound=1000
defaultSearchField / queryFields
BRAND BRAND TYPE STYLE text
numFound=100 numFound=1000
Stemming & Synonyms
text text_stemmed
numFound=100 numFound=1000
BRAND:(Silver AND Jeans)
BRAND:(Silver OR Jeans)
BRAND_stem:(Silver OR Jean) OR
text_stem:(Silver OR Jean)
ORDer by DESCending Precision
numFound=100
numFound=1000
BRAND:(Silver AND Jeans)
BRAND:(Silver OR Jeans)
BRAND_stem:(Silver OR Jean) OR
text_stem:(Silver OR Jean)
numFound>0
numFound>0
Searching Complexity
O(n log k)
It Depends™
num found
page size
Stages Ratios
50% 30%
20%
Compounds - Exact Match
Compounds - Omit Words
Text Match
http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_county.jpg
Think About The User
Part IVMatch Spotting
Facet Counts
Explain Info15.110098 = (MATCH) sum of: 11.197777 = (MATCH) weight(BRAND:Free People in 28807), product of:
0.81810623 = queryWeight(BRAND:Free People), product of: 9.678479 = idf(docFreq=232, maxDocs=1368899) 0.08452839 = queryNorm
13.687436 = (MATCH) fieldWeight(BRAND:Free People in 28807), product of: 1.4142135 = tf(termFreq(BRAND:Free People)=2) 9.678479 = idf(docFreq=232, maxDocs=1368899) 1.0 = fieldNorm(field=BRAND, doc=28807) 3.912321 = (MATCH) weight(PRODUCT_TYPE:SWEATER in 28807), product of:
0.5750671 = queryWeight(PRODUCT_TYPE:SWEATER), product of: 6.8032427 = idf(docFreq=4130, maxDocs=1368899) 0.08452839 = queryNorm
6.8032427 = (MATCH) fieldWeight(PRODUCT_TYPE:SWEATER in 28807), product of: 1.0 = tf(termFreq(PRODUCT_TYPE:SWEATER)=1) 6.8032427 = idf(docFreq=4130, maxDocs=1368899) 1.0 = fieldNorm(field=PRODUCT_TYPE, doc=28807)
Match Spotting
ExplainInfo
FacetCounts
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"alfani" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" TYPE:"dress" STYLE:"silver"
BRAND:"style&co" TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" STYLE:"black"
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"silver jeans" TYPE:"jacket" STYLE: "black"
BRAND:"angie" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" STYLE:"blue"
BRAND:"dotty" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" STYLE:"jeans" "dress"
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"alfani" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" TYPE:"dress" STYLE:"silver"
BRAND:"style&co" TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" STYLE:"black"
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"silver jeans" TYPE:"jacket" STYLE: "black"
BRAND:"angie" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" STYLE:"blue"
BRAND:"dotty" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"chaloree" STYLE:"jeans" "dress"
silver jeans dress
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"alfani" TYPE:"dress" STYLE:"silver","jeans"BRAND:"chaloree" TYPE:"dress" STYLE:"silver"
BRAND:"style&co" TYPE:"jeans dress" STYLE:"silver"BRAND:"silver jeans" TYPE:"dress" STYLE:"black"
BRAND:"silver jeans" TYPE:"dress" STYLE:"white"
BRAND:"silver jeans" TYPE:"jacket" STYLE: "black"
BRAND:"angie" TYPE:"dress" STYLE:"silver","jeans"BRAND:"chaloree" TYPE:"jeans dress" STYLE:"silver"BRAND:"silver jeans" TYPE:"dress" STYLE:"blue"
BRAND:"dotty" TYPE:"dress" STYLE:"silver","jeans"BRAND:"chaloree" STYLE:"jeans" "dress"
BRAND:"silver jeans" TYPE:"dress" TYPE:"dress" STYLE:"silver","jeans"
TYPE:"jeans dress" STYLE:"silver"BRAND:"silver jeans" TYPE:"dress" BRAND:"silver jeans" TYPE:"dress"
TYPE:"dress" STYLE:"silver","jeans"TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"silver jeans" TYPE:"dress" TYPE:"dress" STYLE:"silver","jeans"
TYPE:"jeans dress" STYLE:"silver"BRAND:"silver jeans" TYPE:"dress"
BRAND:"silver jeans" TYPE:"dress"
TYPE:"dress" STYLE:"silver","jeans"TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" TYPE:"dress" STYLE:"silver","jeans"
BRAND:"silver jeans" TYPE:"dress" (4)TYPE:"dress" STYLE:"silver","jeans"
TYPE:"jeans dress" STYLE:"silver"
TYPE:"dress" STYLE:"silver","jeans"TYPE:"jeans dress" STYLE:"silver"
TYPE:"dress" STYLE:"silver","jeans"
BRAND:"silver jeans" TYPE:"dress" (4)TYPE:"dress" STYLE:"silver","jeans"
TYPE:"jeans dress" STYLE:"silver"
TYPE:"dress" STYLE:"silver","jeans"
TYPE:"jeans dress" STYLE:"silver"
TYPE:"dress" STYLE:"silver","jeans"
BRAND:"silver jeans" TYPE:"dress" (4)TYPE:"dress" STYLE:"silver","jeans" (3)
TYPE:"jeans dress" STYLE:"silver"
TYPE:"jeans dress" STYLE:"silver"
BRAND:"silver jeans" TYPE:"dress" (4)TYPE:"dress" STYLE:"silver","jeans" (3)
TYPE:"jeans dress" STYLE:"silver" (2)
BRAND: "Silver Jeans" - (156)
COLOR: "Silver" TYPE: "Jeans" - (34)
found 190
http://www.amazon.com/Silver-Jeans/e/2206256011
BRAND: "Silver Jeans" - (156)
COLOR: "Silver" TYPE: "Jeans" - (34)
found 190
Wrap-Up
● eCommerce Search is Special
● Subphrase Query Parser
● Staged Search
● Match Spotting
CONTACT Mikhail Khludnevmkhludnev@griddynamics.com
http://goo.gl/2OHk2
Recommended