Concept search for e commerce with solr

  • View
    1.531

  • Download
    4

Embed Size (px)

DESCRIPTION

Presented by Mikhail Khludnev, eCommerce Search Platform, Grid Dynamics This talk describes our experience in eCommerce Search: challenges which we’ve faced and the chosen approaches. It’s not indented to be a full description of implementation, because too many details need to be touched. This talk is more like problem statement and general solutions description, which have a number of points for technical or even academic discussion. It’s focused on text search use-case, structures (or scoped) search is out of agenda as well as faceted navigation.

Text of Concept search for e commerce with solr

  • 1. CONCEPT SEARCH FOR ECOMMERCEWITH SOLRMikhail KhludnevPrincipal Engineer,eCommerce Search Teammkhludnev@griddynamics.comhttp://goo.gl/2OHk2
  • 2. Agenda eCommerce Search is Special Subphrase Query Parser Staged Search Match Spotting
  • 3. Part IeCommerce Search is Special
  • 4. BooleanRetrieval(+ -)VectorSpaceModelhttp://upload.wikimedia.org/wikipedia/commons/f/ff/Vector_space_model.jpg
  • 5. Does she search this way?http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_cou+"michael kors" type:handbag
  • 6. http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_coumichael kors hand bagThats how she searches!
  • 7. Plain Text DocumentsElse Jeans Skinny Jeans,Colored Denim Dark Green-WashIn a dark green wash perfectfor fall, these Else Jeans skinnyjeans hit the colored-denimtrend right on the mark! Lyocellcotton rayon polyester LycraMachine washable ImportedLow rise: approx. 8 inchesSkinny fit Skinny leg Zipper flywith button closure 5-pocketstyle Dark green wash, coloreddenim Waistband with beltloops Inseam: approx. 30-1/2inchesLorem ipsum dolor sit amet,consectetur adipisicing elit, seddo eiusmod tempor incididuntut labore et dolore magnaaliqua. Ut enim ad minimveniam, quis nostrudexercitation ullamco laboris nisiut aliquip ex ea commodoconsequat. Duis aute iruredolor in reprehenderit involuptate velit esse cillumdolore eu fugiat nulla pariatur.Excepteur sint occaecatcupidatat non proident, sunt inculpa qui officia deserunt mollitanim id est laborum.
  • 8. Plain Text DocumentsElse Jeans Skinny Jeans,Colored Denim Dark Green-WashIn a dark green wash perfectfor fall, these Else Jeans skinnyjeans hit the colored-denimtrend right on the mark! Lyocellcotton rayon polyester LycraMachine washable ImportedLow rise: approx. 8 inchesSkinny fit Skinny leg Zipper flywith button closure 5-pocketstyle Dark green wash, coloreddenim Waistband with beltloops Inseam: approx. 30-1/2inchesLorem ipsum dolor sit amet,consectetur adipisicing elit, seddo eiusmod tempor incididuntut labore et dolore magnaaliqua. Ut enim ad minimveniam, quis nostrudexercitation ullamco laboris nisiut aliquip ex ea commodoconsequat. Duis aute iruredolor in reprehenderit involuptate velit esse cillumdolore eu fugiat nulla pariatur.Excepteur sint occaecatcupidatat non proident, sunt inculpa qui officia deserunt mollitanim id est laborum.approx velit
  • 9. BRAND: Calvin KleinGENDER: WomensTYPE: JeansFIT: At WaistOCCASION: CasualLEG: Classic StraightWEIGHT: Super SkinnyCOLOR: Light WashProduct Catalog Document
  • 10. Polysemy Problem
  • 11. Polysemy ProblemCOLOR:"Pink" TYPE:"Sweater"BRAND: "Tomas Pink" TYPE:"Sweater"BRAND: "Pink Lotus" TYPE:"Sweater"
  • 12. http://www.shopstyle.com/browse/sweaters?fts=pink+sweater&fl=b7&fl=b28462&fl=b1830&fl=b15382&fl=b12130&fl=b737
  • 13. http://www.shopstyle.com/browse/sweaters?fts=pink+sweater&fl=b7&fl=b28462&fl=b1830&fl=b15382&fl=b12130&fl=b737
  • 14. pink jeans by googlehttps://www.google.com/search?hl=ru&tbm=shop&q=pink+sweater&oq=pink+sweater&gs_l=products-cc.3..0l7j0i5l3.4658.7244.0.7699.12.11.0.1.1.0.298.1153.5j4j1.10.0...0.0...1ac.1.denzbOMlNDg#q=pink+sweater&hl=ru&tbm=shop&ei=oDmIUPXgAoqC4gSylYGACg&start=20&sa=N&num=20&
  • 15. pink jeans by googlehttps://www.google.com/search?hl=ru&tbm=shop&q=pink+sweater&oq=pink+sweater&gs_l=products-cc.3..0l7j0i5l3.4658.7244.0.7699.12.11.0.1.1.0.298.1153.5j4j1.10.0...0.0...1ac.1.denzbOMlNDg#q=pink+sweater&hl=ru&tbm=shop&ei=oDmIUPXgAoqC4gSylYGACg&start=20&sa=N&num=20&
  • 16. right pink jeans
  • 17. right pink jeans
  • 18. Polysemy Problem No.2BRAND:"Pink Rose" TYPE:"Sweater"COLOR:"Pink","Rose" TYPE:"Sweater"sweater
  • 19. pink rose jeansproblempink rose sweater
  • 20. precise pink rose jeanspink rose sweater
  • 21. Ranking Cant Help
  • 22. Ranking Cant Help
  • 23. Precisionis a MUST!
  • 24. http://tustinhairsalon.com/wp-content/uploads/2011/02/Blonde_Hair_stylist_stylists_blondes_tustin_santa_ana_orange_county.jpgThink About the User
  • 25. http://www.amazon.com/Silver-Jeans/e/2206256011
  • 26. Wrap-Up Product Descriptions Differs to Plain Text No Query Syntax Polysemy Problem Ranking Cant Help When Precision is LoweCommerce Search is Special!
  • 27. Part IISubphrase Query Parser
  • 28. Dont tokenize at index time
  • 29. black and decker toaster oven"black""black and""black and decker"... "and""and decker""and decker toaster"... "decker""decker toaster""decker toaster oven""toaster oven"
  • 30. black and decker toaster oven"black""black and""black and decker"... "and""and decker""and decker toaster"... "decker""decker toaster""decker toaster oven"..."toaster oven"ID:123BRAND: "black anddecker"TYPE: "toasteroven"COLOR: white
  • 31. black and decker toaster oven"black""black and""black and decker"... "and""and decker""and decker toaster"... "decker""toaster""decker toaster oven"... "oven"ID:456BRAND: "black anddecker"TYPE: "oven"COLOR: white
  • 32. black and decker toaster oven"black""black and""black and decker"... "and""and decker""and decker toaster"... "decker""decker toaster""decker toaster oven"..."toaster oven"ID:789BRAND: "deckertoaster"TYPE: "toasteroven"COLOR: black
  • 33. fee people sweater
  • 34. Part IIIStaged Search
  • 35. Precision and RecallExpected
  • 36. ExpectedActual
  • 37. ExpectedActual OKTruePositiveFalseNegativeFalsePositive
  • 38. Precision =ExpectedActualActual ExpectedActual
  • 39. ExpectedActualRecall =Actual ExpectedExpected
  • 40. ExpectedActual100% Recall
  • 41. ExpectedActualTuning Trade-Off
  • 42. ExpectedActualTuning Trade-Off
  • 43. ExpectedActualTuning Trade-Off
  • 44. ExpectedActualTuning Trade-Off
  • 45. defaultOperatorAND ORnumFound=100 numFound=1000
  • 46. defaultSearchField / queryFieldsBRAND BRANDTYPESTYLEtextnumFound=100 numFound=1000