Upload
alberto-acevedo-aliaga
View
223
Download
0
Embed Size (px)
Citation preview
7/30/2019 Xapian Motor Inferencia
1/14
Introducing Xapian
Justin Finkelstein | @ilithiumPHP London, November 2011
7/30/2019 Xapian Motor Inferencia
2/14
Background and Alternatives
ReportBuyer.com235,000 reports
1.3 GB of text
Hierarchical categories
MySQL FullText
Search alternatives:
SphinxLucene, etc
Justin Finkelstein | @ilithiumPHP London, November 2011
7/30/2019 Xapian Motor Inferencia
3/14
Benefits
Justin Finkelstein | @ilithiumPHP London, November 2011
Easy to install and portableFast searching
Accurate
Powerful
7/30/2019 Xapian Motor Inferencia
4/14
Drawbacks
Justin Finkelstein | @ilithiumPHP London, November 2011
Not a databaseSingle-writer, many reader
Limited to 4.2 billion documents
OS file size limit
7/30/2019 Xapian Motor Inferencia
5/14
Installation
Justin Finkelstein | @ilithiumPHP London, November 2011
Binaries for WindowsVendor packages & PPA
Source code
Bindings
PHP
C#
Java
LuaPerl
Python, etc
7/30/2019 Xapian Motor Inferencia
6/14
Indexing
Justin Finkelstein | @ilithiumPHP London, November 2011
DatabasesDocuments
Document IDs must be unique
Terms & Stemmers
Term Generator
Values
7/30/2019 Xapian Motor Inferencia
7/14
Querying the Database
Justin Finkelstein | @ilithiumPHP London, November 2011
Simple QueriesPhrases: php development
Logical operators: OR, AND, NOT, MAYBE
Ranges: alpha..omega
NEAR: shop NEAR pubWildcards (report*)
Synonyms
Query Parser make it easy
data management AND NOT real estate AND NEAR data
7/30/2019 Xapian Motor Inferencia
8/14
Relevance and Sorting
Justin Finkelstein | @ilithiumPHP London, November 2011
BM25 Probabilistic Relevancy
Sort by rank/relevance
Sort by values
7/30/2019 Xapian Motor Inferencia
9/14
Getting Started
Justin Finkelstein | @ilithiumPHP London, November 2011
Know your data setWhat are users looking for
How will they refine their search
7/30/2019 Xapian Motor Inferencia
10/14
Report Buyer Product Data
Justin Finkelstein | @ilithiumPHP London, November 2011
item_guidtitle
subtitle
summary
table of contents
price
category
publication dateavailability
product url
7/30/2019 Xapian Motor Inferencia
11/14
Searching on Report Buyer
Justin Finkelstein | @ilithiumPHP London, November 2011
Search by:Product code
Category
Title
Price
Search text of:
Title
SubtitleSummary
Table of Contents
Refine by:Price
Availability
7/30/2019 Xapian Motor Inferencia
12/14
Mapping to Xapian
Justin FinkelsteinPHP London, November 2011
Full text with weighting:name
subtitle
summary
table of contents
Text with prefixes:
titleproduct code
category
Values:price
availability
publication date
Facets:
Category
Availability
7/30/2019 Xapian Motor Inferencia
13/14
Demo Walk-throughs
Justin Finkelstein | @ilithiumPHP London, November 2011
Indexing the data
Query parser
Sorting
MatchSpies
7/30/2019 Xapian Motor Inferencia
14/14
The End
Justin Finkelstein | @ilithiumPHP London, November 2011
http://readthedocs.org/docs/getting-started-with-xapian/
www.redwiredesign.com
blog.ilithium.com