Site Search Analytics

Site Search Analy.cs

Search Quality Ma/ers

Copyright 2014 Sematext Group Inc. - All rights reserved.!! 1!

About me Stefan Thies [email protected]

Twi:er: @se.321

DevOps Evangelist @sematext


About Sematext

!3!Copyright 2014 Sematext Group Inc. - All rights reserved. !

Performance monitoring, !anomaly detection and alerting!

Log management!

Site Search Analytics!

Consulting & Support !for Solr and Elasticsearch!

Implemen.ng Big Data & Search Systems

!4!

Copyright 2014 Sematext Group Inc. - All rights reserved.!!

Know-‐How

Tools

Agenda

l  What is Search Analy.cs and why it ma:ers

l  Example reports and their value


Why


search users

search providers

search experience

!This search sucks!!

It takes 17 tries to find anything here!!F!?@#$%^&?!?!

!

!Cool, the latest search tweaks !

made our site really sticky!!Awesome!!

!

Fill in the Missing Piece


Performance Monitoring

Quality Assurance

Tuning UI!

Blind Leading the Blind


Analy.cs as Compass

Search logs are your Map

Search Analy2cs is your Compass


The Bo:om Line Why

l  Measure and monitor everything. l  Supports (re)design, naviga.on choices l  Helps with content acquisi.on & enhancement

l  Improve search experience


The Moment of Truth

•  Ques.on for the audience #1

What do you use for Search Analy.cs? a) Home grown stuff b) Google Analy.cs c) Other d) Nothing


Search Analy.cs Basics

l  Collect: queries & clicks & interac.ons & ...

l  Analyze: ac.ons / conversions l  Output: reports – over .me l  Output++: feedback loop l  The means, not the goal l  Ongoing, not one-‐off


remember this

Search vs. Web Analy.cs

l  User intent and informa.on needs vs. inferring l  Hand in hand l  Ideally you can relate data from both or even unify it


Report Types

l  Failures vs. non-‐failures l  Ac.onable vs. non-‐ac.onable


Failures vs. Non-‐Failures

l  Zero hits l  Low CTR l  Low MRR l  High bounce rate l  Low conversion rate l  Deep paging l  Deep clicking l  High latency


l  Query rate

l  Query volume

l  Top seen & clicked docs

l  Top queries

l  Terms per query

l  Search sessions

l  Search users

l  Distinct queries

Value of Failure Fixes

l  Zero hits l  Low CTR l  Low MRR l  High bounce rate l  Low conversion rate

l  Deep paging l  Deep clicking l  High latency Performance Tuning!

Copyright 2014 Sematext Group Inc. - All rights reserved. !

Re-search

Findability Relevance Tuning

Measure, then Fix

If you can't measure, it you can't fix it!


Metrics – e.g. Mean Reciprocal Rank

18!Copyright 2014 Sematext Group Inc. - All rights reserved.!!

Wikipedia - MRR!

Relevance A/B Tes.ng


Tracking Zero Hits


Report: Zero Hit Queries

•  Overall pct. (not raw count) vs. Popular queries

•  Misspellings? •  Synonyms? •  No matching content? •  Need different tagging? •  Bad analysis/mapping? •  Mul.lingual issues?

!21!


Tracking Zero Hits


Frequency of Zero Hit


Minimize: # of Zero Hit Queries

•  Use Query Spellchecker (DYM) •  Use Autocomplete •  Using DYM ReSearcher, Query Relaxer, Related Search

•  Check Mapping, Analyzers, Queries •  Indexing process

–  Lost items while bulk indexing? •  Design a ‚No Result‘ page

!24!


!25!


Ellastic search!

Example (search-‐hadoop.com)

Watching Latency


Paging


more search metrics ...


Search Analy.cs & Measuring


If you can't measure it, you can't fix it!

You can't measure it if you don't have Analytics

Ac.onable vs. Non-‐Ac.onable

l  Zero hits l  Low CTR l  Low MRR l  High bounce rate l  Low conversion rate l  Deep paging l  Deep clicking l  High latency


l  Query rate l  Query volume l  Top seen & clicked docs

l  Top queries l  Terms per query l  Search sessions

l  Search users l  Distinct queries

More Fixin' l  Query rate l  Query volume l  Search sessions l  Search users l  Top seen & clicked docs

l  Top queries l  Terms per query l  Dis.nct queries


Navigation & Design Results Shuffling Diversification, Recommendations

AutoComplete!

Search box size!

Output++: Data is Power

l  AutoComplete -‐ $MM improvement l  Be:er DYM Spellchecker l  Related Searches l  Recommenda.ons l  Relevance Feedback l  ...


Closing the Loop

!

33!Copyright 2014 Sematext Group Inc. - All rights reserved. !

search users

search providers

search experience

Resources


http://rosenfeldmedia.com/books/searchanalytics/

Search Analytics for Your Site Louis Rosenfeld

l  http://blog.sematext.com/tag/analytics/

l  http://www.slideshare.net/lucenerevolution/gospodnetic-otis-ppt-search-analytics-what-why-how

Key Take-‐aways

l  Without Analy5cs you are blind

l  If you can't measure it, you can't fix it

l  Use Search Analy5cs to understand, measure and improve search

l  Using Search Analy5cs means having a compe55ve advantage

!

35!Copyright 2014 Sematext Group Inc. - All rights reserved. !

Dig Search?

Dig Analy.cs?

Dig Big Data?

Dig Performance?

Dig Logging?

Dig working with open – source?

•  We‘re hiring planet -‐ wide!

h/p://www.sematext.com/about/jobs.html

We are hiring ...!


Documents

Site Search Analytics