Making Bible Search Results Relevant (BibleTech 2011)

Making Bible Search Results Relevant

Stephen SmithBible Gateway

March 25, 2011

Agenda

1. How people search Bible Gateway1. Four types of searches2. Seven ways people find verses

2. How people browse Bible Gateway1. Verse popularity2. Parsing queries3. Different translations

Source Data

• Bible Gateway logs, May-November 2010• 160 GB• Processed mostly with Perl• Amazon EC2

Search

Long Tail of Searches

Longer tail queries (log)

Num

ber o

f que

ries (

log)

Percentage of Misspellings

Longer tail queries

SermanPerserveranceVengenceJezabelProdical son

Query Types Change Down the Tail

Longer tail queries

1. Quote

Phil 4:13 Search Quotes

I can do all things through Christ who

strengthens me.

Seven Verse Search StrategiesType Example Percent

Exact word / phrase I can do all things 80

Similar verse With God, all things are possible 9

Partial word / misspelling Christ who strengthen me 6

Concept Not alone 3

Keywords Gives strength 2

Minor switch You can do all things 0.6

Paraphrase All things through God 0.5

2. Concept

BG Path Analysis for [strength]Reference Text %

Exod 15:2 The LORD is my strength and my defense… 2.17

Isa 40:31 …will renew their strength. They will soar on wings like eagles… 1.27

Eph 3:16 I pray that out of his glorious riches he may strengthen you… 1.26

Ps 28:7 The LORD is my strength and my shield… 1.22

Phil 4:13 I can do all this through him who gives me strength. 1.04

Reference Text %

Phil 4:13 I can do all this through him who gives me strength. 17.5

Deut 31:6 Be strong and courageous… 11.7

Deut 20:4 The LORD your God is the one who goes with you to fight for you…

10.4

1 Cor 10:13 No temptation has overtaken you except what is common… 5.38

Eccl 4:12 A cord of three strands is not quickly broken. 3.15

Topical Bible Entries for [strength]

3. Named Entity

Types of Named Entities

Other

GroupJesus

God

Place

Person

4. Story

Types of Story Queries

Synecdoche

Section

Longer tail queries

% o

f Que

ry T

ype

Improving Story-Level Results

For sections, use existing headings to improve recall from 48% to 63%

Path analysis works well for synecdoche

Search Summary

1. Quote: exact word / phrase, similar verse, partial word / misspelling, concept, keyword, paraphrase

2. Concept: most-popular queries; use human intelligence

3. Named Entity: most commonly people4. Story: use metadata

Browse

Old vs. New Testament Popularity

Popularity

Physical Length

Book PopularityM

ore

popu

lar (

log)

Chapter PopularityM

ore

popu

lar (

log)

Verse PopularityM

ore

popu

lar (

log)

Verse Popularity DistributionM

ore

popu

lar (

log)

John 3:16

Num 7:51: “one young bull, one ram and one male lamb a year old for a burnt offering”

Acts 27:6: “There the centurion found an Alexandrian ship sailing for Italy and put us on board.”

Passage Types

71% passage

29% search

Single verse 21%

Sub-chapter range 21%

Chapter 53%

Super-chapter range 2%Multiple passages 3%

Query ComponentsType Example

Book Genesis

Book Chapter Verse Genesis 1:2

Book Chapter Genesis 1

Book Verse Philemon 7

Chapter Verse 1:2

Chapter Ch 1

Verse Verse 2

Integer 3

Translation KJV

And following… Genesis 1:2ff.

Range Genesis 1:2-3

Sequence Genesis 1, 4

Ways to Spell “Genesis”

1. Genesis2. Gen3. Genisis4. Gensis5. Genises6. Genisus7. Gene8. Genis9. Ge10. Geneses11. Geneis12. Genes

13. Genasis14. Genesus15. Genese16. Genesi17. Genius18. Genisi19. Genn20. Gennises21. Genise22. Genessis23. Genezis24. Genenis25. Geness26. Geni27. Gn28. Genesia29. Genesses

…60 more

Geonosis

Geonosis

Corner CasesQuery Problem Best Practice

Genesis 1:5-3 Reversed range Gen.1.5,Gen.1.3 + error

Genesis 49-51 End of range too high Gen.49-Gen.50

Psalm 121-22 Omitting digit Ps.121-Ps.122

1 John 1:2-3 John 2 Book name conflict 1John.1.2-3John.1.2

Philemon 1 Chapter-verse conflict Phlm.1

Philemon 1-2 Chapter-verse range conflict Phlm.1.1-Phlm.1.2

Matt 5,6 Localization Matt.5,Matt.6

Gen 315 Ambiguous reference Gen.3.15

Ma 2 Ambiguous book Matt.2

Ambiguous References

50%

Gen 315 = Gen 3:15 or Gen 31:5?97% of the time, it’s Gen 3:15

Ambiguous Book Names

Ma 1:1 = Matthew 1:1, Mark 1:1, or Malachi 1:1?

50%

Translations

Traditional vs. Contemporary Language in English Bible Translations

% p

agev

iew

s

Section

Section Popularity Varies by Translation Type

Summary

1. Four types of searches.2. Concepts dominate most-popular queries;

quotes dominate the long tail.3. Seven search strategies to find verses.4. Optimize for the New Testament.5. Optimize for passage queries, especially

contiguous passages.6. Accept a variety of book names.

stephen.smith@biblegateway.com