Datech2014 - Session 3 - Correcting Noisy OCR: Context Beats Confsusion

Automatic OCR correction http://overproof.projectcomputing.com

Correcting noisy OCR

- Context beats Confusion

[ presentation viewable at http://goo.gl/n85gR6 ]

http://overproof.projectcomputing.com/



who are we?

● Australian software company

● developers John and Kent

● we put theory into practice




● the first draft of history

● popular if made available

● usually poorly digitized

● too extensive for full human

correction

main target - newspapers




goals

● run on commodity cloud server

● optimal for noisy text

● at least 1000 words/sec

● correct at least 50% of errors




division of labour

bad

good

models

models

MANAGER,

TRIAGE

CORE




snippets for the core

● prefer triaged good words at start/end

● column aware

● some easy corrections applied

● some suggestions supplied

● bag of topic words available

● surrounding noise level indicated




error contexts

● spell: vowals or consonnants

● type: you jit teh wrng key

● OCR: roprcroiitativcs cf thc Coveriuient

● random: anygh<eg 0at7happen




confusion cost matrix

93: w ← w 155: e ← e 3750: c ← e 4451: m ← rn 6652: rn ← m 11065: E ← m




word cost (eg rnorniny|morning)

language cost ● lexicon frequency

● entity list

● rare word list

● character 4-gram

error cost ● edit sum

● visual correlation

● generator hint




word character confusion

m o r n i n g

r n o r n i n y




visual correlation




suggestion methods

● gift

● common, cached

● language

● entities

● split/join

● generated (magic)




searching for gold (A*)

l i

i

n e

r

h

hcii h li b n ... c e r o … i i 1 l n u … i i 1 l ...

purple nodes: working priority queue

red nodes: output priority queue




amazing generated suggestions

Parhumuitar} ← Parliamentary I.iulwuvB ← Railways Itegtniont ← Regiment niltfltory ← adultery uj.rccu.eut ← agreement couniutfc.o ← committee cnuipuii ← company dctoimiuatJOu ← determination uiidcrtkikcr’a ← undertaker’s




selecting best combination

unsiejitlv unsightly unseemly unsettle unsteady Unsightly urgently

bohavlour behaviour behavour behavior Behaviour behaviours behaving

abonf about above along been

am am an a in as

unsiejitlv unsightly unseemly unsettle unsteady Unsightly urgently

disgrie disgrace disagree disguise desire degree disease

[NOTE: word joins and splits are also supported]




training

● 5-grams - subset selection

● corpus 1,2,3-grams - statistical build

● extra word lists - easy

● error model - bootstrap or new pairs




testing

● 65000 words ground truth including

foreign (US) newspapers

● all measures exceeded goal:

○ search errors (article word types)

○ read errors (article word tokens)

○ entropy weighted term errors




Before After

Recall 83.8% 94.1% recall misses reduced 63.3%

Raw Error Rate 18.5% 5.5% errors reduced 70.1%

Weighted Error Rate 16.2% 6.7% weighted errors reduced 59.4%

SMH sample




¿preguntas?

Presentation viewable at http://goo.gl/n85gR6







National Library of Australia’s

TROVE

● 1.4m distinct visitors/month

● 16m pageviews/month

● 80% of usage is old newspapers o 13m pages, over 600 titles

o 85k lines corrected/day




Even this massive volunteer effort

cannot keep up

● < 2% of errors have been corrected

● % corrected is declining

● Hence searching is unreliable, OCR’ed text

is hard to read and reuse

● Trove’s accuracy is “typical”







159 randomly selected news

articles from The Sydney

Morning Herald

47.4K words hand-corrected to ground truth




Before After


False positive recall 26.7% 9.1% false positives reduced 65.8%



SMH sample
















49 randomly selected news

articles from LoC

Chronicling America

18.1K words hand-corrected to ground truth




Before After


False positive recall 23.6% 8.8% false positives reduced 62.8%



LOC sample



Technology

Datech2014 - Session 3 - Correcting Noisy OCR: Context Beats Confsusion