K. Zagoris, K. Ergina and N. Papamarkos

Developing a Document Image Retrieval System

Image Processing and Multimedia LaboratoryDepartment of Electrical & Computer

EngineeringDemocritus University of Thrace,

67100 Xanthi, Greece

2Democritus Universiy of Thrace, Greece

There is a huge information stored in the form of document images

These documents do not have any indexing information

How we can retrieve a such document using a quary a single word contained in the document

The Document Retrieval Problem

3Democritus Universiy of Thrace, Greece

The overall structure of the Document Image Retrieval System

4Democritus Universiy of Thrace, Greece

Binarization (Otsu Technique)

Original Document

Preprocessing stage

Median Filter

5

Indentify all the Connected Components (CCs)

Calculate the most common height of the document CCs (CCch)

Reject the CCs with height less than 70% of the CCch. That only reject areas of punctuation points and noise.

Expand the left and right sides of the resulted CCs by 20% of the CCch

The words are the merged overlapping CCs

Using the Connected

Components Labeling

and Filtering method

Word Segmentation

Democritus Universiy of Thrace, Greece

6

Width to Height RatioWord Area Density. The percentage of the

black pixels included in the word-bounding box

Center of Gravity. The Euclidean distance from the word’s center of gravity to the upper left corner of the bounding box:

Features

(1,0) (0,1)

(0,0) (0,0)

,x y

M MC C

M M

( , )qp

pqx y

x yM f x y

width height

Democritus Universiy of Thrace,

Greece

7Democritus Universiy of Thrace, Greece

Vertical Projection. The first twenty (20) coefficients of the Discrete Cosine Transform (DCT) of the smoothed and normalized vertical projection.

Features

Original Image

The Vertical Projection

Smoothed and normalized

8Democritus Universiy of Thrace, Greece

Top – Bottom Shape Projections. A vector of 50 elementsThe first 25 values are the first 25 coefficients of the

smoothed and normalized Top Shape Projection DCT The rest 25 values are equal to the first 25 coefficients of the

smoothed and normalized Bottom Shape Projection DCT.

Features

9Democritus Universiy of Thrace, Greece

Upper Grid Features is a ten element vector with binary values which are extracted from the upper part of each word image.

Down Grid Features is a ten element vector with binary values which are extracted from the lower part of the word image.

Features

10

Upper – Down Grid Features

[0,0,0,1195 ,0,0,0,0,0,0][0,0,0,1 ,0,0,0,0,0,

0]

[0,0,0,0 ,0,0,0, 598 , 50 , 33 ]

[0,0,0,0 ,0,0,0,1,1,0]Democritus Universiy of Thrace, Greece

11

Descriptor

The Structure of

the Descriptor

Democritus Universiy of Thrace, Greece

12Democritus Universiy of Thrace, Greece

User enters a query word The proposed system creates an image of the

query word with font height equal to the average height of all the word-boxes obtained through the Word Segmentation stage of the Offline operation.

For our experimental set the average height is 50

The font type of the query image is ArialThe smoothing and normalizing of the various

features described before, suppress small differences between various types of fonts

Query Image Creation

13Democritus Universiy of Thrace, Greece

The Matching Process

14Democritus Universiy of Thrace, Greece

100 image documents created artificially from various texts

Then Gaussian and “Salt and Pepper” noise was added

Implement in parallel a text search engine which makes easier the verification and evaluation of the search results of the proposed system

Experimental Document Database

15Democritus Universiy of Thrace, Greece

Implementation

o Visual Studio

2008

o Microsoft .NET

Framework 2.0

o C# Language

o Microsoft SQL

Server 2005

http://orpheus.ee.duth.gr/irs2_5/

16Democritus Universiy of Thrace, Greece

Evaluation

o Precision and

the Recall metrics

o 30 searches in

100 document

images

o Font Query:

Arial

Mean Precision: 87.8% Mean Recall: 99.26%

17Democritus Universiy of Thrace, Greece

FineReader® 9.0 OCR Program

EvaluationQuery Font Name “Tahoma”.

Mean Precision: 76.67% Mean Recall: 58.42%

Mean Precision: 89.44% Mean Recall: 88.05%

18Democritus Universiy of Thrace, Greece

The query word is given in text and then transformed to word image

The proposed system extract nine (9) powerful features for the description of the word images

These features describe satisfactorily the shape of the words while at the same moment they suppress small differences due to noise, size and type of fonts

Based on our experiments the proposed system performs better in the same database than a commercial OCR package

Conclusion

Democritus Universiy of Thrace, Greece 19

Thank you!