1
Querying Graphics through Analysis and Recognition
INRIA Lorraine
2
Research fields
•Image processing and segmentation
•Structural pattern recognition
•Statistical pattern recognition
•Information spotting and retrieval
In the context of the analysis and recognition of graphics-rich documents
3
Querying Graphics through Analysis and Recognition
4
Querying Graphics through Analysis and Recognition
5
Querying Graphics through Analysis and Recognition
6
Querying Graphics through Analysis and Recognition
7Scientific staff
Suzanne Collin, Assist. Prof. UHP
Philippe Dosch, Assist. Prof. Nancy 2
Bart Lamiroy, Assist. Prof. INPL/Mines
Gérald Masini, CR CNRS
Salvatore Tabbone, Assist. Prof. U. Nancy 2
Karl Tombre, Prof. INPL/Mines
Laurent Wendling, Assist. Prof. UHP
PhD students
Sabine Barrat, CIFRE contract (pending)
Thi Oanh Nguyen, joint supervision with IFI (Hanoi, Vietnam)
Oriol Ramos Terrades, joint supervision with UAB, Barcelona (Spain)
Jan Rendek, CIFRE France Télécom
Jean-Pierre Salmon, FRESH (European project)
Zhang Wan, joint supervision with City U. Hong Kong (pending)
Daniel Zuwala, MESR grant
Technical staff
Yamina Smail, Epeires project
X, Fresh project (pending)
Administrative staff
Isabelle Herlich (part time)
Françoise Laurent (part time)
8
Main results 2004-05Hierarchical binarization
9
Focus on symbol recognition – Symbol spotting combining Radon-based signature and structural approach
Main results 2004-05
10
Improvement of recognition rates through combination of shape descriptors
Main results 2004-05
The set of images I
Recognition rates by descriptors
010
2030
405060
7080
90100
1 2 3 4 5 6 7 8 9
Clusters
Re
co
gn
itio
n r
ate
s
Compactness
Ellipticity Degree
Angular Signature
Generic FourierDescriptors
R-signature
Weighted Sumwithout weighted map
11
Application : extraction of letters in heritage documents
Descriptors C E SA GFD TRf WS
Before 49 41 70 59 50 55
After 52 39 75 69 50 72Recognition rates
Descriptors C E SA GFD TRf WS
Before 85 78 90 89 80 96
After 93 81 97 94 87 100
Ranking
Improvement of recognition rates through combination of shape descriptors
Main results 2004-05
12
Main results 2004-05Raster-to-vector conversion method based on random sampling and parametric fitting
13
Segmenting the skeleton
RANVEC : Random sampling on pairs of vector pointsExtension of primitive as long as it fits arc or segment (linear regression)
14
Simplification and unification of primitives
15
GREC’01 GREC’03 GREC’05
Winner Dave Elliman JiQiang Song Xavier Hilaire
VRI 0,681 0,609 0,803
Arc segmentation contest
16
Application domains/transferElectrical wiring diagrams in aeronautics FRESH project (FP6 STREP Aeronautics program)
17
Application domains/transfer
Cultural heritage documents ACI Madonne, FP6 STREP proposal QUIMERA-Doc submitted 9/05
18QgarLib : library of C++ classes
QgarApps : applications
QgarGUI : user interface
qgar.org, APP
Refactoring to professional standards
Open architecture (XML)
80,000 lines of C++ code (comments not counted)
30 to 40 downloads of code per month
>10 documentation browses per day (robots excluded)
19
Positioning within INRIA
Fully within one of INRIA’s 7 challenges in strategic plan: Developing multimedia data and information processing
Regular partnership with Imadoc (research group at Irisa)
Joint contacts Texmex (Sym-C)/Qgar with industrial partner
Recent contacts with Lear on browsing of large image bases
20
Collaborations
National: informal consortium Nancy, Rennes, La Rochelle, Rouen, Tours, Lyon with several joint projects (ACI Madonne, RNTL past and submission, Techno-Vision Epeires, IST submission) and coordination of actions
CVC/UAB, Barcelona: long lasting relationship, associated team SymbolRec, joint PhD supervisions
City University Hong Kong: associated in Epeires, PAI submission accepted, joint PhD supervision
IFI, Vietnam: joint PhD supervision
University of Auckland (NZ), University of Bern, Carleton University (Canada)
21
Achievements, strengths, weaknesses
Leadership position at international level on graphics recognition
Announced in project and largely addressed:• Symbol recognition and spotting
• Performance evaluation
Strong and adequate applicative backing
Improvement in number of PhD students
Still low on permanent workforce
22
Future work
Scalability of symbol recognition methods• Large number of models
• Variations within the same shape class
– Combining structural and statistical methods
– Hierarchical approach
23
Future workComplex symbols
24
Future workDynamic, on-the fly recognition and spotting: from model-based recognition to freehand recognition
25
Future work
Multi-modal indexation (text / graphics / image / video) in multimedia and document databases (collaborations with Texmex, Lear, …)
Interactivity with user (relevance feedback)
26
Future work
Performance evaluation• International symbol recognition contests 2003 & 2005
• Epeires
– French Techno-Vision program
– 4 universities, FT R&D, 1 company + foreign partners UAB & CityU
– www.epeires.org
• Future research challenges
– Simple and non-biased metrics
– Ground-truth/recognition output matching methods
– Generation of large sets of training and benchmarking data using realistic image degradation models
27Epeires – ground-truthing
28
Future work
Software : increase number of applications