Multiple Agentsfor

Pattern Recognition

Louis Vuurpijl

http://hwr.nici.kun.nl/~vuurpijl

Contents

• The problem: handwriting recognition

• PR1: The traditional solution– (some) solutions at the NICI

• PR2: Multiple classifiers– (some) solutions at the NICI

• PR3: Could MAPR be a solution?– our current achievements

Sources of variation(Schomaker, Plamondon et al ’00)

Affine transforms

Style Variations

Neuro-biomechanichal

VariationsOrder

Variability

The more writers, the more….

Is this a problem?

• Shrihari ICDAR’01: “No problem”Handwriting is individual, so can be

used in court (as a fingerprint)!

• IWFHRxx, ICDAR, IJDAR... “Yes!”We have 99+ digit recognition, 98+

character recognition and 90+ for isolated words........

Pattern Recognition (I)

Raw data

X(t),Y(t),P(t)

Class labels

Pattern Recognition (I)

Preprocessing

Segmentation

Feature Extraction

Classification

Class labels

Raw data

X(t),Y(t),P(t)

Solutions at the NICI

• More than 20 years of experience in– Handwriting production (Thomassen, van

Galen, Meulenbroek, Maarse, Schomaker, et al)– Handwriting recognition (Schomaker, Teulings,

Vuurpijl, et al)

• Keywords:– Use knowledge about human handwriting– Specialization and......– Fusion

The basis of handwriting

X(t),Y(t),P(t)

Va(t)

UNIPEN data

Zooming in on writing styles

UNIPEN styles: print, cursive, mixed

The lean recognition machine

Clustering on stroke-features (IWFHR’96)

Specialization boosts recognition performance,while reducing computational and memory requirements

Finding structure in diversity (ICDAR’97)

Hierarchy in character shapes

Allograph prototypes

dScript: a MAPR system (’00)

dScript: 9 classifiers

• Neural networks (MLP, Kohonen)

• Nearest neighbour & clustering

• Structural/geometrical

• Support vector machines

• Hidden markov models

Fusion through classifier combination and

Multiple agents (IWFHR’98,’00,’02)

Pattern Recognition (II)

Raw data

Class labels

Classifier 0Classifier 0

Classifier 0Classifier 0

Classifier 0Classifier 0

Classifier 0Classifier 0

Classifier 0Classifier(i)

ClassifierCombination

Classifier combination (van Erp’00,’02)

Changing contexts....

- static architecture

- what if “Go 551” was intended?

Dynamic PR

Through extra heuristic information

Dynamic PRThrough extra features (add PENUP)

Features determinehow you look at data

Dynamic PRKnowing when to use which feature

Dynamic PRKnowing when to use which feature

Dynamic PR

Knowing when to use which feature/algorithm

•Through a knowledge base of PR

•Through a library of PR modules

•Through negotiation protocols

Knowing how to use which PR module

Pattern Recognition (III)

Raw data

Class labels

MAPR

What is an agent?(Wooldridge and Jennings, 1995)

A software system with:

• Goals: ``What do you want?'' or ``What can you do?''– I can solve 0-6 conflicts

• Beliefs and reasoning: ``How do you realize this goal?''– I solve this 0-6 conflict using modules PR1 and PR2– and features #84 and #96, extracted by FE(i) and FE(j)

• Assertions with confidence:– Based on my experience and these features I belief this

input is a ``6'' with confidence 0.9.– I have been correct in 90% of the cases in the past.

Agent framework

MAPR: Our goal

• A distributed intelligent agent framework,

• with PR modules, symbolic equivalents and PR language.

• Driven by problem constraints

• and with learning capabilities.

Our current MAPR system

• Distributed processing over internet using sockets.

• Interfaces to KQML and Jatlite.• Agents know about the environment.• Agents know about the available PR

modules and data.• Agents interact with other agents.• Agents to detect problems and conflicts

A proof of conceptTrained recognition system hclus

– 15557 digits from UNIPEN– 7778 train, 7779 test (95.9%)

1-7 30 5-8 14

7-2 27 1-8 12

5-3 25 0-9 12

1-2 23 8-0 11

1-2 23 8-0 11

4-1 22 4-8 10

4-6 14

1-7, 7-2, 4-6conflictssolved 97%

But......

• This is all hard-wired

if confidence top[0] too low

then solve(top[0],top[1],...)

solve(1,7,2) =

best(1-7,1-2,7-2)

Current research on MAPR

• Knowledge base

• PR language & implementation

• PR negotiation mechanisms

Conclusions

• Online HWR is still unsolved• MCS can improve recognition rates, but.....

– Hard-wired PR modules– Examples where dynamic PR is needed

• MAPR is a new paradigm that exploits knowledge about when to use which features or algorithms

But how to implement shared access to knowledge?

And how to perform agent-like negotiations ?