Upload
sophie
View
62
Download
1
Tags:
Embed Size (px)
DESCRIPTION
August 20th, 2008 ICFHR, Montreal. Ralph Niels , Don Willems and Louis Vuurpijl. Introducing of handwritten icons. Introducing the NicIcon database of handwritten icons. Ralph Niels Don Willems Louis Vuurpijl. NicI. Icon. Introducing the NicIcon database of handwritten icons. - PowerPoint PPT Presentation
Citation preview
Introducing
of handwritten icons
Ralph Niels, Don Willems and Louis Vuurpijl
August 20th, 2008ICFHR, Montreal
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Icon
NicI
• Introduction• Domain: crisis management
• Data collection• Icon design• Method• Data
• Classification experiment• Method• Results
• Conclusion and URL
Overview
=
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
NEW!
Crisis management (CM)• Scenario: tunnel disaster• Distributed computer systems to support CM• Multiple modalities: speech, gestures, and pen
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
The car is here
Iconic pen gestures• Faster than handwriting• Easy to learn & remember• Visual meaningful shape
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Database• Public databases available for several handwriting
applications• Not for iconic pen gestures
• Based on symbology reference by USA Homeland Security Workgroup– Used in e.g., USA, Australia, New-Zealand
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Icons
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Flood Accident Car Bomb Roadblock
Electricity Casualty Fire Fire brigade Police
Injury Gas Paramedics Person
Data collection• 35 participants• Online and offline• Variation in size• Per person:
– 22 pages– 55 instances / icon
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Database content• Online: 26,163 iconic gestures
(24,144 reported in paper)
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Tablet: Wacom Intuos2 A4 oversize
Database content• Offline: 770 scanned pages
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Scanner: HP Scanjet 7400C flatbed(@ 300dpi, 24 bit colour)
Classification experimentData sub sets• Stratified
• Writer dependent (WD), writer independent (WI)
Complete data set(26,163 icons)
Evaluation set(40%)
Train set(36%)
Test set (24%)
WDEvaluation set
(40%)
WDTrain set
(36%)
WDTest set (24%)
WIEvaluation set
(40%)
WITrain set
(36%)
WITest set (24%)
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Classification experimentMulti classifier system• Feature sets
• 28 geometric features (Willems & Vuurpijl)• 30/60 coordinates running features (Schomaker & Vuurpijl)• 1185 features (Willems, Niels, Van Gerven, Vuurpijl)
• Feature classifiers• Support Vector Machine• Multi-Layered Perceptron
• Template matching• Dynamic Time Warping (Niels & Vuurpijl)
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
NEW!
on online data
New feature set (‘m-fs’)• Features from literature
– Geometrical, temporal, pressure
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
D. Rubine, Specifying gestures by example, Computer Graphics. 25 (4) (1991) 329–337.
J. LaViola Jr. & R. Zeleznik, A practical approach for writer-dependent symbol recognition using a writer-independent symbol recognizer, IEEE Transactions on pattern analysis and machine intelligence. 29 (11) (2007) 1917–1926.
L. Zhang & Z. Sun, An experimental comparison of machine learning for adaptive sketch recognition, Applied Mathematics and Computation. 185 (2) (2007) 1138–1148.
and many others…
New feature set (‘m-fs’)• Over complete icon, but also…
– Mean over strokes– Std. dev. over strokes
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Feature definitions in technical report at website
New feature set (‘m-fs’)• Feature selection
– 1185 features, which are the best?– Sort them on best individual performance– Add 1 by 1 to classifier
• Performance maximizes at:– 545 features for WI– 660 features for WD
• Selected features:– ± 1/3 full icon– ± 1/3 mean– ± 1/3 standard deviation
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
The best features
Breaking news: the best features
Area Sine first /last sample
Length ofdiagonal
Verticaloffsets
Averagecentroidal
radius
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Classification resultsClassifier Features WD WISVM g-28 84.8 73.0
m-fs 99.3 96.4af-30 98.1 90.3
af-60 98.0 98.6
MLP g-28 84.0 78.5
m-fs 98.7 96.3af-30 94.5 85.2
af-60 94.4 85.1
DTW - 98.4 93.6
MCS - 99.51 97.83
g-28: 28 geometric features(Willems & Vuurpijl)
m-fs: 1185 features(Willems, Niels, Van Gerven, Vuurpijl)
af-30/af-60: 30/60 running features (Schomaker & Vuurpijl)
DTW: Dynamic Time Warping(Niels & Vuurpijl)
MCS: Multiple classifier system (majority voting)
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
NEW!
Misclassifications• Wrong box
• Sloppy drawing
• Retracing
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Discussion• Results already quite good (WD: 99.5%, WI: 97.8%),
but gain is still possible• Which features are important?
– For different domains• Performance in interactive experiment• Offline data still open• Mapping online -> offline
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
http://unipen.nici.ru.nl/NicIcon
Introducing the NicIcon databaseof handwritten icons
Ralph NielsDon Willems
Louis Vuurpijl
Freely available:• Online (Unipen)• Offline (PNG)• Technical report about
new feature set