240-572: Chapter 1: Introduction
1
Montri [email protected]://fivedots.coe.psu.ac.th/~montri
240-650Principles of Pattern
Recognition
240-572: Chapter 1: Introduction
2
Chapter 1
Introduction
240-572: Chapter 1: Introduction
3
Outline
• Pattern Recognition System• The Design Cycle• Learning and Adaptation
• Read Chapter 1 (Duda, Hart, and Stork)
240-572: Chapter 1: Introduction
4
Motivations
• Pattern recognition has many very valuable civil as well as military applications– Automated target recognition– Automated Processing Systems– New Human Computer Interface– Biometrics
240-572: Chapter 1: Introduction
5
Handwritten Address Interpretation System
• HWAI - http://www.cedar.buffalo.edu/HWAI/– The HWAI (Handwritten Address Interpretation) System
was developed at Center of Excellence for Document Analysis and Recognition (CEDAR) at University at Buffalo, The State University of New York. It resulted from many years of research at CEDAR on the problems of Address Block location, Handwritten Digit/Character/Word Recognition, Database Compression, Information Retrieval, Real-Time Image Processing, and Loosely-Coupled Multiprocessing.
– The following presentation is based on the demonstration pages at HWAI
240-572: Chapter 1: Introduction
6
Handwritten Address Interpretation System – cont.
• Step 1: Digitization
240-572: Chapter 1: Introduction
7
Handwritten Address Interpretation System – Cont.
• Step 2: Address Block Location
240-572: Chapter 1: Introduction
8
Handwritten Address Interpretation System – Cont.
• Step 3: Address Extraction
240-572: Chapter 1: Introduction
9
Handwritten Address Interpretation System – Cont.
• Step 4: Binarization
240-572: Chapter 1: Introduction
10
Handwritten Address Interpretation System – Cont.
• Step 5: Line Separation
240-572: Chapter 1: Introduction
11
Handwritten Address Interpretation System – Cont.
• Step 6: Address Parsing
240-572: Chapter 1: Introduction
12
Handwritten Address Interpretation System – Cont.
• Step 7: Recognition– (a) State Abbreviation Recognition
240-572: Chapter 1: Introduction
13
Handwritten Address Interpretation System – Cont.
• Step 7: Recognition– (b) ZIP Code Recognition
240-572: Chapter 1: Introduction
14
Handwritten Address Interpretation System – Cont.
• Step 7: Recognition– (c) Street Number Recognition
240-572: Chapter 1: Introduction
15
Handwritten Address Interpretation System – Cont.
• Step 8: Street Name Recognition
240-572: Chapter 1: Introduction
16
Handwritten Address Interpretation System – Cont.
• Step 9: Delivery Point Codes
240-572: Chapter 1: Introduction
17
Handwritten Address Interpretation System – Cont.
• Step 10: Bar coding
240-572: Chapter 1: Introduction
18
IBM Voice Systems
• Voice enabling e-bussinesshttp://www-4.ibm.com/software/speech/enterprise/dcenter/demo_0.html
– Get information through speech recognition software ViaVoice
240-572: Chapter 1: Introduction
19
Machine Demonstrates Superhuman Speech Recognition Abilities
• Developed by Jim-Shih Liaw and Theodore W. Berger at University of Southern California
• The following is the claim– “University of Southern California
biomedical engineers have created the world's first machine system that can recognize spoken words better than humans can. A fundamental rethinking of a long-underperforming computer architecture led to their achievement”.
240-572: Chapter 1: Introduction
20
240-572: Chapter 1: Introduction
21
Statistical Pattern Recognition
• In statistical pattern recognition, recognition is done by classifying the input (represented as a set of measurements) into predefined categories
• The core questions we want to address– What is the best we can do (statistically) when
a set of measurements is given for input?– Which measurements should be used if we can
choose a subset of all the measurements?
240-572: Chapter 1: Introduction
22
A Simple Example
• Suppose that we are given two classes 1 and 2
– P(1) = 0.7
– P(2) = 0.3
– No measurement is given• Guessing
– What shall we do to recognize a given input?– What is the best we can do statistically? Why?
240-572: Chapter 1: Introduction
23
An Introductory Example
240-572: Chapter 1: Introduction
24
Terminology
• Features– Measurements available to the pattern
recognition system
• Models– Each class is represented by a description in
mathematical forms, called a model
• Preprocessing– Segmentation
• Isolate the object of interest from the background and other objects
240-572: Chapter 1: Introduction
25
Terminology - cont.
• Feature extraction– Is the measuring process that produces the
measurements, or called features
• Training samples– Models for classes are often specified by
samples with known labels. These samples are called training samples
240-572: Chapter 1: Introduction
26
Terminology - cont.
• Cost/risk– The cost of a decision associated with the
recognition result
• Decision theory– The theory on optimal decision rules
240-572: Chapter 1: Introduction
27
Terminology - cont.• Decision boundary
– Boundaries in the feature space of regions with different classes (decisions)
240-572: Chapter 1: Introduction
28
Terminology - cont.
• Generalization– While classes can be specified by training
samples with known labels, the goal of a recognition system is to recognize novel inputs
– When a recognition system is over-fitted to training samples, it may give bad performance for typical inputs
240-572: Chapter 1: Introduction
29
Terminology - cont.
• Generalization - continued
240-572: Chapter 1: Introduction
30
Terminology - cont.
• Generalization - continued
240-572: Chapter 1: Introduction
31
Terminology - cont.
• Generalization - continued
240-572: Chapter 1: Introduction
32
Terminology - cont.- Analysis by synthesis model
240-572: Chapter 1: Introduction
33
Designing a Pattern Recognition System
240-572: Chapter 1: Introduction
34
Designing a Pattern Recognition System
240-572: Chapter 1: Introduction
35
Steps in a Pattern Recognition System
• Sensing– Measuring of features, such as a digital
camera, or a microphone– We assume the measurements are given
• Segmentation and grouping– In the fish example, we have to isolate a
fish from other fishes, other non-fish objects, or the background
– Segmentation/grouping is a very difficult problem
240-572: Chapter 1: Introduction
36
Steps in a Pattern Recognition System – cont.
• Image segmentation is one of the most difficult problems in computer vision– Face detection, for example, can be viewed
as a image segmentation problem
240-572: Chapter 1: Introduction
37
Steps in a Pattern Recognition System – cont
• In speech recognition, the segmentation problem is called source separation– Mixed speech signal– Separated signal source 1– Separated signal source 2
240-572: Chapter 1: Introduction
38
Steps in a Pattern Recognition System – cont.
• Feature extraction/selection– A critical step for pattern recognition– Seeking distinguishing features that are
invariant to irrelevant transformations of the input
– Biometrics can be viewed as a feature selection problem
• Classification• Post-processing
– Context information– Multiple classifiers
240-572: Chapter 1: Introduction
39
The Design Cycle
240-572: Chapter 1: Introduction
40
Learning• Supervised learning
– A category label is given for each pattern in a training set
• Unsupervised learning– The system forms clusters or natural groupings
of the input patterns– The study of category formation
• Reinforcement learning– No desired output is provided; the feedback is
given