Upload
alexia-reed
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
National Taiwan University
A Road Sign Recognition System Based on a Dynamic
Visual Model
C. Y. Fang Department of Information and Computer Education
National Taiwan Normal University, Taipei, Taiwan, R. O. C.
C. S. Fuh Department of Computer Science and Information Engineering
National Taiwan University, Taipei, Taiwan, R. O. C.
S. W. Chen Department of Computer Science and Information Engineering
National Taiwan Normal University, Taipei, Taiwan, R. O. C.
P. S. Yen Department of Information and Computer Education
National Taiwan Normal University, Taipei, Taiwan, R. O. C.
National Taiwan University [email protected] 2
Outline
Introduction Dynamic visual model (DVM) Neural modules Road sign recognition system Experimental Results Conclusions
National Taiwan University [email protected] 3
Introduction -- DAS Driver assistance systems (DAS)
The method to improve driving safety
Passive methods: seat-belts, airbags, anti-lock braking systems, and so on.
Active methods: DAS
Driving is a sophisticated process The better the environmental information a
driver receives, the more appropriate his/her expectations will be.
National Taiwan University [email protected] 4
Introduction -- VDAS Vision-based driver assistance systems (VDAS) Advantages:
High resolution Rich information Road border detection or lane marking detection Road sign recognition
Difficulties of VDAS Weather and illumination Daytime and nighttime Vehicle motion and camera vibration
National Taiwan University [email protected] 5
Subsystems of VDAS Road sign recognition system System to detect changes in driving
environments System to detect motion of nearby vehicles Lane marking detection Obstacle recognition Drowsy driver detection ……
National Taiwan University [email protected] 6
Introduction -- DVM
DVM: dynamic visual model A computational model for visual analysis
using video sequence as input data
Two ways to develop a visual model Biological principles Engineering principles
Artificial neural networks
Dynamic Visual Model
Conceptualcomponent
Perceptualcomponent
Sensorycomponent
Information acquisition
CART neural module
STA neural module
Yes
No
Video images
Focuses of attention
Spatialtemporal information
Categorical features
Category
Feature detection
Pattern extraction
CHAM neural module
Patterns
Data transduction
Action
Epi
sodi
c M
emor
y
National Taiwan University [email protected] 8
Human Visual Process
Transducer
Sensory analyzer
Class of input stimuli
Perceptual analyzer
Conceptual analyzer
Physical stimuli
Data compression
Low-level feature extraction
High-level feature extraction
Classification and recognition
National Taiwan University [email protected] 9
Neural Modules Spatial-temporal attention (STA)
neural module Configurable adaptive resonance the
ory (CART) neural module Configurable heteroassociative memo
ry (CHAM) neural module
National Taiwan University [email protected] 10
STA Neural Network (1)ak
Output layer(Attention layer)
nj
Inhibitory connection
Excitatory connection
Input layer
wij
ai
xj
nk
ni
National Taiwan University [email protected] 11
STA Neural Network (2) The input to attention neuron ni due to input st
imuli x:
The linking strengths between the input and the attention layers
corresponding neurons
wkj
ni
nj
nk
Input neuron
Attention layer
rk
Gaussian function G
m
jjiji
vi xwI
1
xw
National Taiwan University [email protected] 12
STA Neural Network (3) The input to attention neuron ni due to later
al interaction:
Lateral distance
“Mexican-hat” function of lateral interaction
Interaction
+
ikNk
kikikli
i
aMuI,
rr
National Taiwan University [email protected] 13
STA Neural Network (4) The net input to attention neuron ni :
: a threshold to limit the effects of noise
where 1< d <0
)),(( iii netqBpaAa
li
vii IInet
, 0 if
0 if )(
xdx
xxxA
, 0 if 0
0 if )(
x
xxxB
National Taiwan University [email protected] 14
STA Neural Network (5)
tt
p
1
pd
1
The activation of an attention neuron in response to a stimulus.
stimulus
activation
National Taiwan University [email protected] 15
ART2 Neural Network (1) CART
r
p
u
w
v
x
q
y
Input vector i
Input representation field F1
Attentional subsystemOrienting subsystem
G
G
G
G
G
Category representation field F2
Reset signal
+
+
+
+
+
+
+
++
++
++
+
+
+
+
+
-
---
-
Signal generatorS
National Taiwan University [email protected] 16
ART2 Neural Network (2) The activities on each of the six sublayers on F 1:
where I is an input pattern
where
where the J th node on F 2 is the winner
iii auIw
we
wx i
i
)()( iii qbfxfv
uv
e vii
iJii dzup
qp
e pii
x0 0)(
xxxf
National Taiwan University [email protected] 17
ART2 Neural Network (3) Initial weights:
Top-down weights:
Bottom-up weights: Parameters:
0)0( ijz
Mdji
)1(
1)0(z
0, ba
10 d
11
d
cd
10
10
1e
National Taiwan University [email protected] 18
HAM Neural Network (1) CHAM
j
Output layer(Competitive
layer)Excitatoryconnection
Input layer
wij
xj
i
viv1 v2 vn
National Taiwan University [email protected] 19
HAM Neural Network (2) The input to neuron ni due to input stimuli x:
nc: the winner after the competition
m
jjijii xwnet
1
xw
)).max(arg( ii
c netn
otherwise. 0
if 1 ci
ni v
National Taiwan University [email protected] 20
Road Sign Recognition System Objective
Get information about road Warn drivers Enhance traffic safety Support other subsystems
National Taiwan University [email protected] 21
Problems
contrary light
side by side shaking occlusion
National Taiwan University [email protected] 22
Information Acquisition
Color information Example: Red color
Shape information Example: Red color edge
National Taiwan University [email protected] 23
Results of STA Neural Module— Adding Pre-attention
National Taiwan University [email protected] 24
Locate Road Signs — Connected Component
National Taiwan University [email protected] 25
Categorical Feature Extraction
Normalization: 50X50 pixels Remove the background pixels Features:
Red color horizontal projection: 50 elements Green color horizontal projection: 50 elements Blue color horizontal projection: 50 elements Orange color horizontal projection: 50 elements White and black color horizontal projection: 50
elements Total: 250 elements in a feature vector
National Taiwan University [email protected] 26
Conceptual Component— Classification results of the CART
Training Set
Test Set
National Taiwan University [email protected] 27
Conceptual Component— Training and Test Patterns for the CHAM
National Taiwan University [email protected] 28
Conceptual Component— Training and Test Patterns for the CHAM
National Taiwan University [email protected] 29
Conceptual Component— Another Training Patterns for the CHAM
National Taiwan University [email protected] 30
Experimental Results of the CHAM
National Taiwan University [email protected] 31
Experimental Results
National Taiwan University [email protected] 33
Other Examples
National Taiwan University [email protected] 34
Discussion
Vehicle and camcorder vibration Incorrect recognitions
Input patterns
Recognition results
Correct patterns
National Taiwan University [email protected] 35
Conclusions (1) Test data: 21 sequences
Detection rate (CART): 99% Misdetection: 1% (11 frames) Recognition rate (CHAM): 85% of detected
road signs Since our system only outputs a result for
each input sequence, this ratio is enough for our system to recognize road signs correctly.
National Taiwan University [email protected] 36
Conclusions (2)
A neural-based dynamic visual model Three major components: sensory, perceptual
and conceptual component Future Researches
Potential applications Improvement of the DVM structure DVM implementation