View
214
Download
1
Category
Tags:
Preview:
Citation preview
DSP Based Motion Tracking SystemDSP Based Motion Tracking System
Dani Cherkassky
Ronen Globinski
Advisor: Mr Slapak Alon
Presentation contents
IntroductionIntroduction
Summery
Results
Implementation
•Objective•The tracking task•Utilization potential•Design stages
•System diagram•Data flow diagram•Tracking flow chart•Motion detection algorithm•Tracking algorithm
•Conclusions •Future work
Design and implement a DSP based vision system that can detect and track a single, rigid, moving object
Design and implement a DSP based vision system that can detect and track a single, rigid, moving object
Objective
Introduction ResultsImplementation Summery
• The problem:Detecting and Tracking moving objects in a camera vision field often requires the constant attention of human operators, thus making the tracking task exposed to human errors and working hours problems
The tracking task
Introduction ResultsImplementation Summery
•The solution:We offer a DSP based machine vision application that can perform the motion detection and tracking tasks automatically.Our solution is based on an image processing algorithms that can be implemented on a digital signal processor platform and deliver a real-time performance.
Utilization potential
Introduction ResultsImplementation Summery
Motion Detection and tracking system
Security SurveillanceMilitary systems
Traffic control
Design stages
Introduction ResultsImplementation Summery
Stage CStage C Stage DStage D Stage EStage EStage AStage A Stage BStage B
PossibleAlgorithms
Examination(literature survey)
ImplementingThe selectedAlgorithms
(Matlab real time)
ImplementingThe selectedAlgorithms
(DSP platform)
Building theCamera motor
system
Tests andBenchmarks
System Diagram
Introduction ResultsImplementation Summery
Motors Pan & tilt system
B&W CMOS Camera
Analog Video
Stream
Master DSP BoardVideo Encoding & Image Processing
Slave DSP BoardVideo Decoding
PWM Motor Angle Control
Analog Video
StreamDigital Image
StreamVia SPORT
B&W TV Monitor
Data Flow Diagram
Introduction ResultsImplementation Summery
Bank 1Input
Bank 2NTSC frame 1
Bank 3NTSC frame 2
Input Image244 x 360Gray scale
Init empty NTSC frame before
transfer begins
Slave SDRAM
SPORT, DMA CoreUpdate NTSC frame with new image data
TVDMA, PPI
Software access (core)
Hardware access (DMA)
SlaveDSP
Video Data Flow on the slave DSP board
Introduction ResultsImplementation Summery
MasterDSP
Tracking Flow Chart
Motion tracking algorithms running on the master DSP board
Digital Video Stram
Separate Active Video
Current State
Idle
Tracking Detecting
∑-∆ Algorithm
EKLT Tracking Algorithm
Lost Motion Detected
Current State ß Tracking
Current State ß Detecting
NO YES
Current State ß Detecting
Current State ß Tracking
YESNO
Introduction ResultsImplementation Summery
Motion Detection Algorithm
In this section a motion detection algorithm(∑-∆ filter) proposed by A. Manzanera J. C. Richefeu is introduced.
Methods for motion detection :
• Background subtraction algorithms (the naive method)
• The background must stay constant• Irrelevant motion is not discarded
• Last K frames analysis algorithms (histogram , entropy , linear prediction)
• Consumes memory (K have to be large)
Not suitable for DSP applications !
• Recursive methods that use fixed number of estimates (Kalman filter , ∑-∆ filter)
OK! low complexity, low memory consumption, statistical measure on the temporal activity
Not compatible with noisy real world situations!
Motion detection , how ???
The principle of the motion detection method is to build a model of the static scene (i.e. without moving objects) called background, and then to compare every frame of the sequence to this background in order to discriminate the
regions of unusual motion, called foreground (the moving objects).
Introduction ResultsImplementation Summery
Motion Detection AlgorithmThe ∑-∆ algorithm – a recursive approximation of the temporal statistics, allowing a simple and efficient pixel-level change detection
0 0
t-1 t t-1
t-1 t t-1
for each pixel :
M ( ) I ( )
for each pixel :
Initializ
if M ( ) ( ) then M ( ) M ( )
if M ( ) ( ) then M ( ) M ( )
ation
for each frame t
t
t
x
x x
x
x I x x x X
x I x x x X
t t
for each pixel :
( ) M ( ) I (
for each ame
)
fr t
t
x
x x x
1 2
4
0 0
t
t-1 t t t-1
t-1 t
for each pixel :
V ( ) ( )
for each pixel such that ( ) 0 :
if V ( ) ( ) then V ( ) V ( )
if V ( ) ( ) then
Initialization
for each frame t
V
x
x x
x x
x N x x x Y
x N x
t t-1( ) V ( )x x Y
3
t
t
for each pixel :
if ( ) ( )
D ( ) 0;
else
fo
D (
r each frame t
) 1;
t t
x
x V x
x
x
t
- Input Frame
- Estimated Background value (approximation of the median of )
- First observation field / difference field (the mot )ion likelihood measure
V - Second observation fi
t
t t
t
I
M I
t
eld (the variance)
D - Motion field
Background estimation:Computation of the ∑-∆ mean (an approximation of the median)
Motion likelihood measure:Computation of the difference between the image and the median
Computation of the variance:Defined as the ∑-∆ mean of N times the non-zero differences
Motion computation:Comparison between the difference and the variance
, ,
parameters used for
sensitivity adjustments
X Y N
Introduction ResultsImplementation Summery
Motion Detection AlgorithmExample: two frames from a sequence acquired by a stationary camera
Still area
Clutter area
Motion areaAlgorithm results for 3 particular pixels from 3 different areas
The algorithm can discard irrelevant (clutter) motion - tree moving with the wind, and also has low sensitivity to noise
Moving object(foreground)
The motion field corresponds to the Boolean indicator of the condition: “the green line is over the black line” – every time the difference is greater than the variance the pixel is considered to be moving.
Introduction ResultsImplementation Summery
Motion Detection AlgorithmExample - cont’dThe motion field of the moving car
The sensitivity in this case is too high, thus the motion field is very noisy and unusable
The effect of the parameters X, Y, N on the motion field:
X=1, Y=1, N=4 X=1, Y=3, N=4 X=1, Y=3, N=4 Filtered
Changing the parameter Y affects the variance calculation and in this case lowering the sensitivity which results in a cleaner motion field
The motion field can be further cleaned by applying a 3x3 median filter to remove the “salt” noise.
Salt noise
Introduction ResultsImplementation Summery
Motion Tracking Algorithm
In this section a method proposed by J. Shi and C. Tomasi is introduced. It is an extension of a method proposed by B. D. Lucas and T. Kanade andwill therefore, from now on, be called Extended Lucas-Kanade Tracking (ELKT).
Image Motion Model (affine motion model)
The ELKT is based on the assumption that most changes between two frames are caused by image motion and affine transformations.
Moving template in Frame 1,
Moving template in Frame 2,
( , ) ( ( , , ), ) m mI t I t t x ξ x
Affine Model
image sequence
image coordinates with respect to center of template
transformation function
deformation matrix
translation (displacement) of the center of the template
the six parameters of
mI
x
D
d
the Affine motion model
T
,
xxx xy
yx yy y
xx xy yx yy x y
dd d
d d d
d d d d d d
ξ = Ax + d
A = I + D
D d
μ
( , )mI tx ( , )mI t x
Lets look at two sequential frames and express the transformation of the
moving object in mathematical terms
Introduction ResultsImplementation Summery
Motion Tracking Algorithm
( ) ( ) K JAx + d x0 0( , ), ( , )m mJ I x t K I x t Affine transform
two samples of the image sequence
Image Motion Estimation
2The dissimilarity:
The area of integration (the object templat
( ) (
e spatial coordinates
)
) W
J K d
W
x Ax + d x
We cannot assume perfect correspondence between J(x) and K(Dx + d). This due to image noise and imperfections in the image motion model.
There for, some dissimilarity measure should be minimized. In ELKT the sum of squared differences (SSD), is chosen as the dissimilarity measure.
Motion estimation = minimizing the dissimilarity . The SSD method ensures that the dissimilarity function has only one minimum.
• The natural approach to minimize the dissimilarity would beto differentiate the dissimilarity function and set the results to zero.
Methods for minimizing the dissimilarity:
This equation would however be non-linear, and therefore difficult to solve. Not a good solution for our DSP Real-time system.
• The better solution is to linearize the function with a truncated Taylor expansion with respect to some good estimate and solve the new system by iterating the procedure.
Introduction ResultsImplementation Summery
Motion Tracking Algorithm
The tracking algorithm
arg min ( ) μμ μ
Tracking = determining the six parameters motion vector µ.
The final equation for estimating the six parameters of the Affine motion vector:
( )1 1 ( )( 1) ( )' 'n nn n T T T
μ μ φ φ φ E
0
1
1
1
( ) ( ) ( ( ))
( ) ( ) ( )
0
0 0 0y 0
0 0 and 0
0 0 1 0
0 1
T
x J K
X x J
x
x
y
x
0
0
0
E x ξ x
φ x x
D
D X(x)
D
The tracking is performed by updating µ using the iterative calculation of µ’ for every input frame. The number of iterations can be determined in advance according to the tracking performance.
For each frame n:1.Calculate µ’(n+1) by iterating this
equation.2.Update the motion vector µ. 3.Use µ to update the position of
the target object.
Introduction ResultsImplementation Summery
Motion Tracking Algorithm
The EKLT algorithm is recursive, we need to determine the number of iterations
• We have discovered that if the algorithm doesn't converge after 4-5 iterations there will be practically no convergence at all. Instead of making the exhaustive iterative process we offer a different approach:
We have found that method problematic for our system because :
• T have to be larger then some picture noise level (noise level must be measured).
• It may take too long for the algorithm to converge (not capable with RT systems).
• Run the EKLT algorithm for a fix number of iterations.
• Calculate the tracking error using the next equation :
• Check the tracking quality:
• If the previous inequality is satisfied continue tracking, else, go back to motion detection.
1
1
( 1) (n)
N
Tracking ii
Tracking Tracking
E EN
E n E
• It was suggested in [4] to continue the iterations till : ( 1)n
T
E
The average error calculated using Alfa filter
( ) (1 ) ( 1) for 0( 1)
( 1)
for 0
Tracking TrackingTracking
Tracking
E n E n nE n
E n n
The Advantages of our method are :1. No need to measure picture noise level because is an
approximation of that level.2. Tracking quality is estimated for each frame, it is easy to change
the template when that estimate goes low.
(1)TrackingE
Introduction ResultsImplementation Summery
Results
• Algorithm Complexity:
•CPU Utilization (for 10 x 20 Template size ) :a. Motion detection: 82,070,156 (Cycles)
b. Feature extraction: 20,857 (Cycles)
c. Tracking initialization: 6,550,668 (Cycles)
d. Solving EKLT equation: 5,205,380 (Cycles)
a. Motion detection: O{mxn} , mxn = Frame Size
b. Feature extraction: O {N} , N = Template Size
c. Tracking initialization – compute : O {N}
d. Solving EKLT equation: O {N}
1T φ φ φ
Introduction ResultsImplementation Summery
• System limitation : Results - cont’d
The system’s tracking speed is limited
• Tracking results :
α
y
x
Object plain Camera position
Calculation of the angular velocity of the target object:
Introduction ResultsImplementation Summery
Conclusions
• The motion detection algorithm:The ∑-∆ filter, combined with 3x3 median filter, proved to be a robust and accurate method of detection of moving objects for a small cost in memory consumption and computational complexity.
• The tracking algorithm:The EKLT tracking algorithm performed well on our system. The algorithm is efficient – the on-line calculations are relatively small and the convergence time is fairly good. The performance can be greatly improved by better selection of the initial features for the tracking process.
The good performance of the algorithms proved our concept that a real-time tracking system can be
implemented successfully on DSP platforms
The good performance of the algorithms proved our concept that a real-time tracking system can be
implemented successfully on DSP platforms
Introduction ResultsImplementation Summery
Future work
The DSP implementation must be optimized The DSP implementation must be optimized
Consider using a DSP with a dual PPI core to improve the performance Consider using a DSP with a dual PPI core to improve the performance
Improve the pan and tilt mechanismImprove the pan and tilt mechanism
Improve the system’s transition from motion detection to tracking Improve the system’s transition from motion detection to tracking
Introduction ResultsImplementation Summery
References
[1] A.Manzanera, J.C.Richefeu "A robust and computationally efficient motion detection algorithm based on Background estimation"
[2] G.Jing, C.Eng and D.Rajan. "Foreground motion detection by difference-based spatial temporal entropy"
[3] J.Shi and C.Tomashi, "Good Features to Track" Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.593-600 IEEE CS Press 1994.
[4] D.Hager and N.Belhumeur, "Efficient Region Tracking Whit Parametric Models of Geometry and Illumination" IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 20, No.10, October 1198.
Recommended