Upload
meagan-alice-baker
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Pyramidal Implementation of Lucas Kanade Feature Tracker
Jia HuangXiaoyan Liu
Han XinYizhen Tan
Abstract
IntroductionTracking algorithm
Lucas-Kanade algorithm Iterative implementation
Tracking features analysis Feature lost Feature selection
Objective For a given point u in image A, find its
corresponding location v = u + d in image B.
Image A Image B
d
Residual function and Window size
2( ) ( , ) ( ( , ) ( , ))y yx x
x x x x
u wu w
x y x yx u w y u w
d d d A x y B x d y d
To find the location Minimize residual function:
: Integration window size
Small integration window Higher accuracy
Larger integration window Higher robustness
Nature tradeoff:
,x yw w
Pyramid Implementation of LK algorithm Calculate a set of pyramid representations of original image Apply traditional tracking algorithm for each level Results of current iteration is propagated to next iteration Key point: the same window size is used for each level
Top View Side View
Lucas-Kanade algorithm(1)
At the level L, we define images A and B: ( , ) [ 1, 1] [ 1, 1]x x x x y y y yx y p w p w p w p w
( , ) [ , ] [ , ]x x x x y y y yx y p w p w p w p w
( , ) ( , )LA x y I x y
( , ) ( , )L L Lx yB x y J x g y g
2( ) ( , ) ( ( , ) ( , ))y yx x
x x y y
p wp w
x y x yx p w y p w
v v v A x y B x v y v
Lucas-Kanade algorithm(2) At the optimum, the first derivative of
After first order Taylor expansion
Components in the equation above
( )| [0 0]
optv v
v
v
,
( )( ) ( ( , ) ( , ) )
y yx x
x x y y
p wp w
x p w y p w
v B B B Bv A x y B x y v
v x y x y
( , ) ( , ) ( , )I x y A x y B x y T
x
y
I B BI
I x y
2
2
1 ( )
2
y yx x
x x y y
T p wp wxx x y
x p w y p w yx y y
I II I Ivv
I II I Iv
Lucas-Kanade algorithm(3) Two derivative images are expressed:
With these notation, we can get:
The optimum optical flow vector is ( , ) ( 1, ) ( 1. )( , )
2x
A x y A x y A x yI x y
x
( , ) ( , 1) ( . 1)
( , )2y
A x y A x y A x yI x y
y
1optv G b
bG
Pyramidal diagram
Inner loop: K-level K initialized to 1, assume that the previous
computations from iterations 1,2,...,k-1 provide an initial guess
The new translated image according to
Iterative scheme of LK algorithm(1)
. , 3me g L
0mL
1 1 1[ ]k k k Tx yv v v
1kv
( , ) [ , ] [ , ]x x x x y y y yx y p w p w p w p w 1 1( , ) ( , )k k
k x yB x y B x v y v
Iterative scheme of LK algorithm(2) The goal: to compute the residual pixel motion vector
, that minimizes the error function
Image mismatch vector , where the image difference delta I k defined as:
New pixel displacement guess is computed for the next iteration step k+1:
[ ]k k kx y
2( ) ( , ) ( ( , ) ( , ))y yx x
x x y y
p wp wk k k kx y k x y
x p w y p w
A x y B x y
( , ) ( , )
( , ) ( , )
y yx x
x x y y
p wp wk x
k
x p w y p w k y
I x y I x yb
I x y I x y
( , ) ( , ) ( , )k kI x y A x y B x y
kbthk
1k k kv v
Iterative scheme of LK algorithm(3)
On average, 5 iterations are enough At the 1st iteration (k=1), the initial guess is set
to zero
The final solution for the optical flow vector is
Outer loop: L-levelThe vector d is propagated to the next level
L-1 and overall procedure is repeated L-1, L-2, …, 0
1
KL K k
k
v d v
0 [0 0]T
Declaring a Feature Lost
Several cases of lost feature the point falls outside of the image image patch around the tracked point varies
too much between image A and image B too large displacement
How to solve it combine a traditional tracking approach with an affine image matching
Feature Lost Example(1)
Image A Image B
Feature Lost Example(2)
Image A Image B
Feature Selection
Intuitive To select the point u on image A good to track.
Process steps: Compute the G matrix and λm
Call λmax the maximum value of λm Retain the pixels that have a λm value larger than a percentage of λmax Retain the local max. pixels Keep the subset of those pixels so that the minimum distance between pixels is larger than a threshold
Example of LK Feature Tracking
Image A Image B
More Examples
Image BImage A
Summary
Lucas-Kanade Feature Tracker is one of the most popular versions of two-frame differential methods for motion estimation
Iterative implementation of the Lucas-Kanade optical flow computation provides sufficient local tracking accuracy.
Thanks for your attention
Any question?