Brightness Constancy The Aperture problem Regularization
Lucas-Kanade Coarse-to-fine Parametric motion models Direct depth
SSD tracking Robust flow
Slide 4
Optical Flow: Where do pixels move to?
Slide 5
Motion is a basic cue Motion can be the only cue for
segmentation
Slide 6
Motion is a basic cue Motion can be the only cue for
segmentation
Slide 7
Motion is a basic cue Even impoverished motion data can elicit
a strong percept
Slide 8
Applications tracking structure from motion motion segmentation
stabilization compression mosaicing
Slide 9
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 10
Definition of optical flow OPTICAL FLOW = apparent motion of
brightness patterns Ideally, the optical flow is the projection of
the three-dimensional velocity vectors on the image
Slide 11
Caution required ! Two examples : 1. Uniform, rotating sphere
O.F. = 0 2. No motion, but changing lighting O.F. 0
Slide 12
Mathematical formulation I (x,y,t) = brightness at (x,y) at
time t Optical flow constraint equation : Brightness constancy
assumption:
Slide 13
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 14
The aperture problem 1 equation in 2 unknowns
Slide 15
Aperture Problem: Animation
Slide 16
The Aperture Problem 0
Slide 17
Slide 18
What is Optical Flow, anyway? Estimate of observed projected
motion field Not always well defined! Compare: Motion Field (or
Scene Flow) projection of 3-D motion field Normal Flow observed
tangent motion Optical Flow apparent motion of the brightness
pattern (hopefully equal to motion field) Consider Barber pole
illusion
Slide 19
Slide 20
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 21
Horn & Schunck algorithm Additional smoothness constraint :
besides OF constraint equation term minimize e s + e c
Slide 22
The calculus of variations look for functions that extremize
functionals and
Slide 23
Calculus of variations Suppose 1. f(x) is a solution 2. (x) is
a test function with (x 1 )= 0 and (x 2 ) = 0 for the optimum
:
Slide 24
Calculus of variations Using integration by parts : Therefore
regardless of (x), then Euler-Lagrange equation
Slide 25
Calculus of variations Generalizations 1. Simultaneous
Euler-Lagrange equations : 2. 2 independent variables x and y
Slide 26
Hence Now by Gausss integral theorem, so that = 0 Calculus of
variations
Slide 27
Given the boundary conditions, Consequently, for all test
functions . is the Euler-Lagrange equation
Slide 28
Horn & Schunck The Euler-Lagrange equations : In our case,
so the Euler-Lagrange equations are is the Laplacian operator
Slide 29
Horn & Schunck Remarks : 1. Coupled PDEs solved using
iterative methods and finite differences 2. More than two frames
allow a better estimation of I t 3. Information spreads from
corner-type patterns
Slide 30
Slide 31
Horn & Schunck, remarks 1. Errors at boundaries 2. Example
of regularization (selection principle for the solution of
ill-posed problems)
Slide 32
Results of an enhanced system
Slide 33
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 34
Slide 35
Slide 36
Slide 37
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 38
Slide 39
Slide 40
Slide 41
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 42
Slide 43
Slide 44
Slide 45
Slide 46
Slide 47
Slide 48
Slide 49
Slide 50
Slide 51
Slide 52
Slide 53
Slide 54
Slide 55
Slide 56
Slide 57
Slide 58
Slide 59
Slide 60
Slide 61
Slide 62
Slide 63
Slide 64
Slide 65
Slide 66
Slide 67
Slide 68
Slide 69
Slide 70
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 71
Slide 72
Slide 73
Slide 74
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 75
Slide 76
Slide 77
Slide 78
Slide 79
Slide 80
Optical Flow Brightness Constancy The Aperture problem
Regularization Lucas-Kanade Coarse-to-fine Parametric motion models
Direct depth SSD tracking Robust flow
Slide 81
Slide 82
Slide 83
Slide 84
Slide 85
Slide 86
Slide 87
Textured Motion http://www.cs.ucla.edu/~wangyz/Research/w
yzphdresearch.htm http://www.cs.ucla.edu/~wangyz/Research/w
yzphdresearch.htm Natural scenes contain rich stochastic motion
patterns which are characterized by the movement of a large amount
of particle and wave elements, such as falling snow, water waves,
dancing grass, etc. We call these motion patterns "textured
motion".