Upload
tuyen
View
23
Download
0
Embed Size (px)
DESCRIPTION
Optimization & Learning for Registration of Moving Dynamic Textures. Junzhou Huang 1 , Xiaolei Huang 2 , Dimitris Metaxas 1 Rutgers University 1 , Lehigh University 2. Outline. Background Goals & Problems Related Works Proposed Method Experiment Results Discussion & Conclusion. - PowerPoint PPT Presentation
Citation preview
ICCV 2007
Optimization & Learning for Registration of Moving Dynamic Textures
Junzhou Huang1, Xiaolei Huang2, Dimitris Metaxas1
Rutgers University1, Lehigh University2
ICCV 2007
Outline
• Background• Goals & Problems• Related Works• Proposed Method• Experiment Results• Discussion & Conclusion
ICCV 2007
Background
• Dynamic textures (DT)– static camera, exhibits a certain stationary
• Moving Dynamic textures (MDT)– dynamic textures captured by a moving camera
DT, [Kwatra et al. SIGGRAPH’03] MDT, [Fitzgibbon ICCV’01]
ICCV 2007
Background
• Video registration– Required by many video analysis applications
• Traditional assumption– Static, rigid, brightness constancy – Bergen et al. ECCV’92, Black et al. ICCV’93
• Relaxing rigid assumption– Dynamic textures– Doretto et al. IJCV’03, Yuan at al. ECCV’04, Chan
et al. NIPS’05, Lin et al. PAMI’07, Rav-Acha at al. Workshop at ICCV’05
ICCV 2007
Our Goals
• Registration of MDT– Recover the camera motion and register the image
sequences including moving dynamic textures
Left Translation Right Translation
ICCV 2007
Complex Optimization Problems
• Complex optimization– Camera motion, dynamic texture model– Chicken-and-Egg Problems
• Challenges– About the mean images– About LDS model– About the camera motion?
ICCV 2007
Related Works
• Fitzgibbon, ICCV’01– Pioneering attempt– Stochastic rigidity– Non-linear optimization
• Vidal et al. CVPR’05– Time varying LDS model– Static assumption in small time window– Simple and general framework but under estimation
ICCV 2007
Formulation
• Registration of MDT– I(t), the video frame– camera motion parameters– y0 , the desired average image of the video – y(t), related with appearance of DT– x(t), related with dynamics of DT
)(t
ICCV 2007
Generative Model
x(t-1) x(t) x(t+1)
y(t-1) y(t) y(t+1)
I (t-1) I (t) I ( t+1)
y0
W(t-1) W(t) W(t+1)
Generative image model for a MDT
ICCV 2007
First Observation
• Good registration– a good registration according to the accurate camera
motion should simplify the dynamic texture model while preserving all useful information
– Used by Fitzgibbon, ICCV’01, Minimizing the entropy function of an auto regressive process
– Used by Vidal, CVPR’05, optimizing time varying LDS model by optimizing piecewise LDS model
ICCV 2007
Second Observation
• Good registration– A good registration according to the
accurate camera motion should lead to a sharp average image whose statistics of derivative filters are similar to those of the input image frames.
• Image statistics – Student-t distribution / heavy tailed image priors– Huang et al. CVPR’99, Roth et al. CVPR’05
ICCV 2007
Prior Models
• The Average image priors• The motion priors• The dynamic priors
ICCV 2007
Average Image Priors
• Student-t distribution– Model parameters / contrastive divergence method
(a) Before registration, (b) in the middle of registration (c) after registration
ICCV 2007
Motion / Dynamic Priors
• Gaussian Perturbation (Motion)– Uncertainty in the motion modeled by a
Gaussian perturbation about the mean estimation M0 / the covariance matrix S ( a diagonal matrix.)
– Motivated by the work [Pickup et al. NIPS’06]• GPDM / MAR model (Dynamic)
– Marginalizing over all possible mappings between appearance and dynamics
– Motivated by the work [Wang et al. NIPS’05] [Moon et al. CVPR’06]
ICCV 2007
Joint Optimization
• Generative image model
• Optimization– Final marginal likelihood
– Scaled conjugate gradients algorithm (SCG)
ICCV 2007
Procedures
• Obtaining image derivative prior model• Dividing the long sequence into many short
image sequences• Initialization for video registration• Performing model optimization with the
proposed prior models until model convergence.
• With estimated y0, Y and X, the camera motion is then obtained
ICCV 2007
Obtaining Data
• Three DT video sequences– DT data, [Kwatra et al. SIGGRAPH’03]
• Synthesized MDT video sequence– 60 frames each, no motion from 1st to 20th frame and
from 41st to 60th – Camera motions with speed [1, 0] from 21st to 40th
ICCV 2007
Grass MDT Video
• The average image
(a) One frame, (b) the average image after registration, (c) before registration
ICCV 2007
Grass MDT Video
• The statistics of derivative filter responses
-60 -40 -20 0 20 40 600
0.05
0.1
0.15
Gradient
Pro
babi
lity
dist
ribut
ion
Input ImagesAfter RegistrationBefore Registration
ICCV 2007
Evaluation / Comparison
• False Estimation Fraction
• Comparison with two classical methods– Hybrid method, [Bergen et al. ECCV’92] [Black et
al. ICCV’93]– Vidal’method, [Vidal et al. CVPR’05]
ICCV 2007
Waterfall MDT Video
• Motion estimation
(a) Ground truth, (b) by hybrid method, (c) by Vidal’s, (d) proposed
ICCV 2007
Waterfall MDT Video
• The average Image and its statistics
The average image and related distribution after registration by (a) proposed method, (b) Vidal’s method, (c) hybrid method
ICCV 2007
FEF Comparisons
• On three synthesized MDT video
ICCV 2007
Real MDT Video
• Moving flower bed video• Ours
– 554 frames totally– Ground truth 110 pixels– Estimation 104.52 pixels
( FEF 4.98%)• Vidal’s
– 250 frames– Ground truth 85 pixels – Estimation 60 pixels ( FEF 29.41%)
ICCV 2007
Conclusions• What proposing:
– Powerful priors for MDT registration• What getting out:
– Camera motions– Average image – Dynamic texture model
• What learning?– Registration simplify DT model while preserving
useful information– Better registration lead to sharper average image
ICCV 2007
Thanks !
ICCV 2007
Thanks !
ICCV 2007
Future Works• More complex camera motions• Different Metric functions for evaluation• Multiple dynamic texture segmentation