Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Siwei LyuComputer Science Department
University at Albany, SUNY, USA
Stefan RothComputer Science Department
Technische Universität Darmstadt, Germany
tutorial web page: http://www.gris.informatik.tu-darmstadt.de/teaching/iccv2009/index.en.htm
Modeling Natural ImageStatistics for Computer Vision
Part III - MRF Models in the Wavelet DomainLecturer: Siwei Lyu
09/27/2009Siwei Lyu and Stefan Roth
MRFs in wavelet domain
■ extend local statistical models for wavelet coefficients to a global extent
■ examples• tree based models [Ronberg et.al., 2001; Wainwright et.al., 2003]• field of GSM (FoGSM) [Lyu & Simoncelli, NIPS 2006; PAMI 2009]• implicit MRF model [Lyu, CVPR 2009]
■ we will focus on the latter two models in this tutorial
2
09/27/2009Siwei Lyu and Stefan Roth
x field
z field
3
field of GSM (FoGSM)
p(x)
x
x = u×√
zmarginal
GSM
blockGSM
field of GSM
x = u×√
z
x = u⊗√
z
single coefficient
coefficient block
one subband
09/27/2009Siwei Lyu and Stefan Roth
x u log z
decomposition & samples
4
09/27/2009Siwei Lyu and Stefan Roth
marginal
joint
model evaluation
5
09/27/2009Siwei Lyu and Stefan Roth
original image noisy image (! = 25) matlab wiener2 FoGSM
(14.15dB) (27.19dB) (30.02dB)
denoising with FoGSM
6
20.17
09/27/2009Siwei Lyu and Stefan Roth
pairwise conditional density
x1
x 2
p(x2|x1)
E(x2|x1)
E(x2|x1)+std (x2|x1)
E(x2|x1)-std (x2|x1)
“bow-tie”[Buccigrossi & Simoncelli, 1997]
7
09/27/2009Siwei Lyu and Stefan Roth
pairwise conditional density
x1
x 2
E(x2|x1)
E(x2|x1)+std (x2|x1)
E(x2|x1)-std (x2|x1)
E(x2|x1) ≈ ax1 var(x2|x1) ≈ b + cx21
8
09/27/2009Siwei Lyu and Stefan Roth
■ constraints
■ maximum entropic conditional density
■ known as the singleton conditional density
conditional density
µi = E(xi|xj,j∈N (i)) =�
j∈N(i)
ajxj
σ2i = var(xi|xj,j∈N (i)) = b +
�
j∈N(i)
cjx2j
p(xi|xj,j∈N (i)) =1�2πσ2
i
exp�− (xi − µi)2
2σ2i
�
9
09/27/2009Siwei Lyu and Stefan Roth
Brook’s Lemma
■ in an MRF, all singleton conditional densities ⇔ joint density
■ the joint density may not have closed form■ thus the resulting MRF is implicit
10
{p(xi|xj,j∈N (i))|∀i}⇔ p(x)
[Brook, 1964]
09/27/2009Siwei Lyu and Stefan Roth
implicit MRF model
■ defined by all singletons■ joint density (and clique potential) is implicit■ learning: maximum pseudo-likelihood
11
θMPL = argmaxθ
�
i
log p(xi|xj,j∈N (i); θ)
09/27/2009Siwei Lyu and Stefan Roth
ICM-MAP denoising
- set initial value for �x(0), and t = 1
- repeat until convergence
- repeat for all i
- compute the current estimation for xi, as
x(t)i = argmax
xi
log p(x(t)1 , · · · , x(t)
i−1,
xi, x(t−1)i+1 , · · · , x(t−1)
d |�y).
- t← t + 1
12
argmaxx
p(x|y) = argmaxx
p(y|x)p(x) = argmaxx
log p(y|x) + log p(x)
iterative conditional mode (ICM)
09/27/2009Siwei Lyu and Stefan Roth
ICM-MAP denoising
local adaptive and iterative Wiener filtering
xi =σ2
wσ2i
σ2w + σ2
i
yi
σ2w
+µi
σ2i
−�
i�=j
wij(xj − yj)
.
13
argmaxxi
log p(y|x) + log p(x)
= argmaxxi
log p(y|x) + log p(x1, · · · , xi−1, xi, xi+1, · · · , xn)
= argmaxxi
log p(y|x)� �� �can be further simplified
+ log p(xi|xj,j∈N(i))� �� �singleton conditional
+ ✭✭✭✭✭✭✭✭✭✭✭✭✭✭log p(xj,j∈N(i))� �� �constant w.r.t xi
.
09/27/2009Siwei Lyu and Stefan Roth
summary
imagerepresentation
statisticalobservations
computer visionapplications
mathematical model
14
09/27/2009Siwei Lyu and Stefan Roth
edge + texture?■ primal sketch model [Guo, Zhu & Wu 2005]
■ a unified statistical image model for texture and structures
15
09/27/2009Siwei Lyu and Stefan Roth
related challenges■ time -- natural videos■ 3D -- natural range images■ motion -- natural optical flows■ chromatics -- natural color images■ lighting - natural illuminations■ properties of specific image class
• medical images [Pineda et.al., SPIE 2008]• satellite images [Jager & Hellwich, IGRASS 2005]• face images [Liu et.al., IJCV 2004]• human bodies [Norouzi, CVPR 2009]
16
09/27/2009Siwei Lyu and Stefan Roth
big question marks■ what are natural images, anyway?
■ white noises are “natural” as they are the result of cosmic radiations
■ naturalness is in the eyes of the beholders
17
09/27/2009Siwei Lyu and Stefan Roth
big question marks■ what are natural images, anyway?
■ white noises are “natural” as they are the result of cosmic radiations
■ naturalness is in the eyes of the beholders
17
“unnatural” to a prehistoric human
09/27/2009Siwei Lyu and Stefan Roth18
natural!
image!
statistics
math
statisticsbiology
computer!
science
image!
processing
machine!
learning
computer!
vision
optimization
perception
neuro-!
science
signal!
processing
09/27/2009Siwei Lyu and Stefan Roth19
future directions
■ comprehensive model capturing all known statistical properties of natural images
■ efficient algorithms for learning and inference■ tighter connection to mid and high level computer
vision• principled framework to find effective feature types based on
image statistics• and many more … …
09/27/2009Siwei Lyu and Stefan Roth
resources■ D. L. Ruderman. The statistics of natural images. Network: Computation in
Neural Systems, 5:517–548, 1996. (good introduction)■ E. P. Simoncelli and B. Olshausen. Natural image statistics and neural
representation. Annual Review of Neuroscience, 24:1193–1216, 2001. (neural science perspective)
■ S.-C. Zhu. Statistical modeling and conceptualization of visual patterns. IEEE Trans PAMI, 25(6), 2003. (computer vision perspective)
■ A. Srivastava, A. B. Lee, E. P. Simoncelli, and S.-C. Zhu. On advances in statistical modeling of natural images. J. Math. Imaging and Vision, 18(1):17–33, 2003. (mathematical perspective)
■ E. P. Simoncelli. Statistical modeling of photographic images. In Handbook of Image and Video Processing, 431–441. Academic Press, 2005. (signal processing perspective)
■ A. Hyvärinen, J. Hurri, and P. O. Hoyer. Natural Image Statistics: A probabilistic approach to early computational vision. Springer, 2009. (statistical modeling and recent accounts)
20
09/27/2009Siwei Lyu and Stefan Roth21
thank you