Variational methods for restoration of phase or orientation data

Martin Storath

joint works with Laurent Demaret, Michael Unser, Andreas Weinmann

Image Analysis and Learning GroupUniversität Heidelberg

Symposium on Mathematical Optics, Image Modelling, and Algorithms

June 21, 2016

Phase or orientation data

Phase or orientation data: images (or time series) whose pixels take theirvalues on the unit circle i.e. fij ∈ S1 = T

Time [h]

0 1000 2000 3000 4000 5000 6000 7000 8000

Direction [ra

d]

-π/2

0

π/2

π

Hourly wind directions at station VENF1 (Venice, FL) in 2014 Interferometric SAR image ofVesuvius used for creation of

digital elevation maps

Goal: devise variational regularization (total variation, Potts, Mumford-Shah, ...)for cirlce-valued data

2 / 20

Total variation regularization for circle-valued data

Total variation (TV) regularization for real valued data (Rudin/Osher/Fatemi ’92)

u∗ = arg minu∈RM×N

γ∑

i,j

|uij − ui+1,j |+ γ∑

i,j

|uij − ui,j+1|+∑

i,j

|uij − fij |p

f ∈ RM×N given image, γ > 0 model parameter

widely used for edge/jump preserving regularization

convex computationally tractable

TV for circle-valued data, f ∈ TM×N (Giaquinta/Modica/Soucek ’93; Strekalovskiy/Cremers ’11)

u∗ = arg minu∈TM×N

γ∑

i,j

dT(uij , ui+1,j) + γ∑

i,j

dT(uij , ui,j+1) +∑

i,j

dT(uij , fij)p

where dT(x, y) is the arc length distance between two point x, y ∈ S1

nonlinear data space vector space methods not applicable

nonconvex globally optimal solutions possible?

3 / 20

Total variation regularization for circle-valued data

Total variation (TV) regularization for real valued data (Rudin/Osher/Fatemi ’92)

u∗ = arg minu∈RM×N

γ∑

i,j

|uij − ui+1,j |+ γ∑

i,j

|uij − ui,j+1|+∑

i,j

|uij − fij |p

f ∈ RM×N given image, γ > 0 model parameter

widely used for edge/jump preserving regularization

convex computationally tractable

TV for circle-valued data, f ∈ TM×N (Giaquinta/Modica/Soucek ’93; Strekalovskiy/Cremers ’11)

u∗ = arg minu∈TM×N

γ∑

i,j

dT(uij , ui+1,j) + γ∑

i,j

dT(uij , ui,j+1) +∑

i,j

dT(uij , fij)p

where dT(x, y) is the arc length distance between two point x, y ∈ S1

nonlinear data space vector space methods not applicable

nonconvex globally optimal solutions possible?

3 / 20

L1-TV for circle-valued data in 1D

L1-TV for circle-valued signals f ∈ TN , (e.g., time series of orientations),

arg minu∈TN

γ

N−1∑n=1

dT(un, un+1) +N∑

n=1

dT(un, fn),

is computationally tractable:

Theorem (S., Weinmann, Unser)

There is an exact algorithm for the total variation problem with circle-valued data.Its complexity is O(KN), where K ≤ N the number of unique values in f .

Sketch of proof:

Show that there is a minimizer whose values are subset of the data valuesand its antipodal points.

Utilize the Viterbi algorithm (a type of dynamic programming).

Generalize infimal convolution for fast message passing (Felzenszwalb/Huttenlocher)

to circular data.

S., Weinmann, Unser. Exact algorithms for L1-TV regularization of real-valued or circle-valued signals. SIAM J. Sci.Comp. (2016).

4 / 20

Discontinuity-preserving smoothing of time series

Time [h]

0 1000 2000 3000 4000 5000 6000 7000 8000

Direction [ra

d]

-π/2

0

π/2

π

Hourly wind directions at station VENF1 (Venice, FL) in 2014

Time [h]

0 1000 2000 3000 4000 5000 6000 7000 8000

Direction [ra

d]

-π/2

0

π/2

Global minimizer of L1-TV with circle-valued data (CPU time 3.3 sec)

Data available at http://www.ndbc.noaa.gov/historical_data.shtml.5 / 20

TV for circle-valued data NP hard in 2D

TV problem for circle-valued images fij ∈ T = S1,

arg minu∈Tm×n

γ∑

i,j

dT(ui,j , ui,j+1) + γ∑

i,j

dT(ui+1,j , ui,j) +∑

i,j

dT(uij , fij),

NP-hard in 2D (Cremers/Strekalovskiy ’13)

resort to approximative strategies

Our approach: Cyclic proximal point algorithm (CPPA)

devise algorithm for general (Riemannian) manifold-valued TV problem

arg minu∈Mm×n

γ∑

i,j

dM(ui,j , ui,j+1) + γ∑

i,j

dM(ui+1,j , ui,j) +∑

i,j

dM(uij , fij),

where dM is the distance induced by the Riemannian metric

algorithm should be globally convergent on “nice” manifoldsM, e.g., tensormanifolds

6 / 20

TV for circle-valued data NP hard in 2D

TV problem for circle-valued images fij ∈ T = S1,

arg minu∈Tm×n

γ∑

i,j

dT(ui,j , ui,j+1) + γ∑

i,j

dT(ui+1,j , ui,j) +∑

i,j

dT(uij , fij),

NP-hard in 2D (Cremers/Strekalovskiy ’13)

resort to approximative strategies

Our approach: Cyclic proximal point algorithm (CPPA)

devise algorithm for general (Riemannian) manifold-valued TV problem

arg minu∈Mm×n

γ∑

i,j

dM(ui,j , ui,j+1) + γ∑

i,j

dM(ui+1,j , ui,j) +∑

i,j

dM(uij , fij),

where dM is the distance induced by the Riemannian metric

algorithm should be globally convergent on “nice” manifoldsM, e.g., tensormanifolds

6 / 20

Tensor manifolds in diffusion tensor imaging

Diffusion tensor imaging (DTI) (Basser/Mattiello/LeBihan ’94)

Based on gradient weighted MR-images Dv w.r.t. direction v ∈ R3

Relation to diffusion tensor P by Stejskal-Tanner equation

Dv = A0e−b vT Pv

Clinical application: study of neurodegenerative diseases, e.g., Alzheimer

The diffusion tensor manifold (Pennec et al. ’04)

A diffusion tensor P is a symmetric positivedefinite 3 × 3 matrix (P ∈ Pos3).

Pos3 is a Riemannian manifold with theRiemannian metric

gP(A ,B) = trace(P−1/2AP−1BP−1/2),

where P ∈ Pos3, and A ,B tangent vectors(symmetric 3 × 3 matrices) at P.

affine invariant distance, unique geodesics

A diffusion tensor image visualizedby ellipsoids

7 / 20

Proposed minimization approach

Split the TV functional into atomic functionals

F(u) = γ∑

i

d(ui , ui−1) +∑

j

d(uj , fj)p =∑

i

Fi(u) + G(u),

where Fi(u) = γ d(ui , ui−1) and G =∑

j d(uj , fj)p .

Use a cyclic proximal point strategy (Bertsekas, Bacak), i.e.,iterate the proximal mappings of G and Fi ,

proxλG(u) = arg minv

12 d(u, v)2 + λG(v),

proxλFi(u) = arg min

v

12 d(u, v)2 + λFi(v).

(For simplicity: 1D functional; 2D/3D analogously)

Weinmann, Demaret, S. Total variation regularization of manifold-valued images. SIAM J. Imag. Sci. (2014)8 / 20

Computation of proximal points on a manifold

Key observation: Proximal mappings can be computed explicitly

Proximal mapping of G given by

proxλG(u)i = [ui , fi]t , t =

2λ(1+2λ)

d(ui , fi), for p = 2,

min(λ, d(ui , fi)), for p = 1.

Proximal mapping of Fi given by

proxλFi(u)j =

uj , if j , i, i − 1,[ui , ui−1]t , if j = i,[ui−1, ui]t , if j = i − 1,

with t = min(λγ, 12 d(ui , ui−1)).

Efficient computation of geodesics [P,Q] and Riemannian distances d(P,Q)

[P,Q]t = expP(t · logP(Q)) and d(P,Q) = ‖ logP(Q)‖P

via Riemannian exponential and logarithmic mappings (Pennec et al. ’06)

logP Q = P12 log(P−

12 QP−

12 )P

12 , and expP A = P

12 exp(P−

12 AP−

12 )P

12 .

Weinmann, Demaret, S. Total variation regularization of manifold-valued images. SIAM J. Imag. Sci. (2014)9 / 20

Results of TV regularization for synthetic DT images

Original Rician noise (σ = 90) Result using the proposed method

10 / 20

Results for real DT images

Diffusion tensor image of a human brain TV result using proposed cyclic proximalpoint algorithm (CPU-Time: 496 sec)

Data by courtesy of the CAMINO project11 / 20

Analytic results

Cartan-Hadamard manifolds: (e.g., DTI manifold Pos3)For each x0, x1 there is a point y on the geodesic [x0, x1] such that for every z,

d2(z, y) ≤ 12 d2(z, x0) + 1

2 d2(z, x1) − 14 d2(x0, x1).

Theorem (Weinmann/Demaret/S. ’14)

In a Cartan-Hadamard manifold (complete, simply connected), the proposedcyclic proximal point algorithm for TV minimization converges towards a globalminimizer.

Related approaches for TV with manifold-valued data

a priori discretization of manifold and convex relaxation (Lellmann et al. ’13)

iteratively reweighted least squares minimization (Grohs/Sprecher ’15)

Douglas-Rachford splitting for manifolds with constant sectional curvature(Bergmann et al. ’16)

12 / 20

Analytic results

Cartan-Hadamard manifolds: (e.g., DTI manifold Pos3)For each x0, x1 there is a point y on the geodesic [x0, x1] such that for every z,

d2(z, y) ≤ 12 d2(z, x0) + 1

2 d2(z, x1) − 14 d2(x0, x1).

Theorem (Weinmann/Demaret/S. ’14)

In a Cartan-Hadamard manifold (complete, simply connected), the proposedcyclic proximal point algorithm for TV minimization converges towards a globalminimizer.

Related approaches for TV with manifold-valued data

a priori discretization of manifold and convex relaxation (Lellmann et al. ’13)

iteratively reweighted least squares minimization (Grohs/Sprecher ’15)

Douglas-Rachford splitting for manifolds with constant sectional curvature(Bergmann et al. ’16)

12 / 20

CPPA for phase data

Application of proposed CPPA to circle-data using

expa(v) = e i(θ+v),

with a = e iθ and v ∈] − π; π[, and

exp−1a (b) = arg(b/a),

where a, b complex number representations of values on the unit circle

Results

Interferometric SARimage of Vesuvius (real

data)

L2-TV L1-TV TV with Huber data term

S1-values visualized as hue component in the HSV colorspace.13 / 20

Further applications of our methods for manifold-valued regularization

Shape data (RPd manifold)

Top: Time series of segmentation curves; Bottom: TV regularization in shape space

X-ray tensor tomography (Malecki et al. ’14) (Pos3 manifold)

Carbon fibers X-ray tensors TV denoising

Position data (SO(3) manifold)

0

10

20

30

40

50

60

70

80

90

100

Original

0

10

20

30

40

50

60

70

80

90

100

Noisy SO(3) data

0

10

20

30

40

50

60

70

80

90

100

TV denoising

Baust, Demaret, S., Navab, Weinmann. Total variation regularization for shape signals. CVPR (2015)M. Wieczorek et al. Total variation regularization for x-ray tensor tomography. Fully3D (2015)Weinmann, Demaret, S. Total variation regularization of manifold-valued images. SIAM J. Imag. Sci. (2014)

14 / 20

Mumford-Shah regularization for vectorial data

Mumford-Shah model for vectorial data (Mumford/Shah ’85/’89; Blake/Zisserman ’87)

arg minu,K

γ length(K) + α

∫KC|∇u|2 dx +

∫(u − f)p dx

where K ⊂ R2 denotes a discontinuity set piecewise smooth approximation

+ Preserves discontinuities

– Computationally challenging

Blurred and noisy image Tikhonov regularization Mumford-Shah Induced edge set K

15 / 20

Mumford-Shah regularization for manifold-valued data

Mumford-Shah model for manifold-valued data

arg minu,K

γ|K |+α

q

∫Ω\K|Du(x)|qdx +

1p

∫Ω

d(u(x), f(x))pdx

Prior work: Level-set active contour approach for two-phase variant(manifold-valued Chan-Vese model) (Wang/Vemuri ’04)

Our approach: (i) Finite difference discretization of Blake-Zisserman-type

arg minx∈Mm×n

1p

dp(x, f) + α

R∑s=1

ωsΨas (x),

with the directional penalty

Ψa(x) =∑

i,j

ψ(x(i,j)+a , xij) and ψ(w, z) =1q

min(γq/α, d(w, z)q).

Weinmann, Demaret, S. Mumford-Shah and Potts regularization for manifold-valued data. J. Math. Imag. Vis. (2016)16 / 20

Mumford-Shah regularization for manifold-valued data

Mumford-Shah model for manifold-valued data

arg minu,K

γ|K |+α

q

∫Ω\K|Du(x)|qdx +

1p

∫Ω

d(u(x), f(x))pdx

Prior work: Level-set active contour approach for two-phase variant(manifold-valued Chan-Vese model) (Wang/Vemuri ’04)

Our approach: (i) Finite difference discretization of Blake-Zisserman-type

arg minx∈Mm×n

1p

dp(x, f) + α

R∑s=1

ωsΨas (x),

with the directional penalty

Ψa(x) =∑

i,j

ψ(x(i,j)+a , xij) and ψ(w, z) =1q

min(γq/α, d(w, z)q).

Weinmann, Demaret, S. Mumford-Shah and Potts regularization for manifold-valued data. J. Math. Imag. Vis. (2016)16 / 20

Algorithm for 2D Mumford-Shah problem

(ii) Penalty decomposition

arg minx1 ,...,xR

R∑s=1

ωspRαΨas (xs) + dp(xs , f) + µk dp(xs , xs+1).

with an increasing coupling parameter µk

(iii) Block coordinate descent

xk+11 ∈ arg min

xpRω1αΨa1 (x) + dp(x, f) + µk dp(x, xk

R ),

xk+12 ∈ arg min

xpRω2αΨa2 (x) + dp(x, f) + µk dp(x, xk+1

1 ),

...

xk+1R ∈ arg min

xpRωRαΨaR (x) + dp(x, f) + µk dp(x, xk+1

R−1 ).

univariate Mumford-Shah-type problems

17 / 20

Exact solver for univariate Mumford-Shah problems

(iv) Solve univariate Mumford-Shah problems

arg minx

1p

n∑i=1

d(xi , fi)p +α

q

∑i<J(x)

d(xi , xi+1)q + γ|J(x)|,

where J is the jump set of x.

Theorem (Weinmann/Demaret/S. ’16)

In a Cartan-Hadamard manifold, there is an algorithm that produces a globalminimizer for the univariate Mumford-Shah problem.

Sketch of proof:

Employ dynamic programming strategy (Friedrich et al. ’08)

subproblems of TVq-Lp type

Solve subproblems using CPPA.

18 / 20

Results for diffusion tensor images

Result using proposed splitting method

Corpus callosum of human brain(CAMINO project)

Mumford-Shah regularization using proposed method

19 / 20

Summary

Algorithm for TV regularization with circle-valued data in 1D

Exact solver for univariate L1-TV

Worst case complexity O(N2)

Algorithms for TV regularization for manifold-valued images

Cyclic proximal point strategy with explicit computation of proxies

Globally convergent algorithm for Cartan-Hadamard manifolds

Satisfactory results for (non-convex) circle-valued data

Algorithms for Mumford-Shah regularization with manifold-valued data

Based on penalty decomposition, dynamic programming, and CPPA

No restrictions on discontinuity curve (e.g. number of segments)

20 / 20

Thank you!