22
GAMM-Mitteilungen, 19 August 2014 Shearlets: Theory and Applications Gitta Kutyniok 1, , Wang-Q Lim 1, ∗∗ , and Gabriele Steidl 2, ∗∗∗ 1 Institut f¨ ur Mathematik, Technische Universit¨ at Berlin, 10623 Berlin, Germany 2 Fachbereich Mathematik, Technische Universit¨ at Kaiserslautern, 67653 Kaiserslautern, Germany Received XXXX, revised XXXX, accepted XXXX Published online XXXX Key words Applied harmonic analysis, frames, imaging science, inverse problems, multi- scale systems, shearlets, sparse approximation, wavelets MSC (2010) 42C40, 42C15, 65J22, 94A12, 65F22, 65T60, 94A20, 68U10, 90C25, 15B52 Many important problem classes are governed by anisotropic features such as singularities concentrated on lower dimensional embedded manifolds, for instance, edges in images or shock fronts in solutions of transport dominated equations. While the ability to reliably cap- ture and sparsely represent anisotropic structures is obviously the more important the higher the number of spatial variables is, the principal difficulties arise already in two spatial dimen- sions. Since it was shown that the well-known wavelets are not capable of efficiently encoding such anisotropic features, various directional representation systems were suggested during the last years. Of those, shearlets are the most widely used today due to their optimal sparse approximation properties in combination with their unified treatment of the continuum and digital realm, leading to faithful implementations. This article shall serve as an introduction to and a survey about shearlets. Copyright line will be provided by the publisher 1 Introduction Recent advances in modern technology have created a new world of enormous and multi- dimensional data structures. In medical imaging, seismic imaging, astronomical imaging, computer vision, and video processing among others, the capabilities of modern computers and high-precision measuring devices have generated 2D, 3D, and even higher dimensional data sets of sizes that were infeasible just a few years ago. The need to efficiently handle such diverse types and huge amounts of data initiated an intense study in developing efficient multivariate encoding methodologies in the applied harmonic analysis research community. 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing in the follow- ing way. Given a class of data C in a Hilbert space H, and a carefully constructed associated * E-mail: [email protected], Phone: +49 30 314 25758, Fax: +49 30 314 27364 ** E-mail: [email protected], Phone: +49 30 314 25751, Fax: +49 30 314 27364 *** E-mail: [email protected], Phone: +49 631 205 5320, Fax: +49 631 205 5309 Copyright line will be provided by the publisher

Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

GAMM-Mitteilungen, 19 August 2014

Shearlets: Theory and Applications

Gitta Kutyniok 1,∗, Wang-Q Lim1,∗∗, andGabriele Steidl2,∗∗∗

1 Institut fur Mathematik, Technische Universitat Berlin, 10623 Berlin, Germany2 Fachbereich Mathematik, Technische Universitat Kaiserslautern, 67653 Kaiserslautern,

Germany

Received XXXX, revised XXXX, accepted XXXXPublished online XXXX

Key words Applied harmonic analysis, frames, imaging science, inverse problems, multi-scale systems, shearlets, sparse approximation, waveletsMSC (2010) 42C40, 42C15, 65J22, 94A12, 65F22, 65T60, 94A20, 68U10, 90C25, 15B52

Many important problem classes are governed by anisotropicfeatures such as singularitiesconcentrated on lower dimensional embedded manifolds, forinstance, edges in images orshock fronts in solutions of transport dominated equations. While the ability to reliably cap-ture and sparsely represent anisotropic structures is obviously the more important the higherthe number of spatial variables is, the principal difficulties arise already in two spatial dimen-sions. Since it was shown that the well-known wavelets are not capable of efficiently encodingsuch anisotropic features, various directional representation systems were suggested duringthe last years. Of those, shearlets are the most widely used today due to their optimal sparseapproximation properties in combination with their unifiedtreatment of the continuum anddigital realm, leading to faithful implementations. This article shall serve as an introductionto and a survey about shearlets.

Copyright line will be provided by the publisher

1 Introduction

Recent advances in modern technology have created a new world of enormous and multi-dimensional data structures. In medical imaging, seismic imaging, astronomical imaging,computer vision, and video processing among others, the capabilities of modern computersand high-precision measuring devices have generated 2D, 3D, and even higher dimensionaldata sets of sizes that were infeasible just a few years ago. The need to efficiently handlesuch diverse types and huge amounts of data initiated an intense study in developing efficientmultivariate encoding methodologies in the applied harmonic analysis research community.

1.1 The Applied Harmonic Analysis Approach to Data Processing

The area of applied harmonic analysis approaches problems of data processing in the follow-ing way. Given a class of dataC in a Hilbert spaceH, and a carefully constructed associated

∗E-mail: [email protected], Phone: +49 30 314 25758, Fax: +49 30 314 27364∗∗E-mail: [email protected], Phone: +49 30 314 25751, Fax: +49 30 314 27364∗∗∗E-mail: [email protected], Phone: +49 631 205 5320, Fax: +49 631 205 5309

Copyright line will be provided by the publisher

Page 2: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

2 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory andApplications

collection of functions(ψλ)λ∈Λ ⊂ H with Λ being a countable indexing set. On the one handthe data isdecomposedby

C ∋ f 7→ (〈f, ψλ〉)λ∈Λ (1)

with the coefficient sequence enabling an analysis of the dataf such as detection of importantfeatures, e.g., singularities. On the other hand the data isexpandedas

f =∑

λ∈Λ

c(f)λψλ for all f ∈ C (2)

with the coefficient sequence preferably being sparse in thesense of rapid decay to allowefficient encoding of the dataf . It is well-known thatc(f)λ = 〈f, ψλ〉 for all λ ∈ Λ in thesituation of an orthonormal basis. However, often redundancy is desirable which leads to thenotion of a frame.

1.2 Redundancy comes into Play

Frame theory provides a general framework for representation systems(ψλ)λ∈Λ to enable re-dundancy – in the sense of non-unique expansions – yet retaining stability. A system(ψλ)λ∈Λ

is called aframefor H, if there exist constants0 < A ≤ B <∞ such that

A‖f‖22 ≤∑

λ∈Λ

|〈f, ψλ〉|2 ≤ B‖f‖22 for all f ∈ H.

In caseA = B = 1, it is referred to as aParsevsal frame; and in fact, such Parseval framesare the most general systems which allowc(f)λ = 〈f, ψλ〉 for all λ ∈ Λ.

Letting theframe operatorbe defined by

S : H → H, f 7→∑

λ∈Λ

〈f, ψλ〉ψλ,

and setting(ψλ)λ∈Λ := (S−1ψλ)λ∈Λ to be thecanonical dual frame, we obtain the followingformulas for the reconstruction of somef ∈ H from the decomposition (1) and for a particularcoefficient sequence in the expansion (2):

f =∑

λ∈Λ

〈f, ψλ〉ψλ and f =∑

λ∈Λ

〈f, ψλ〉ψλ for all f ∈ H,

respectively. For further information on frame theory, we refer to [10].

1.3 Wavelets

One of the most well known examples of frames forL2(Rd) are wavelet frames [18].

Definition 1.1 Forψ1, . . . , ψL ∈ L2(Rd), the associatedwavelet systemis defined by

ψℓj,m = 2dj2 ψℓ(2j · −m) : j ∈ Z,m ∈ Z

d, ℓ = 1, . . . , L.

Copyright line will be provided by the publisher

Page 3: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 3

In fact, finitely many generating functionsψ1, . . . , ψL can be constructed so that the asso-ciated wavelet system forms an orthonormal basis (more generally, a frame) forL2(Rd). Inthis case, the functionsψ1, . . . , ψL are referred to aswavelets. The parameterj is typicallyreferred to asscaleand the parameterm asposition.

On the more theoretical side, wavelet systems are proven to highly efficiently approximateL2-functions which are smooth except for finitely many point singularities. On the moreapplication oriented side, wavelets are, for instance, used in the new compression standardJPEG2000 [61].

1.4 From Wavelets to Shearlets

While wavelets can be shown to derive optimal decay rates of the error of bestN -term approx-imation for functions governed by point singularities, it is evident that starting from dimen-sion 2 anisotropic singularities appear such as curve-likesingularities. In fact, for instance,in imaging science the discontinuity curves of an image are one of the most important fea-tures, in particular, since the human visual system is designed to react strongly to directionalfeatures. But also solutions of, for instance, transport dominated equations are governed byshock fronts. In general, one can argue that many important classes of multivariate problemsare governed by anisotropic features. Moreover, in high-dimensional data most information istypically contained in lower-dimensional embedded manifolds, thereby also presenting itselfas anisotropic features.

Although wavelets are highly efficient in capturing point singularities, they are isotropicbasis (more generally, frame) elements due to the isotropicscaling matrix with a dyadic scal-ing factor2j. Therefore, they are only able to deliver a suboptimal approximation rate ofsuch data. The anisotropic structures can also only be distinguished by location and orien-tation/direction. This indicates that our way of analyzingand representing the data shouldcapture not only location, but also directional information – requiring anisotropically shapedelements – to overcome the drawback of wavelet systems. These two different approximationschemes with isotropic and anisotropic basis elements are illustrated in Figure 1.

(a) (b)

Fig. 1 (a) Approximation of a curve by isotropic basis elements. (b) Approximation of a curve byanisotropic basis elements.

Inspired by this observation, numerous approaches for efficiently representing directionalfeatures of multivariate data have been proposed in the areaof applied harmonic analysis.A perfunctory list includes:steerable pyramidby Simoncelli et al. [60],directional filter

Copyright line will be provided by the publisher

Page 4: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

4 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory andApplications

banksby Bamberger and Smith [6],2D directional waveletsby Antoine et al. [2],curveletsby Candes and Donoho [8],contourletsby Do and Vetterli [19],bandeletsby LePennec andMallat [58], andshearlets[35, 52]. Shearlet systems are among the most versatile and suc-cessful systems, since they, in particular, satisfy the following list of properties commonlydesired for directional representation systems:

(I) Underlying group structure for availability of deep mathematical tools.(II) Provably optimal sparse approximations of anisotropic features.(II) Compactly supported analyzing elements for high spatial localization.

(IV) The continuum and digital realm should be treated uniformly.(V) The associated decomposition should admit a fast implementation.

This article shall serve as an introduction to and a survey about shearlets. For additionalinformation, we refer to the book chapter [45].

1.5 Outline

This survey paper is organized as follows. The continuous and discrete shearlet systems andtheir associated transforms are introduced and discussed in Sections 2 and 3, respectively.Section 4 is devoted to the optimal sparse approximation properties of shearlets for anisotropicfeatures. A faithful digitalization is presented in Section 5. Finally, in Section 6 severalimaging applications and numerical results are discussed.

2 Continuous Shearlet Systems

In the setting of continuous parameter sets, it is well-known that the short time Fourier trans-form and the wavelet transform are associated with square-integrable representations of theWeyl-Heisenberg group [33] and the affine group [30,31], respectively. Among all directionaltransforms mentioned in the introduction, the continuous shearlet transform1 is outstanding,because it stems also from a square-integrable group representation, namely from the so-calledshearlet group. Therefore, powerful tools of group representation theorycan be exploited tostudy, for instance, reconstruction properties or the resolution of wavefront sets. In the follow-ing, we briefly introduce continuous shearlet systems and the associated continuous shearlettransform using this general group theoretical framework.

The main idea in the construction of (continuous) shearlet systems is to select very few gen-erating functions to which scaling operators for differentresolution levels, shearing operatorsas a means to change the orientation, and translation operators to select different positions areapplied. The first two operators are based on thedilation operator, which for someM ∈ R2×2

is defined by

DM : L2(R2) → L2(R2), (DMf)(x) 7→ | det(M)|−1/2f(M−1x).

SelectingM to be aparabolic scaling matrixAa, a ∈ R∗ := R \ 0, or ashearing matrix

Ss, s ∈ R, given by

Aa =

(

a 00 |a|1/2

)

and Ss =

(

1 s0 1

)

, (3)

1‘Continuous’ in the sense of continuous parameter sets.

Copyright line will be provided by the publisher

Page 5: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 5

yield the set of scaling and shearing operators, respectively. We remark that the choice ofchanging the orientation by shearing is superior to rotation due to the fact that in the discreteversion the matricesSk, k ∈ Z, leave the digital gridZ2 invariant. Finally, fort ∈ R2, thetranslation operatoris defined by

Tt : L2(R2) → L2(R2), (Ttf)(x) 7→ f(x− t).

This leads to the definition of continuous shearlet systems.

Definition 2.1 Forψ ∈ L2(R2), thecontinuous shearlet systemSH(ψ) is defined by

SH(ψ) = ψa,s,t := TtDSsDAaψ = a−3

4ψ(A−1a S−1

s ( · −t)) : a ∈ R∗, s ∈ R, t ∈ R

2.

2.1 The Shearlet Group and the Continuous Shearlet Transform

The just defined continuous shearlet system arises from the(full) shearlet groupS := R∗ ×R× R2 endowed with the group operation

(a, s, t) (a′, s′, t′) = (aa′, s+√

|a|s′, t+ SsAat′).

This is a locally compact group with left Haar measuredµ(a, s, t) = da/|a|3dsdt. The proofsare given in [11] and in [13] for higher dimensions. For the later, notice that the previousdefinitions can be naturally extended toL2(Rd) by settingAa,γ = diag(a, sgn(a)|a|γ I) withγ > 0 and appropriate generalizations of the shearing and translation operator as well as of thegroup operation. Instead of shearing matrices, triangularToeplitz matrices were considered in[14]. Recently, it was shown that the shearlet group is isomorphic to the extended Heisenberggroup and to a subgroup of the symplectic group [17] which leads to interesting new insightsinto the structure of the shearlet transform and related function spaces.

Letting the unitary representationπ : S → U(L2(R2)) from S into the group of unitaryoperators onL2(R2) be defined byπ(a, s, t)ψ := ψa,s,t, the continuous shearlet systemSH(ψ) can be written asSH(ψ) = π(a, s, t)ψ : (a, s, t) ∈ S. This mapping is alsosquare-integrable, i.e., it is irreducible and there exists a nontrivialadmissiblefunctionψ ∈L2(R2) fulfilling, for all f ∈ L2(R2), theadmissibility condition

S

|〈f, π(a, s, t)ψ〉|2 dµ(a, s, t) <∞.

It can be proven thatψ ∈ L2(R2) is admissible if and only if

R2

|ψ(ξ)|2|ξ1|2

dξ <∞, (4)

whereψ denotes the Fourier transform ofψ. For a more general approach we refer to [27].

Definition 2.2 A functionψ ∈ L2(R2) satisfying (4) is called ashearletand the operatorSψ : L2(R2) → L2(S) given by

Sψf(a, s, t) = 〈f, π(a, s, t)ψ〉

is referred to ascontinuous shearlet transform.

Copyright line will be provided by the publisher

Page 6: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

6 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory andApplications

Intuitively, Sψ mapsf to its coefficientsSψf(a, s, t) associated with thescalevariablea ∈ R∗, theorientationvariables ∈ R, and thelocationvariablet ∈ R2. In the general groupcontext, this transform is known asvoice transform.

Shearlets of both band-limited and compactly supported type exist and are well-studied,see [11,15,42,50]. One prominent example of a shearlet is theclassical shearletψ ∈ L2(R2),which is a band-limited function introduced in [52] and was the first shearlet extensivelystudied.

Example 2.3 A classical shearletψ ∈ L2(R2) is defined by

ψ(ξ) = ψ(ξ1, ξ2) = ψ1(ξ1) ψ2(ξ2ξ1),

whereψ1 ∈ L2(R) is a wavelet, i.e., it satisfies the discrete Calderon condition given by

j∈Z

|ψ1(2−jξ)|2 = 1 for a.e.ξ ∈ R,

with ψ1 ∈ C∞(R) and suppψ1 ⊆ [− 5

4,− 1

4] ∪ [ 1

4, 54], andψ2 ∈ L2(R) is a ‘bump function’,

namely

1∑

k=−1

|ψ2(ξ + k)|2 = 1 for a.e.ξ ∈ [−1, 1],

satisfyingψ2 ∈ C∞(R) and suppψ2 ⊆ [−1, 1]. Figure 2(a) illustrates the support of theassociated Fourier transformψ.

Group theoretic considerations lead to the following result about invertibility of the con-tinuous shearlet transform.

Theorem 2.4([11]) Letψ ∈ L2(R2) be admissible. ThenSψ is an isometry.

2.2 Shearlet Coorbit Spaces

With a square-integrable group representation at hand, there exists a very natural link to an-other useful concept, namely the coorbit space theory introduced by Feichtinger and Grochenigin a series of papers [24–26, 32]. By means of coorbit space theory, it is possible to derivein a canonical way associated scales of smoothness spaces. In this framework, the smooth-ness of functions is measured by the decay of the associated voice transform. Moreover, by acareful discretization of the representation, it is possible to obtain (Banach) frames for thesesmoothness spaces.

The continuous shearlet transform does fulfill all necessary conditions for the application ofcoorbit space theory, which leads to novel canonical smoothness spaces, theshearlet coorbitspaces[12] together with their atomic decompositions and (Banach) frames for these spaces.In fact, shearlet coorbit spaces are related to Besov spacesin the sense that certain subspacesof shearlet coorbit spaces can be embedded into homogeneousBesov spaces and traces ofshearlet coorbit spaces onto certain hyperplanes are contained in Besov spaces or again inshearlet coorbit spaces, see [15,16]. These subspaces resemble the concept of shearlets on thecone which is considered in the sequel.

Copyright line will be provided by the publisher

Page 7: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 7

2.3 Cone-Adapted Continuous Shearlet Systems and Resolution of the Wave-front Set

Despite the advantages of the pure group theory-based approach, one drawback is the fact thatthe associated continuous shearlet systems do exhibit a directional bias. For an illustration ofthe problem, we refer to Figure 2(b).

C1

C2

C3

R

C4

(a) (b) (c)

Fig. 2 (a) Support of the Fourier transform of a classical shearlet. (b) Fourier domain support of severalelements of the shearlet system, for different values ofa ands. (c) The conesC1 – C4 and the centeredrectangleR in the frequency domain.

This problem can be resolved by partitioning the Fourier domain into four conic regionsand considering the low frequency part separately; see Figure 2(c). This leads to the follow-ing variant of continuous shearlet systems with the function φ being responsible for the lowfrequency part as well asψ andψ for the horizontal (C1 ∪ C3) and vertical (C2 ∪ C4) conicregions, respectively.

Definition 2.5 For φ, ψ, ψ ∈ L2(R2), the cone-adapted continuous shearlet systemisdefined bySH(φ, ψ, ψ) = Φ(φ) ∪Ψ(ψ) ∪ Ψ(ψ), where

Φ(φ) = φt := φ(· − t) : t ∈ R2,

Ψ(ψ) = ψa,s,t = a−3

4ψ(A−1a S−1

s ( · − t)) : a ∈ (0, 1], |s| ≤ 1 + a1/2, t ∈ R2,

Ψ(ψ) = ψa,s,t := a−3

4 ψ(A−1a S−T

s ( · − t)) : a ∈ (0, 1], |s| ≤ 1 + a1/2, t ∈ R2,

andAa = diag(a1/2, a).

Similar as in the situation of continuous shearlet systems,also for cone-adapted continu-ous shearlet systems an associated transform can be defined and isometry conditions can bedetermined (cf. [45]). Moreover, by considering those pairs (s, t), for which |〈f, ψa,s,t〉| or|〈f, ψa,s,t〉| does not decay rapidly asa → 0, the wavefront set of a distributionf can beprecisely determined [46].

3 Discrete Shearlet Systems

We now turn to discrete shearlet systems, which can be regarded as arising from their contin-uous counterparts by discretizing the associated set of parameters. Thus, in a similar way we

Copyright line will be provided by the publisher

Page 8: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

8 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory andApplications

can derive a discrete shearlet system as well as a cone-adapted variant. Since in applicationsonly the later one is used, we focus in the sequel on this type of shearlet systems.

3.1 2D Shearlets

Recalling the definitions of parabolic scaling and shearingfrom (3), a common discrete ver-sion of those operations areA2j andSk with j, k ∈ Z. Applying the discretization(a, s, t) 7→(2−j ,−k2−j/2, A−1

2jS−1

k cm) (for c ∈ (R+)2 to add flexibility), which essentially arises from

the previously mentioned coorbit theory, to Definition 2.5 yields the following definition.

Definition 3.1 Let c = (c1, c2) ∈ (R+)2. Forφ, ψ, ψ ∈ L2(R2) thecone-adapted discrete

shearlet systemSH(φ, ψ, ψ; c) = Φ(φ; c1) ∪Ψ(ψ; c) ∪ Ψ(ψ; c) is defined by

Φ(φ; c1) = φm := φ(· −m) : m ∈ c1Z2,

Ψ(ψ; c) = ψj,k,m := 23

4jψ(SkA2j · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉,m ∈McZ

2,Ψ(ψ; c) = ψj,k,m := 2

3

4jψ(SkA2j · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉,m ∈ McZ

2,

whereMc = diag(c1, c2) andMc = diag(c2, c1).

If SH(φ, ψ, ψ; c) is a frame forL2(R2), we refer toφ as ascaling functionandψ andψas(discrete) shearlets. Also notice that 2D shearlets are in the spatial domain of size2−j ×2−j/2, which shows that they will become ‘needle-like’ (hence more and more anisotropic)asj → ∞.

3.1.1 Band-Limited Shearlets

Band-limited shearlets have the advantage of allowing the construction of Parseval framesfor L2(R2). Moreover, some applications such as seismology have a natural band-limitedstructure which makes the associated shearlet systems preferable.

Constructions of band-limited shearlets were introduced in [35] with refinements containedin [37]. The so-calledclassical shearlets, which we already discussed in Example 2.3, areperhaps the most widely used version. In this case, typically the generatorψ is chosen asψ(x1, x2) = ψ(x2, x1). The tiling of Fourier domain which they induce is illustrated inFigure 3.

Fig. 3 Tiling of Fourier domain induced by the cone-adapted discrete shearlet system associated withclassical shearlets.

Copyright line will be provided by the publisher

Page 9: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 9

For this particular choice, it was proved in [35, Thm. 3] thatthe associated cone-adapteddiscrete shearlet systemSH(φ, ψ, ψ; (1, 1)) with a slight modification of the boundary ele-ments and with an appropriate scaling functionφ for the centered rectangle forms a Parsevalframe forL2(R2).

3.1.2 Compactly Supported Shearlets

Despite the advantageous functional analytic properties of classical, and in general band-limited shearlets often applications seek high spatial localization. This makes compactly sup-ported shearlets desirable. However, it is still not clear whether a cone-adapted discrete shear-let system associated with compactly supported shearlets can be introduced, which forms aParseval frame forL2(R2). The best known construction forms a frame forL2(R2) withnumerically proven ratio of the frame bounds of approximately 4, see [42].

3.1.3 Shearlet Frames

In [42], a general framework was introduced which provides sufficient conditions for a cone-adapted discrete shearlet system to form a frame with theoretical estimates for the associatedframe bounds, which we will now describe.

For this, we first define

Ω0 = ξ ∈ R2 : max|ξ1|, |ξ2| ≤ 1

2, Ω1 = ξ ∈ R

2 : 1

2< |ξ2| < 1, |ξ2|/|ξ1| < 1,

and assume that

ess infξ∈Ω0

|φ(ξ)| > 0 and ess infξ∈Ω1

|ψ(ξ)| > 0. (5)

The above conditions ensure that

ess infξ∈R2

|φ(ξ)|2 +∑

j≥0

|k|≤⌈2j/2⌉

(

|ψ(STk A2−j ξ)|2 + | ˆψ(STk A2−j ξ)|2)

> 0

whereψ(x1, x2) = ψ(x2, x1). In fact, this inequality can be used to derive a lower framebound forSH(φ, ψ, ψ, c). The following result shows that any functionsφ, ψ ∈ L2(R2)

(and ψ(x1, x2) = ψ(x2, x1)), whose Fourier transform decays fast enough with sufficientvanishing moments and satisfies the lower bound conditions (5) indeed generate a shearletframeSH(φ, ψ, ψ, c).

Theorem 3.2([42]) Letφ, ψ ∈ L2(R2) be functions such that

φ(ξ1, ξ2) ≤ C1 ·min 1, |ξ1|−γ ·min 1, |ξ2|−γ and

|ψ(ξ1, ξ2)| ≤ C2 ·min1, |ξ1|α ·min 1, |ξ1|−γ ·min 1, |ξ2|−γ, (6)

for some positive constantsC1, C2 < ∞ andα > γ > 3. Defineψ(x1, x2) = ψ(x2, x1)and assume thatφ, ψ satisfy (5). Then, there exists some positive constantc∗ such thatSH(φ, ψ, ψ, c) forms a frame forL2(R2) for anyc = (c1, c2) with maxc1, c2 ≤ c∗.

Obviously, band-limited shearlets satisfy condition (6).More interestingly, also a largeclass of compactly supported functions satisfies this condition, and we refer to the construc-tions in [42].

Copyright line will be provided by the publisher

Page 10: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

10 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

3.2 3D Shearlets

Continuous shearlet systems in higher dimensions than dimension 2 were first consideredin [14] in the pure group theory-based approach. In the following, we now focus as beforeon a ‘cone-adapted variant’ in the 3D situation. These so-called pyramid-adapted shearletsystems again treat directions in a uniform way similar as their cone-adapted 2D counterparts.The continuous (band-limited) version of such was first introduced in [36], and it was shownthat the location and the local orientation of the boundary set of certain three-dimensionalsolid regions can be precisely identified by the associated transform.

We now turn to the discrete setting and the definition of pyramid-adapted discrete shearletsystems. These systems will be generated by scaling according to theparaboloidal scalingmatricesA2j , A2j or A2j , j ∈ Z, and directionality will be encoded by theshear matricesSk, Sk, or Sk, k = (k1, k2) ∈ Z2, defined by

A2j = diag(2j , 2j/2, 2j/2), A2j = diag(2j/2, 2j, 2j/2), A2j = diag(2j/2, 2j/2, 2j)

and

Sk =

1 k1 k20 1 00 0 1

, Sk =

1 0 0k1 1 k20 0 1

, Sk =

1 0 00 1 0k1 k2 1

,

respectively. The translation lattices will be defined through the following matrices:Mc =

diag(c1, c2, c2), Mc = diag(c2, c1, c2), andMc = diag(c2, c2, c1), wherec1, c2 > 0. Similaras for cone-adapted shearlet systems, we now partition Fourier domain into a rectangularregion and six pyramids as shown in Figure 4.

ξ2

ξ1

ξ3

ξ2

ξ1

ξ3

Fig. 4 The partition of Fourier domain by four of the six pyramids.

These considerations are now made precise in the following definition, where each part ofthe system is responsible for covering one set of pyramids.

Definition 3.3 For c = (c1, c2) ∈ (R+)2, thepyramid-adapted discrete shearlet system

SH(φ, ψ, ψ, ψ; c) generated byφ, ψ, ψ, ψ ∈ L2(R3) is defined by

SH(φ, ψ, ψ, ψ; c) = Φ(φ; c1) ∪Ψ(ψ; c) ∪ Ψ(ψ; c) ∪ Ψ(ψ; c),

Copyright line will be provided by the publisher

Page 11: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 11

where

Φ(φ; c1) = φm := φ(· −m) : m ∈ c1Z3,

Ψ(ψ; c) = ψj,k,m := 2jψ(SkA2j · −m) : j ≥ 0, ‖k‖∞ ≤ ⌈2j/2⌉,m ∈McZ3,

Ψ(ψ; c) = ψj,k,m := 2jψ(SkA2j · −m) : j ≥ 0, ‖k‖∞ ≤ ⌈2j/2⌉,m ∈ McZ3,

Ψ(ψ; c) = ψj,k,m := 2jψ(SkA2j · −m) : j ≥ 0, ‖k‖∞ ≤ ⌈2j/2⌉,m ∈ McZ3.

The construction of pyramid-adapted shearlet systemsSH(φ, ψ, ψ, ψ; c) runs along thelines of the construction of the 2D cone-adapted shearlet systems described in the previoussubsection. Similar as in the 2D case, also compactly supported shearlet frames with con-trolled frame bounds can be constructed [50]. We also wish tomention that this constructioncan be easily generalized to even higher dimensions.

Let us finally discuss whether this is the most natural extension to 3D. For this, notice that3D shearlets are in the spatial domain of size2−j×2−j/2×2−j/2, which shows that they willbecome ‘plate-like’ asj → ∞. Figure 5 illustrates some examples of 2D and 3D shearletsin the spatial domain. One could also exploit the scaling matrix A2j = diag (2j , 2j, 2j/2)

with similar changes forA2j and A2j . This would lead to ‘needle-like’ shearlet elementsinstead of the ‘plate-like’ elements introduced in Definition 3.3. In particular, those shearletsseem well adapted for 1D singularities, while ‘plate-like’shearlets seem better suited for 2Dsingularities. More generally, it is possible to even consider non-paraboloidal scaling matricesof the formAj = diag (2j , 2αj, 2βj) for 0 < α, β ≤ 1. One drawback of allowing suchgeneral scaling matrices is the lack of fast algorithms for non-dyadic multiscale systems, andin case of the ‘needle-like’ shearlet construction the lackof frame properties. On the otherhand, the parametersα andβ allow us to precisely shape the shearlet elements, ranging fromvery ‘plate-like’ to very ‘needle-like’, according to the application at hand, i.e., choosing theshape of the shearlet which is the best‘fit’ for the geometric characteristics of the considereddata. We refer to [28,50] for more details on these general settings.

(a) (b)

Fig. 5 Examples of (a) 2D shearlets and (b) 3D shearlets in the spatial domain.

4 Quest for Optimal Sparse Approximations

As mentioned in the introduction, shearlets were introduced with the goal to provide optimallysparse approximations of anisotropic features. We now firstmake the notion of ‘anisotropic

Copyright line will be provided by the publisher

Page 12: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

12 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

features’ mathematically precise by introducing a mathematical model for such. Let us startwith the most basic definition of this class which was also historically first, stated in [20]. Weallow ourselves to phrase this definition in thed-dimensional situation ford = 2, 3, hence tocombine the 2D with the 3D setting from [50].

Definition 4.1 For fixedν > 0 andd = 2, 3, theclassE2ν (R

d) of cartoon-like functionsisthe set of functionsf : Rd → C of the form

f = f0 + f1χB,

whereB ⊂ [0, 1]d andfi ∈ C2(Rd) with suppfi ⊂ [0, 1]

d and‖fi‖C2 ≤ 1 for eachi = 0, 1.For dimensiond = 2, we assume that∂B is a closedC2-curve with curvature bounded byν,and, ford = 3, the discontinuity∂B shall be a closedC2-surface with principal curvaturesbounded byν.

Donoho then proved the following benchmark result in [20] for d = 2; its extension tod = 3 was derived in [50]. We wish to mention that in fact the initial statements are moregeneral than the one for frames we present here.

Theorem 4.2([20, 50]) Let d = 2, 3, and let(ψλ)λ∈Λ be a frame forL2(Rd). Then theoptimal asymptotic approximation error off ∈ E2(Rd) is given by

‖f − fN‖22 ≤ C ·N− 2

d−1 asN → ∞, wherefN =∑

λ∈ΛN

cλψλ

is the (non-linear) bestN -term approximation andC > 0.

Thus a system satisfying this decay conditions can be justifiably called a system which pro-videsoptimally sparse approximationsof cartoon-like functions. The following result showsthat shearlets indeed satisfy this benchmark result up to a (negligible) log-factor. We wishto mention that this result is in fact part of a much more general framework, the frameworkof parabolic molecules[34], which provides approximation results for very general systemsbased on parabolic scaling.

Theorem 4.3([47]) Letc > 0, and letφ, ψ, ψ ∈ L2(R2) be compactly supported. Supposethat, in addition, for allξ = (ξ1, ξ2) ∈ R2, the shearletψ satisfies

(i) |ψ(ξ)| ≤ C1 ·min1, |ξ1|α ·min1, |ξ1|−γ ·min1, |ξ2|−γ and

(ii)∣

∂∂ξ2

ψ(ξ)∣

∣ ≤ |h(ξ1)| ·(

1 + |ξ2||ξ1|

)−γ

,

whereα > 5, γ ≥ 4, h ∈ L1(R), andC1 is a constant, and suppose that the shearletψ

satisfies (i) and (ii) with the roles ofξ1 andξ2 reversed. Further, suppose thatSH(φ, ψ, ψ; c)

forms a frame forL2(R2). Then, for anyν > 0, the shearlet frameSH(φ, ψ, ψ; c) provides(almost) optimally sparse approximations of functionsf ∈ E2

ν (R2) in the sense that there

exists someC > 0 such that

‖f − fN‖22 ≤ C ·N−2 · (logN)3 asN → ∞,

wherefN is the nonlinear N-term approximation obtained by choosingthe N largest shearletcoefficients off .

Similar results were shown forE2ν (R

3) using pyramid-adapted discrete shearlets, see [50].

Copyright line will be provided by the publisher

Page 13: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 13

5 Digital Shearlet Transforms

The first implementation for computing shearlet coefficients has been developed in [22]. Thisnumerical algorithm implements the discrete shearlet transform associated with band-limitedshearlets. For this, digital shearlet filter coefficients are obtained by approximating the inverseFourier transform of band-limited shearlets discretized on the 2D pseudo-polar grid where theshear operatorSk is given as a translation.

We now discuss implementation strategies for computing shearlet coefficients associatedwith the cone-adapted discrete shearlet system based on compactly supported shearlets, asintroduced in Section 3. One main focus will be on deriving a digitization which is faithfulto the continuum setting. We refer to [55] and [56] for more details on this approach. Forthis, we will only consider shearletsψj,k,m for the horizontal cone, i.e., belonging toΨ(ψ, c).Notice that the same procedure can be applied to compute the shearlet coefficients for thevertical cone, i.e., those belonging toΨ(ψ, c), except for switching the order of variables.

We first define a separable shearlet generatorψ ∈ L2(R2) and an associated scaling func-tion φ ∈ L2(R2) as follows. Letφ1 ∈ L2(R) be a compactly supported 1D scaling functionsatisfying

φ1(x1) =∑

n1∈Z

h(n1)√2φ1(2x1 − n1)

for some appropriately chosen filterh. An associated compactly supported 1D waveletψ1 ∈L2(R) can then be defined by

ψ1(x1) =∑

n1∈Z

g(n1)√2φ1(2x1 − n1),

where againg is an appropriately chosen filter. Then, the separable shearlet generatorψ isdefined to be

ψ(x1, x2) = ψ1(x1)φ1(x2).

The filtersh andg are required to be chosen so thatψ satisfies a certain decay condition(cf. [42]) to guarantee a stable reconstruction from the shearlet coefficients. With those filtersh andg, we define the scaling and wavelet filtershj andgj by the Fourier coefficients of thetrigonometric polynomialsHj andGj , j > 0, defined by

Hj(ξ1) =

j−1∏

k=0

n1∈Z

h(n1)e−2πi2kn1ξ1

and

Gj(ξ1) = Hj−1(ξ1)(

n1∈Z

g(n1)e−2πi2j−1n1ξ1

)

,

respectively.For the signalf ∈ L2(R2) to be analyzed, we now assume that, forJ > 0 fixed, f is of

the form

f(x) =∑

n∈Z2

fJ(n)2Jφ(2Jx1 − n1, 2

Jx2 − n2), (7)

Copyright line will be provided by the publisher

Page 14: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

14 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

whereφ(x1, x2) = φ1(x1)φ1(x2). Note that the scaling coefficients(fJ(n))n∈Z2 in (7) canbe viewed as sample values off – in fact fJ(n) ≈ f(2−Jn) with appropriately chosenφ.Now aiming towards a faithful digitization of the shearlet coefficients〈f, ψj,k,m〉 for j =0, . . . , J − 1, we first observe that

〈f, ψj,k,m〉 = 〈f(Sk·2−j/2(·)), ψj,0,m(·)〉. (8)

WLOG we will from now on assume thatj/2 is integer; otherwise either⌈j/2⌉ or ⌊j/2⌋would need to be taken. Our observation (8) shows us in fact precisely how to digitize theshearlet coefficients〈f, ψj,k,m〉, namely by applying the discrete separable wavelet trans-form associated with the anisotropic sampling matrixA2j to the sheared version of the dataf(Sk·2−j/2 (·)). This however requires – compare the assumed form off given in (7) – thatf(Sk·2−j/2 (·)) is contained in the scaling space

VJ = 2Jφ(2J · −n1, 2J · −n2) : (n1, n2) ∈ Z

2.

It is easy to see that, for instance, if the shear parameterk · 2−j/2 is non-integer, this isunfortunately not the case. The true reason for this failureis that the shear matrixSk·2−j/2

doesnotpreserve the regular grid2−JZ2 in VJ , i.e.,

Sk·2−j/2(Z2) 6= Z2.

In order to resolve this issue, we consider the new scaling spaceV kJ+j/2,J defined by

V kJ+j/2,J = 2J+4/jφk(2J+j/2 · −n1, 2

J · −n2) : (n1, n2) ∈ Z2

whereφk(·) = φ(Sk·). We remark that the scaling spaceV kJ+j/2,J is obtained by refining the

regular grid2−JZ2 along thex1-axis by a factor of2j/2. With this modification, the new grid2−J−j/2Z× 2−JZ is now invariant under the shear operatorSk·2−j/2 . In fact, we have

2−J−j/2Z× 2−JZ = S2−j/2k(2−J−j/2

Z× 2−JZ).

This allows us to rewritef(Sk·2−j/2(·)) in (8) in the following way.

Lemma 5.1 ([55]) Let ↑ 2j/2 and∗1 denote the 1D upsampling operator by a factor of2j/2 and the 1D convolution operator along thex1-axis, respectively. Then, we obtain

f(S2−j/2k(x)) =∑

n∈Z2

fJ(Skn)2J+j/4φk(2

J+j/2x1 − n1, 2Jx2 − n2),

wherefJ(n) = ((fJ )↑2j/2 ∗1 hj/2)(n).The second term to be digitized in (8) is the shearletψj,k,m itself with k = 0, for which

the following result can be employed.

Lemma 5.2([55]) Retaining the notations and definitions from this section, we obtain

ψj,0,m(x) =∑

d∈Z2

gJ−j(d1 − 2J−jm1)hJ−j/2(d2 − 2J−j/2m2)2Jφ(2Jx− d).

Copyright line will be provided by the publisher

Page 15: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 15

As already indicated before, we will make use of the discreteseparable wavelet transformassociated with an anisotropic scaling matrix, which, forj1, j2 > 0 as well asc ∈ ℓ(Z2), wedefine by

Wj1,j2(c)(n1, n2) =∑

m∈Z2

gj1(m1−2j1n1)hj2(m2−2j2n2)c(m1,m2), (n1, n2) ∈ Z2.

Finally, Lemmata 5.1 and 5.2 yield the following digitilized form of the shearlet coefficients〈f, ψj,k,m〉.

Theorem 5.3([55]) Retaining the notations and definitions from this section, and letting↓ 2j/2 be 1D downsampling by a factor of2j/2 along the horizontal axis, we obtain

〈f, ψj,k,m〉 =WJ−j,J−j/2

((

(fJ(Sk·) ∗ Φk) ∗1 hj/2)

↓2j/2

)

(m),

whereΦk(n) = 〈φk(·), φ(· − n)〉 for n ∈ Z2, andhj/2(n1) = hj/2(−n1).

Theorem 5.3 shows that for anyf ∈ L2(R2) of the form (7), the shearlet coefficients〈f, ψj,k,m〉 can be faithfully computed in the digital domain, similar towavelet theory. Formore details as well as a digitalization of the inverse shearlet transform, we refer to [51,55,56].

In fact, this framework can be extended to allow a nonseparable shearletψ, which signifi-cantly improves the stability and directional selectivityof the shearlet transform [56]. Figure 6shows the refined essential support of a nonseparable shearlet in Fourier domain as comparedto a separable shearlet. The Matlab software package calledShearLab2 provides code for theshearlet transforms associated with those separable and nonseparable shearlets.

(a) (b)

Fig. 6 (a) Separable shearlet. (b) Nonseparable shearlet.

Finally, the 3D shearlet transform can also be implemented faithfully by following theideas outlined in this section. For details we refer to [51, 56] and for an implementation toShearLab.

6 Applications

Recently, sparse and redundant representation modeling has received much attention due toits efficiency in imaging sciences, see, e.g., [1, 23]. The facts that shearlets provide (almost)

2http://www.shearlab.org

Copyright line will be provided by the publisher

Page 16: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

16 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

optimally sparse approximations of anisotropic features and that most imaging data and, inparticular, natural images are governed by such features – the neurons in the human visualcortex are most sensitive to such edge-like structures – nowallow the utilization of shearletsin this general methodological approach. In the sequel, we will discuss some exemplaryapplications. For other applications, we refer the interested reader to the book chapter [38].

For shearlet-based algorithms currently three toolboxes are available, which also containcode for some of the presented applications: (i) Local Shearlet Toolbox3 was the first shearlettransform implementation for band-limited shearlets, fordetails see [22]. (ii) ShearLab4 con-sists of three different implementations: The band-limited shearlet transform implemented onboth the pseudo-polar grid and the Cartesian grid, as well asa compactly supported shearlettransform, for details see [51,55]. (iii) Fast Finite Shearlet Transform (FFST)5 provides fullyfinite and translation invariant shearlet implementation with band-limited shearlets, for detailssee [41].

6.1 Image Separation

The task of separating imagesf ∈ L2(R2) into morphologically distinct components hasrecently attracted a lot of attention, see, e.g., [3,4,62].One particular instance of this problemis the separation of curve-like and point-like objects, which arises in several applications. Inneurobiological imaging, it would, for instance, be desirable to separate spines (point-likeobjects) from dendrites (curve-like objects) in order to analyze them independently aimingto detect characteristics of Alzheimer disease. In astronomical imaging, astronomers wouldoften like to separate stars from filaments for further analysis. Other applications appear inmaterial sciences.

The mathematically precise problem can be formulated as follows: Given a point-like ob-ject u01 ∈ L2(R2) and a curve-like objectu02 ∈ L2(R2), recoveru01 andu02 from knowledgeof f = u01 + u02. This is obviously an ill-posed inverse problem.

The key idea now is to approach this problem by compressed sensing methodologies, see,e.g., [44]. This mainly consists in utilizing two sparsifying systems each for one of the com-ponents followed by solving anℓ1 minimization problem, since theℓ1 norm promotes sparsity.In our situation, we choose wavelets and shearlets for the point-like and curve-like objects,respectively. The separation is then performed by solving the following convex optimizationproblem:

argminu1,u2

‖(〈u1, φλ〉)λ∈Λ‖1+‖(〈u2, ψγ〉)γ∈Γ‖1 subject to ‖u1+u2−f‖2 ≤ σ, (9)

where(φλ)λ∈Λ and(ψγ)γ∈Γ are wavelets and shearlets, respectively, andσ is the noise level.Figure 7 shows the numerical results of the separation scheme (9) with wavelets and shearletsfor an artificial image (cf. [48]). Note that curve-like objects are well separated by shearletseven for regions where a curve has high curvature. A theoretical analysis of this separationscheme can be found in [21].

3http://www.math.uh.edu/ ˜ dlabate/software.html4http://www.shearlab.org5http://www.mathematik.uni-kl.de/imagepro/software/

Copyright line will be provided by the publisher

Page 17: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 17

In [39] the separation approach was applied in material science, more precisely to polymercomposites filled with glass fibers that are used in many industrial branches. Their character-istics strongly depend on the orientation of the fibers (lines), hence the orientation distributionof the fibers is of great importance. On the other hand also thepoints—they represent defectsin the material—have a strong influence on the performance ofthe material. For a furtheranalysis, the both components have to be analyzed separately. The three-dimensional imagesare obtained via full-field optical coherence microscopy. Figure 8 shows the decompositionresults. For technical details, we refer to [39].

(a) (b) (c)

Fig. 7 Image separation: (a) Noisy input image (curves+points). (b) Separated image (curves). (c) Sep-arated image (points).

(a) (b) (c)

Fig. 8 Image separation of an FF-OCT image of polymer composites filled with glass fibers: (a) Origi-nal image (b) Lines/Fibers (c) Points/Defects. (cf. [39])

6.2 Image/Video Inpainting

Inpainting is the process of reconstructing lost or deteriorated parts of images and videos.The term ‘inpainting’ first appeared in [7], but earlier workon disocclusions was done, e.g.,in [9, 57]. In this respect also interpolation, approximation, and extrapolation problems maybe viewed as inpainting problems. Inpainting is a very active field of research which wastackled by various approaches. For a nice overview, we referto [29].

Copyright line will be provided by the publisher

Page 18: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

18 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

(a) (b) (c)

Fig. 9 Image inpainting: (a) Original image (512×512). (b) Corrupted image with 80% missing pixels.(c) Inpainted image.

(a) (b) (c)

Fig. 10 Video inpainting: (a) Original frame of a(192 × 192 × 192) video sequence. (b) Corruptedframe with 80% missing pixels. (c) Inpainted frame.

The mathematically precise problem can be formulated as follows: Given an image domainΩ0 = 1, . . . ,m × 1, . . . , n, the inpainting regionΩ ⊂ Ω0 is the subset where the pixelvaluesf(i, j), (i, j) ∈ Ω are unknown. The (noiseless) inpainting problem consists of findinga functionu onΩ0 from dataf given onΩ = Ω0\Ω such thatu is a suitable extension off toΩ0. With (ψγ)γ∈Λ a shearlet system, the shearlet based model reads as follows:

argminu

‖(〈u, ψλ〉)λ∈Λ‖1 subject to u|Ω = f |Ω.

Again, the backbone of this approach comes from the theory ofcompressed sensing.

In [28, 43], a shearlet based inpainting scheme has been theoretically analyzed. Figures 9and 10 show inpainting results for an image and, using 3D shearlets, a video. For moreextensive test results as well as comparisons with other state-of-the-art methods we refer to[51].

Copyright line will be provided by the publisher

Page 19: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 19

6.3 Sparse Sampling of Fourier Data

One major problem of imaging procedures in clinical research and health care such as Mag-netic Resonance Imaging (MRI) or X-ray Computed Tomography(X-ray CT) is the very slowdata acquisition. In those applications, the data acquisition process can also be modeled bytaking Fourier measurements. One main challenge is therefore to significantly reduce thenumber of Fourier samples without degrading the quality of image reconstruction by devel-oping novel sampling and reconstruction schemes. For this,we consider

argminu

‖(〈u, ψλ〉)λ∈Λ‖1 subject to ‖PΩFu− f‖2 ≤ σ

where(ψγ)γ∈Λ is a shearlet system,Ω is a set of sampling points in Fourier domain,PΩ isthe corresponding sampling operator,F is the Fourier operator, andσ is the noise level.

One major task is to design a sampling setΩ, which allows a successful image reconstruc-tion using only a few number of Fourier samples. Figure 11 shows the reconstructed imageby a shearlet-based sparse sampling scheme using only5% Fourier coefficients of the originalimage. A detailed description and a theoretical analysis ofthis shearlet-based sparse samplingscheme using methods from compressed sensing can be found in[49].

(a) (b) (c)

Fig. 11 Fourier Sampling: (a) Original image (512 × 512). (b) Fourier mask for the subsamplingoperatorPΩ containing5% sampling points in Fourier domain. (c) Reconstructed image.

6.4 Image Segmentation

Segmentation is a fundamental task in image processing which plays a role in many prepro-cessing steps. Recently, convex relaxation methods for image multi-labeling were addressedby several authors [5,53,54,59,63]using a total variationregularizer. For texture segmentationcontaining coherent curves, shearlet regularization showed superior results [40].

More precisely, to segment a given color imagef = (f(i, j))m,ni,j=1 ∈ Rm,n into L seg-ments, we assign a codebookc = (c1, . . . , cL) to the image which contains the central colorsck, k = 1, . . . , L of the segments (this codebook can be updated during the iterative segmen-tation process). For each pixel(i, j) we are searching for the probabilitiespk(i, j) such thatf(i, j) lies in segmentk. In other words,p(i, j) = (p1(i, j), . . . , pL(i, j)) is in the probability

Copyright line will be provided by the publisher

Page 20: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

20 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

simplex

= π ∈ RL :

L∑

k=1

πk = 1, π1, . . . , πL ≥ 0.

Thenf(i, j) is assigned to those segment with the highest probability. Letsk := (sk(i, j))m,ni,j=1

:= (‖f(i, j)− ck‖22)m,ni,j=1. We then findpk, k = 1, . . . , L as the solution of

argminp

L∑

k=1

(〈sk, pk〉+ α‖(〈pk, ψλ〉)λ∈Λ‖1) subject to p(i, j) ∈ ∀(i, j)

with a suitable regularization parameterα > 0. Figure 12 shows an example of a segmentedimage using the described method. For details, we refer to [40].

(a) (b)

Fig. 12 Image segmentation: (a) Noisy image of a clown-surgeon fish,(b) Segmented image with foursegments.

References

[1] M. Aharon and M. Elad, SIAM J. Imaging Sci.1, 228–247 (2008).[2] J. P. Antoine, P. Carrette, R. Murenzi, and B. Piette, Signal Process.31, 241–272 (1993).[3] J.-F. Aujol, G. Aubert, L. Blanc-Feraud, and A. Chambolle, J. Math. Imaging Vision22, 71–88

(2005).[4] J.-F. Aujol, G. Gilboa, T. Chan, and S. Osher, J. Math. Imaging Vision67, 111–136 (2006).[5] E. Bae, J. Yuan, and X.-C. Tai, Int. J. Comput. Vision92, 112–129 (2011).[6] R. H. Bamberger and M. J. T. Smith, IEEE Trans. Signal Process.40, 882–893 (1992).[7] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, Proceedings of SIGGRAPH 2000 (New

Orleans, USA, 2000), pp. 417–424.[8] E. J. Candes and D. L. Donoho, Comm. Pure Appl. Math.56, 216–266 (2004).[9] V. Caselles, J.-M. Morel, and C. Sbert, IEEE Trans. ImageProcess.7, 376–386 (1998).

[10] O. Christensen, An Introduction to Frames and Riesz Bases (Birkhauser, Boston, 2003).[11] S. Dahlke, G. Kutyniok, P. Maass, C. Sagiv, H.-G. Stark,and G. Teschke, Int. J. Wavelets

Multiresolut. Inf. Process.6, 157–181 (2008).[12] S. Dahlke, G. Kutyniok, G. Steidl, and G. Teschke, Appl.Comput. Harmon. Anal.27, 195–214

(2009).

Copyright line will be provided by the publisher

Page 21: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

gamm header will be provided by the publisher 21

[13] S. Dahlke, G. Steidl, and G. Teschke, J. Fourier Anal. Appl. 16, 340–354 (2010).[14] S. Dahlke and G. Teschke, in: Group Theory: Classes, Representation and Connections, and

Applications, edited by C. W. Danellis (Nova Publishers, 2010), chap. 6.[15] S. Dahlke, G. Steidl, and G. Teschke, J. Fourier Anal. Appl. 17, 1232–1255 (2011).[16] S. Dahlke, S. Hauser, G. Steidl, and G. Teschke, Monatsh. Math.169, 15–32 (2013).[17] S. Dahlke, F. de Mari, E. de Vito, S. Hauser, G. Steidl, and G. Teschke, ArXiv-Preprint 1404.4545

(2014).[18] I. Daubechies, Ten Lectures on Wavelets (SIAM, Philadelphia, PA, 1992).[19] M. N. Do and M. Vetterli, IEEE Trans. Image Process.14, 2091–2106 (2005).[20] D. L. Donoho, Constr. Approx.17, 353–382 (2001).[21] D. L. Donoho and G. Kutyniok, Comm. Pure Appl. Math.66, 1–47 (2013).[22] G. Easley, D. Labate, and W.-Q Lim, Appl. Comput. Harmon. Anal.25, 25–46 (2008).[23] M. Elad, IEEE Signal Process. Lett.19, 922–928 (2012).[24] H. G. Feichtinger and K. Grochenig, J. Funct. Anal.86, 307–340 (1989).[25] H. G. Feichtinger and K. Grochenig, Monatsh. Math.108, 129–148 (1989).[26] H. G. Feichtinger and K. Grochenig, in: Wavelets and Their Applications, edited by G. Beylkin,

R. R. Coifman, and I. Daubechies (Jones and Bartlett, Boston, 1992), pp. 353–376.[27] H. Fuhr and R. Raisi-Tousi, ArXiv preprint 1407.0824 (2014).[28] M. Genzel and G. Kutyniok, SIAM J. Imaging Sci. to appear.[29] C. Guillemot and O. Le Meur, IEEE Signal Process. Mag.31, 127–144 (2014).[30] A. Grossmann, J. Morlet, and T. Paul, J. Math. Phys.26, 2473–2479 (1985).[31] A. Grossmann, J. Morlet, and T. Paul, Ann. L’Institute H. Poincare Math. Phys.45, 293–309

(1986).[32] K. Grochenig, Monatsh. Math.112, 1–42 (1991).[33] K. Grochenig, Foundations of Time–Frequency Analysis (Birkhauser, Boston, 2001).[34] P. Grohs and G. Kutyniok, Found. Comput. Math.14, 299-337 (2014).[35] K. Guo, G. Kutyniok, and D. Labate, in: Wavelets and Splines (Athens, GA, 2005), edited by C.

Guanrong and M.-J. Lai (Nashboro Press, Nashville, TN, 2006), pp. 189–201.[36] K. Guo and D. Labate, Appl. Comput. Harmon. Anal.30, 231–242 (2011).[37] K. Guo and D. Labate, Math. Model. Nat. Phenom.8, 82–105 (2013).[38] G. Easley and D. Labate, in: Shearlets: Multiscale Analysis for Multivariate Data, edited by G.

Kutyniok and D. Labate (Birkhauser, Boston, 2012), pp. 283–325.[39] B. Heise, S. E. Schausberger, S. Hauser, B. Plank, D. Salaberger, E. Leiss-Holzinger, and D.

Stifter, Opt. Fiber Technol.5, 403–410 (2012).[40] S. Hauser and G. Steidl, Int. J. Comput. Math.90, 62–81 (2013).[41] S. Hauser, ArXiv Preprint 1202.1773 (2012).[42] P. Kittipoom, G. Kutyniok, and W.-Q Lim, Constr. Approx. 35, 21–72 (2012).[43] E. J. King, G. Kutyniok, and X. Zhuang, J. Math. Imaging Vision48, 205–234 (2014).[44] G. Kutyniok, GAMM Mitteilungen36, 79-101 (2013).[45] G. Kutyniok and D. Labate, in: Shearlets: Multiscale Analysis for Multivariate Data, edited by

G. Kutyniok and D. Labate (Birkhauser, Boston, 2012), pp. 1–38.[46] G. Kutyniok and D. Labate, Trans. Amer. Math. Soc.361, 2719–2754 (2009).[47] G. Kutyniok and W.-Q Lim, J. Approx. Theory163, 1564–1589 (2011).[48] G. Kutyniok and W.-Q Lim, in: Curves and Surfaces (Avignon, France, 2010), edited by J.-

D. Boissonnat, P. Chenin, A. Cohen, C. Gout, T. Lyche, M.-L. Mazure, and L. L. Schumaker,Lecture Notes in Computer Science6920(Spinger, 2012), pp. 416–430.

[49] G. Kutyniok and W.-Q Lim, in preparation.[50] G. Kutyniok, J. Lemvig, and W.-Q Lim, SIAM J. Math. Anal.44, 2962–3017 (2012).[51] G. Kutyniok, W.-Q Lim, and R. Reisenhofer, ArXiv Preprint 1402.5670 (2014).[52] D. Labate, W.-Q Lim, G. Kutyniok, and G. Weiss, WaveletsXI, SPIE Proc.5914(SPIE, Belling-

ham, WA, 2005), pp. 254–262.[53] J. Lellmann, F. Becker, and Ch. Schnorr, Proceedings of 12th International Conference on Com-

puter Vision (Kyoto, Japan, 2009), pp. 646–653.

Copyright line will be provided by the publisher

Page 22: Shearlets: Theory and Applications - TU Berlin · 1.1 The Applied Harmonic Analysis Approach to Data Processing The area of applied harmonic analysis approaches problems of data processing

22 G. Kutyniok, W.-Q Lim, and G. Steidl: Shearlets: Theory and Applications

[54] J. Lellmann and Ch. Schnorr, SIAM J. Imaging Sci.4, 1049–1096 (2011).[55] W.-Q Lim, IEEE Trans. Image Process.19, 1166–1180 (2010).[56] W.-Q Lim, IEEE Trans. Image Process.22, 2056–2065 (2013).[57] S. Masnou and J.-M. Morel, Proceedings of the 1998 IEEE International Conference on Image

Processing (Chicago, IL, 1998), pp. 259–263.[58] E. L. Pennec and S. Mallat, IEEE Trans. Image Process.14, 423–438 (2005).[59] T. Pock, A. Chambolle, D. Cremers, and H. Bischof, IEEE Conference on Computer Vision and

Pattern Recognition (Miami, FL, 2009), pp. 810–817.[60] E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, IEEE Trans. Inform. Theory,

38, 587–607 (1992).[61] A. Skodras, C. Christopoulos, and T. Ebrahimi, IEEE Signal Processing Mag.18, 36–58 (2001).[62] J.-L Starck, M. Elad, and D. Donoho, IEEE Trans. Image Proc.14, 1570–1582 (2005).[63] C. Zach, D. Gallup, J. M. Frahm, and M. Niethammer, Proceedings of Vision, Modeling, and

Visualization (Konstanz, 2008), pp. 243–252.

Copyright line will be provided by the publisher