IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XXX, NO. …IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XXX, NO. YYY, MONTH 2011 3 reduced to their eigenspace, as indicator functions [37],

Copyright (c) 2011 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing [email protected].

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XXX, NO. YYY, MONTH 2011 1

Tracking monotonically advancing boundaries inimage sequences using graph cuts and recursive

kernel shape priorsJoshua Chang, KC Brennan, Tom Chou

Abstract—We introduce a probabilistic computer vision tech-nique to track monotonically advancing boundaries of objectswithin image sequences. Our method incorporates a noveltechnique for including statistical prior shape information intograph-cut based segmentation, with the aid of a majorization-minimization algorithm. Extension of segmentation from singleimages to image sequences then follows naturally using sequentialBayesian estimation. Our methodology is applied to two unrelatedsets of real biomedical imaging data, and a set of synthetic images.Our results are shown to be superior to manual segmentation.

Index Terms—Contour tracking, Level set method, Particle fil-ter, Gaussian Process, Bayesian Vision, Graph cut, Segmentation,Shape Prior, Shape Statistics, Cortical Spreading Depression,Wound healing assay, Optical Intrinsic Signal Imaging, GaussianMarkov Random Fields

I. INTRODUCTION

BOUNDARY tracking is a common problem in manyimaging applications, particularly in biomedical imaging

where the dynamics of a wavefront or the boundary of a regionof interest provide insights into the underlying physiologicalprocesses. Many processes are characterized by monotonicboundary movement, where a boundary crosses each point atmost once. In biology and medicine, examples of such movingboundaries include cardiac impulse waves [1], tumors [2],extent of cerebral infarct [3], glial cell signal waves [4], andspreading depression waves [5].

The motion of monotonically advancing boundaries is de-scribed by the eikonal equation [6, 7]

V (s)|∇T (s)| = 1, (1)

where V (s) : Rd → R+ is the speed of the interfacealong its normal when it crosses a point s ∈ Rd, andT (s) : Rd → R+ is the time when the boundary first reachess. For collapsing boundaries, V (s) is strictly negative, and−V (s)|∇T (s)| = 1. Computationally, several algorithms forapproximating solutions to the eikonal equation exist, notablythe fast marching method [6], a quick O(N logN) solver.This method has an interpretation as a special boundary valuecase of the more general initial value level set method [7] fordescribing boundary motion.

J. Chang is with the UCLA Department of BiomathematicsT. Chou is with the UCLA Departments of Biomathematics and Mathemat-

icsK.C. Brennan is with the University of Utah Department of NeurologyCopyright c© 2010 IEEE. Personal use of this material is permitted.

However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected].

Fig. 1. Level-set embedding of shapes Embedding of a region Ω into asigned distance function φΩ defined on a discrete lattice. φΩ takes valuesaccording to the signed Euclidean distance from the boundary ∂Ω, withnegative values inside Ω. The boundary ∂Ω is implicitly embedded as thezero-level set of φΩ. Shown is a 2-d lattice, however, level set method worksfor Rd with any arbitrary d.

The level set method, as proposed by Osher and Sethian[8], is another approach to computing boundary motion. Itdescribes the evolution in time t of a closed hyper-surface Γ(t)of dimension d−1 that bounds an evolving region Ω(t) ⊂ Rd.Instead of directly tracking the hyper-surface Γ(t), one embedsΓ(t) in an object of higher dimension known as a level set.The interface is embedded as the zero-level set of a functionφΩ(t)(s) : Rd → R, which for every s ∈ Rd is the signedEuclidean distance from s to the boundary of Ω(t). φΩ(t)(s)is by convention negative if s ∈ Ω(t), and positive if s ∈Rd \ Ω(t). The level set equation

∂φΩ

∂t+ V (s)|∇φΩ| = 0 (2)

then describes the interface undergoing movement with speedV (s) along its normal vectors. Computationally, one typicallydiscretizes the level set function (Fig 1) and solves Eq 2 usingfinite differences. The main advantages of this approach are itsfreedom from parameters, and its ease of handling topologicalchanges, especially in comparison to Lagrangian methods ofcurve evolution.

Both fast marching and level set methods have been usefulfor computationally describing moving interfaces in a varietyof physical applications; the speed of the interface encodesthe interesting physics of the observed process, and can be afunction of any number of observed or unobserved covariates.

mailto:[email protected]




For example, a curvature-dependent speed has been usedto model flame propagation [8]. In biomedical applications,Malladi et al. [9] and others [2, 10, 11] have used level setsto model tumor growth. Sermesant et al. [12] used level setsto model cardiac electrophysiology. Recently, Wolgemuth andZajac [13] used level sets to model cell motility.

Here, we develop a theory for the segmentation and trackingof boundaries that move according to the eikonal equation. Weregularize our tracking by recursively estimating the positionof the boundary, using a statistical model for the boundaryspeed based upon its observed history. The recursive estima-tion is weighted against evidence of the boundary in the imagesequence within a Bayesian filtering framework.

We provide three applications of our method. First, weidentify a boundary in a synthetic image sequence, validatingour method against a “ground truth.” Second, we track corticalspreading depression waves in in-vivo image sequences. Cor-tical spreading depression (CSD) arises in many pathologiessuch as stroke, brain trauma, epilepsy, and migraine. It ischaracterized by a slow-moving, concentric, traveling waveof runaway excitation propagating through contiguous regionsof brain gray-matter. Lastly, we demonstrate the ability ofour method to detect the boundary of a collapsing unhealedwound region. Here, a “wound” is scratched into a monolayerof cultured cells and the closing wound edge formed by theleading cells is tracked.

A. Related prior work in literature

There has been much work in tracking boundary motionin image sequences, using a variety of methods. One methodis the use of level set-based active contours on the gradientfield of image sequences [14, 15, 16]. In these methods, theminimum of a phenomenologically defined energy definesthe contour positions. Other successful methods have utilizedstatistical models in order to define energies. Mansouri [17]developed a Bayesian tracking method where motion infor-mation is not computed. Many other authors have developedvariants of Kalman filtering [18, 19, 20], and more generally,Bayesian filtering [21, 22, 23, 23].

Bayesian filtering is a sequential technique for estimating anunknown probability density as it evolves in time [24]. Whenthe underlying state space is first-order Markovian, Bayesianfiltering amounts to recursively predicting the next state giventhe observations up to the current state, and updating the pre-diction of the next state as the next observation comes in. Theprediction of the next state acts as a prior. This prediction stepentails propagating the probability distribution for the currentstate through the system dynamics (generally nonlinearly) togenerate a predictive distribution for the next state. A popularvariant of Bayesian filtering is the particle filter [25]. In theparticle filter, one draws a weighted sample of states from thecurrent state posterior, and then propagates each sample stateaccording to the system dynamics to construct the predictiondistribution.

We now take a moment to provide some backgroundabout image segmentation, the labeling of regions in images.Bayesian filtering for boundary detection can be thought of

as recursive segmentation regularized by a motion model. Tosegment a closed subset of a region into a foreground set,Mumford and Shah [26] proposed an optimization problem.Suppose that an image I : S ⊂ R2 → R contains pixelsthat represent two regions, foreground (Ω), and background(∆ = S \ Ω), separated by a closed boundary Γ = ∂Ω.The optimal segmentation is the construction of a piecewise-smooth image I0 that is found by minimizing an energyfunctional involving pixel-mismatches and a boundary-lengthpenalty. Motivated by the idea of minimizing this functional,a class of methods known loosely as active contour methodswere born [27]. Within this class of methods, level set ap-proaches have achieved possibly the most success. Chan andVese [28] were among the first to employ the level set methodto minimize a Mumford-Shah like energy, where they usedlevel sets to describe a gradient descent minimization. Thesemethods, however, are plagued by slow convergence, and byreliance on the placement of an initial labeling.

Fig. 2. Embedding of an image into a graph In the graph cuts framework,pixels are nodes in a graph. Connections between neighboring pixels are made,as well as connections between each pixel and two special nodes called thesource (foreground) and sink (background). Depicted is an eight-neighborsystem, where s and u are neighbors. These connections are weightedaccording to the strength of the association between two nodes to the sameclass (either foreground or background). A segmentation is found by cuttingthe graph into two parts separating the source and sink such that the edgeweights along the cut are minimal. The segmented foreground then consistsof the pixels that have intact edges with the source (depicted as solid blacklines).

Recently, computer vision researchers have employedgraph-cuts based optimization to solve the segmentation prob-lem. In this method, a segmentation corresponds to a two-coloring of a graph, where pixels constitute nodes with col-oring representing membership to either the foreground orbackground sets. This coloring is typically found in low orderpolynomial time using a maximum-flow/minimum-cut algo-rithm [29]. However, graph cut methods can be applied onlyon a restricted class of energies [30, 31]. Despite much work inthe field, it is challenging to incorporate prior knowledge aboutshapes into an energy that is minimizable within the graph-cuts framework. For static image segmentation of generalshapes, popular priors in use are the star-shaped prior [32],elliptical shape prior [33], and the compact shape prior [34].For more application-specific segmentation, where the shape isknown, researchers have expressed shapes in kernel principalcomponent space [35, 36], where parameterized shapes are




reduced to their eigenspace, as indicator functions [37], orimplicitly embedded in levelsets [38, 39, 40].

Shape priors are useful for tracking applications since theycan represent models for the motion of objects in an image se-quence. Dynamical and statistical shape priors are particularlyuseful since they can account for uncertainty of an object’smotion. There are many strategies for modeling this motionand generating associated shape priors. One approach doesnot seek to model the motion of the object, but instead relieson propagation of contours by gradient flow [14, 15]. Suchmethods however are incapable of detecting large changesin a boundary that correspond to shape changes, and arecomputationally expensive due to the reliance on solving aPDE.

Another common approach is to identify features of anobject to track from a set of training images. For exam-ple, PCA-based methods [41] model deformation of featureslearned from a set of training templates. Other studies directlyparameterize a particular target object [19], or contour [21, 22].All of the aforementioned tracking methods have had successin many tracking applications, but they are not suitable foruse in our desired applications without major modifications.These methods are restrictive in that they are intended to trackobjects that are expected to retain their overall structure in animage sequence.

B. Motivation for our method

Fig. 3. Examples of monotonic boundary movement. Shown are stillsfrom three image sequences depicting monotonic boundary motion. Top tworows are cortical spreading depression hemodynamical waves. Bottom row isa wound undergoing healing. The noise characteristics in these images, andthe shapes that develop, differ markedly.

In identifying the motion of a boundary evolving accordingto the eikonal equation, we seek to solve a subtly differenttracking problem. As a boundary evolves, the region enclosedby the boundary need not retain any particular shape (Fig 3).Furthermore, it is not sufficient to only know the approximatelocation of the object, we seek precise discrimination of theobject’s boundaries. Finally, we are also motivated by the goalof online estimation, where we do not wish to rely on relativelyslow techniques such as PDE-based energy minimization.

To solve these problems, our method introduces the conceptof using recursive statistical shape priors in graph cuts seg-mentation. With the aid of a majorization-minimization step,

we show how one can iteratively use graph cuts to arrive ata segmentation that takes into consideration an ensemble ofpredicted interface positions. This ensemble is constructed bymodeling the speed of the interface using its observed history.The predicted ensemble is weighted against evidence of theinterface position in the image, providing regularization thatmakes our method robust to real-world noise encountered inbiomedical imaging.

II. MATHEMATICAL APPROACH

Our overall approach is to develop a recursive Bayesianfilter to regularize segmentation of the evolving boundary. Ourfilter stochastically samples the possible motion of the bound-ary by evolving it against speeds sampled from a stochasticautoregressive model, whose parameters we recursively infer.The boundary is embedded implicitly as level sets in the arrivaltime function of Eq 1, and is propagated by the fast marchingmethod. For the motion of the interface, we make only theloose assumption that its speed is locally correlated in space.

Given a set of interface positions ΩkKk=0, measured attimes tkKk=0, we calculate the past speed of the interface bybilinearly interpolating its past arrival times over the discreteimage lattice, and then solving the eikonal equation (Eq 1)using second order upwind finite differences [42]. Denotinga vector containing these past speeds V0, we predict futurespeeds V using our stochastic model. Propagation of theinterface against stochastic samples of V provides us withestimates of the future position of the interface.

In this section we will restrict our discussion to strictlypositive speeds (advancing fronts). The same method appliesfor strictly negative speeds (collapsing fronts).

A. Motion prediction by Gaussian Process modeling

We use a Gaussian Markov Random Field (GMRF) to modelthe speed. With it, we can represent the spatial correlationof the speed of the boundary, where we expect speeds ofthe interface at neighboring locations to be close. In essence,we use a GMRF to penalize large spatial variations in ourpredicted speeds.

A GMRF is a stochastic field that follows a multivariateGaussian distribution, and is Markovian with respect to agraph. The random variates of a GMRF constitute the nodesof a graph G, with covariance matrix Σ such that an entryin the precision matrix Q = Σ−1 is nonzero if and only ifthere is an edge connecting the two corresponding randomvariates. A GMRF has a sparse precision matrix in that itscorresponding graph is not complete – we assume that we canpredict the speed at a position given only the speeds at nearbylocations. GMRFs have been shown to approximate arbitrarycovariance structures well in practice [43, 44], even when thecorrelation range is much larger than the size of the Markovianneighborhoods. We say that the speed V : R2 → R+ of theinterface as it crosses over s ∈ Rd is

V (s) = X(s)β + η(s),

where η : R2 → R is a GMRF. Of perhaps most interest,X(s) is a 1× p vector of any known linear predictors whose




influence is unknown. We determine the influence of thesepredictors by inferring the p× 1 vector of coefficients β. Forexample, if one has an anatomical labeling distinguishing twodifferent tissue types in an imaging field, X1(s) could be anindicator of one of the tissue types. The speed of the interfaceis then V (s) = β0+X1(s)β1+η(s), so that the expected speedof the interface is β0 + β1 inside the region, and β0 outside.Under this construct, it is possible to infer the coefficient β1,and ascertain if it is distinct from zero, thereby statisticallytesting whether the interface travels through the two regionsat different speeds. It is worthy to note that the predictors Xi

are also allowed to be nonlinear functions of any auxiliaryvariables. It is convenient to infer the GMRF using data fromall locations simultaneously, so we define a matrix X to bean n× p matrix, where each of the n rows corresponds to anobservation point (in our case, speed at a particular location),and each row has p entries, corresponding to known predictorvariables. Such a matrix is commonly known as a covariatematrix. X(s) refers to the row in X corresponding to point s.

It may be illuminating to write the model in the followinghierarchical form

V (s) = X(s)β + η(s) (3)

η|τ2η ∼ N (0,Q) (4)

β, τ2η ∼ N

(β;β0, τ

2ηP)G(τ2

η ; aη, bη). (5)

N (x;µ, τ2) represents a Gaussian distribution on randomvariable x with mean µ and precision τ2. G(y; a, b) representsa statistician’s gamma distribution on random variable y withshape a and rate b. For convenience, throughout this paperwe parametrize Gaussian distributions with precisions τ2

(·) =

1/σ2(·) rather than the more commonly encountered variances

σ2(·). Eq 3 states that the speed is a linear composition of the

covariates and their associated coefficients, plus a normallydistributed spatial noise term with distribution given in Eq 4. Qhas entries Qsu that are a function of the distance between twopoints s, and u, hence it is an n×n matrix. We set Q = τ2

ηR,where R encodes spatial correlation between locations, and τ2

η

is a scalar parameter independent of η. For simplicity we willassume we know R; we use the 5 × 5 Markovian Gaussianprocess given by Rue and Tjelmeland [43], to yield a sparsebanded R.

In Eq 5, we place the normal-gamma conjugate priordistribution on the parameters β, and τ2

η . The hyperparametersaη and bη are set small (0.001) to be weakly informative [45].P represents the precision of the prior knowledge of β, scaledby τ2

η . This model bears more than superficial resemblance tocommon spatial interpolation methods used in geostatistics. Inparticular, this model can be considered a Bayesian variant ofuniversal kriging [46]. Inference on this model can be per-formed exactly and analytically as a generalized least squaresproblem [47]. It is important to note that we desire positivevalues for speeds V (s). In this model we do not explicitlyenforce positivity of V ; however, if µ is large enough, as inthe applications we discuss, fields with negative values of Vhave negligible measure. We simply discard any samples ofV that have negative values. In this section, we provide themaximum a-posterior estimates for β, and V. Please see the

supplementary document for details of the derivations.Suppose one has a n× 1 vector V0 of observed speeds, at

locations with associated n × n spatial correlation encodingmatrix R0, and n × p covariate matrix X0. Then one mayperform maximum-a-posterior inference to predict speeds atunobserved locations. The model given in Eqs 3–5 yields aposterior t-distribution on β with location

β =(XT

0 R0X0 + P)−1 (

XT0 R0V0 + Pβ0

), (6)

and scale

Aβ =1

n+ 2aη

(XT

0 R0X0 + P)−1

×(

2bη + βT0 Pβ0 + VT0 R0V0

−βT (

XT0 R0X0 + P

)β). (7)

Let V be a vector of unknown speeds that we would like toestimate at m new locations. Suppose that these locations areassociated with the known m×p covariate matrix X. Denotingthe m×m spatial correlation encoding matrix for these newpoints R, and a m×n matrix that encodes correlation betweenthese points and the original n points as U, one finds that theposterior V is t-distributed with location

V =

mean︷︸︸︷Xβ −

spatially correlated noise︷︸︸︷R−1U

(V0 −X0β

)(8)

and scale

A =(X + R−1UX0

)Aβ

(X + R−1UX0

)T+

1

n+ 2aηR−1

(2bη + βT0 Vβ0 + VT

0 R0V0

−βT (

XT0 R0X0 + P

)β). (9)

One sees from Eq 8 that the mean predicted speed is a sumof a mean term and a spatially correlated term that appears asa convolution. Entries in the convolution term correspond tovalues of η(s) in Eq 3.

We sample from the posterior speed field by using the nor-mal approximation to the t-distribution. Such an approximationis justified since the degrees of freedom in the model is quitelarge. To sample the m desired speeds we first calculate theCholesky decomposition A = LLT [48]. Then we samplea m × 1 vector of independent standard normal values Z.Finally, the vector V + LZ constitutes a sample from thedesired distribution.

Finally, we sample predictions of future interface positionsat tk+1 by solving the eikonal equation with Dirichlet bound-ary condition T (s ∈ ∂Ωk) = tk, by fast marching the interfacewith the sampled speed fields for times tk < t ≤ tk+1.

B. Generative model for static image segmentation

With the ability to predict the future position of an interfacefrom its past history, we now turn our attention to extractinginformation about the interface position from images. At thecore of this exercise is a generative computer vision model.




We model an image probabilistically with normal distributionsof intensity values conditional on region:

I(s) ∼

N(µΩ, τ

2Ω

)s ∈ Ω (foreground);

N(µ∆, τ

2∆

)s ∈ ∆ (background).

(10)

Additionally, we incorporate information about Ω as a priordistribution p(Ω), and perform inference on the joint posteriorp(Ω, θ|I) ∝ p(I|Ω, θ)p(θ|Ω)p(Ω) = e−U(Ω,θ), where U(Ω, θ)is an energy, and θ = [µΩ, µ∆, τ

2Ω, τ

2∆] is a vector of the

Gaussian image intensity parameters.In this formulation, p(θ|Ω) represents the prior knowledge

of the regional image intensities, and p(Ω) represents theprior knowledge of the underlying foreground shape andlocation. For p(θ|Ω), we use the normal-gamma conjugateprior distribution

p(θ|Ω) = N(µΩ; µΩ, τ

2Ω

)N(µ∆; µ∆, τ

2∆

)×G

(τ2Ω; aΩ, bΩ

)G(τ2∆; a∆, b∆

)(11)

with gamma hyper-prior [45] over the image-intensity preci-sions. The parameters µΩ, µ∆ are the prior regional mean im-age intensities, and aΩ, bΩ, a∆, b∆ are hyperparameters whichare set to be weakly informative (all equal to 0.001) unlessotherwise stated.

To represent shape-knowledge, we use a kernel densityestimate of the distribution of possible shapes embedded asdiscrete level sets. Let us denote χΩ the characteristic functionfor a region Ω,

χΩ(s) =

1, s ∈ Ω;

0, s 6∈ Ω.(12)

Then, for two shapes Ω and Λ, embedded as discrete signeddistance functions φΩ and φΛ, we introduce an asymmetricshape divergence

d(Ω,Λ) =

∑s

indicator of pixel mismatch︷︸︸︷|χΩ(s)− χΛ(s)| |φΛ (s)|α

+∑s,u∈N

1 indicates s,u lie on opposite sides of the boundary of Ω1︷︸︸︷|(χΩ(s)(1− χΩ(u)) + χΩ(u)(1− χΩ(s))|

× wsu∣∣∣∣φΛ

(s+ u

2

)∣∣∣∣α , (13)

where the parameter α represents how severely we penalizeshape mismatch. The parameter wsu is one divided by thelength of the edge (Euclidean distance between s and u), andN denotes the set of neighboring grid points. We take thegrid points s to lie in the center of each grid cell, and use theeight neighbor system detailed in El Zehiry et al. [49], andrepresented pictorially in Fig 2. The term φΛ

(s+u

2

)refers to

the signed distance between the midpoint of the s−u segmentand the boundary of Λ. This expression penalizes mismatchesbetween the two shapes, and particularly penalizes protrusionsin Ω not represented in Λ. Given this distance measure, we

can define a Gaussian kernel of the form

K(Ω,Λ) =

√τ2

2πe−

τ2

2 d(Ω,Λ). (14)

Now, using some available reference shapes ΩjJj=1 andcarefully-chosen weighting coefficients wj , we can representa distribution over shapes, pS(Ω) as follows:

pS(Ω) ∝J∑j=1

wjK(Ω,Ωj).

This representation of the prior is the kernel density estimate(KDE) of the distribution of shapes. Like in Cremers [41] andCremers et al. [50], we empirically set τ2 to the followingvalue:

τ2 =

J∑j=1

wj mink 6=j

d(Ωk,Ωj)

−1

. (15)

To impose smoothness on the boundary of Ω, we additionallyassume a prior on the boundary length, pl(Ω) ∝ e−νH(dΩ),where H(dΩ) is the Hausdorff measure or length of the curve.We use the discrete approximation of this measure found in ElZehiry et al. [49]. Finally, we can write our complete prior overΩ as follows:

p(Ω) ∝ e−νH(dΩ)J∑j=1

wjK(Ω,Ωj).

We wish to infer the segmentation Ω by maximizing theposterior probability relative to Ω. To this end, we will maxi-mize the logarithm of the posterior, or equivalently, minimizethe following energy:

U(Ω, θ) = − log[

likelihood︷︸︸︷p(I|Ω, θ)

prior︷︸︸︷p(θ|Ω)p(Ω)]

=1

2

∑s∈Ω

[log

(2π

τ2Ω

)+ τ2

Ω (I(s)− µΩ)2

]+

1

2

∑s∈∆

[log

(2π

τ2∆

)+ τ2

∆ (I(s)− µ∆)2

]

+ νH(dΩ)︸︷︷︸length penalty

− log

J∑j=1

wjK(Ω,Ωj)︸︷︷︸KDE of shapes

− log p(θ|Ω) + const. (16)

We take an iterative two-step approach to minimizing thisenergy. Given Ω, we minimize the energy with respect toθ directly by setting the gradient of the energy with respectto θ to zero, and solving for θ (see O’Hagan et al. [45]).Then, given θ, we find the optimal Ω using the majorization-minimization algorithm described in the following section. Werepeat this two-step procedure until a stable energy is reached.

C. Majorization-minimization (MM) algorithm

The shape contribution log∑wjK(Ω,Ωj) can make mini-

mization of the energy difficult, since its formulation involvesa sum within a logarithm. To separate the contributions from




the reference shapes Ωj , we will derive a surrogate functionwith separated terms. A function f(x|xk) is said to majorizea function g(x) at xk if g(x) ≤ f(x|xk), ∀x, and if f(xk) =g(xk|xk) [51]. We perform inference by iteratively computingΩ(n+1) = arg minΩQ(Ω|Ω(n)), where Q(Ω|Ω(n)) majorizesEq 16. Noting that − log(·) is convex, we will use a definitionfor convexity f(

∑i αiti) ≤

∑i αif(ti), to show that for any

segmentation Ω(n), the following holds:

− log

shape kernel density︷︸︸︷J∑j=1

wjK(Ω,Ωj) ≤

−J∑j=1

wjK(Ω(n),Ωj)∑Jk=1 wkK(Ω(n),Ωk)

× log

[∑Jk=1 wkK(Ω(n),Ωk)

wjK(Ω(n),Ωj)wjK(Ω,Ωj)

]

= −J∑j=1


logK(Ω,Ωj)

+ constsubstituting Eq 14,

=τ2

2

J∑j=1


d(Ω,Ωj)︸︷︷︸separated log shape kernel density

+ const. (17)

Since Eq 17 majorizes the log-kernel density, we can minimizeour original energy by iteratively minimizing

Q(Ω|Ω(n)) =

1

2

∑s∈Ω

[log

(2π

τ2Ω

)+ τ2

Ω (I(s)− µΩ)2

]+

1

2

∑s∈∆

[log

(2π

τ2∆

)+ τ2

∆ (I(s)− µ∆)2

]+ νH(dΩ)− log p(θ|Ω)

+τ2

2

J∑j=1


d(Ω,Ωj). (18)

Since the distance function can be written as a sum over thevertices, so can Eq 18. As a result, it is possible to minimizeEq 18 within the graph cuts framework described in the nextsection.

D. Graph cuts for segmentation

Here, we describe minimization of the surrogate energyin Eq 18 using graph cuts, which quickly finds a globalminimum of a restricted set of energies. Graph cut methodshave their grounding in combinatorial optimization theory, andare concerned with finding the minimum cut in an undirectedgraph. A cut is a partition of a connected graph into twodisconnected sets. The cost of a cut is the sum of the edge

weights along a cut, and a max-flow min-cut algorithm findsthe cut with the lowest cost. To use graph cuts for imagesegmentation, we must express our energy function in termsof edge-weights on a graph. We will describe an image asa connected graph, where each pixel represents a node, andedges exist between neighboring nodes (Fig 2). Note thatedges in this context refer to connections between nodes ina graph, and not to edges in an image. We want to inferan unknown two-coloring on the nodes of the graph thatrepresents inclusion of a node s into either the foregroundset Ω, or the background set ∆. Following El Zehiry et al.[49], we begin by expressing the energy given in Eq 18 as afunction of the vertices V and edges E of a graph G = (V, E):

U(G) =∑s∈V

UV (s) +∑

(s,u)∈E

UE(s, u).

From equation Eq 18 we find that

UV (s) =1

2

[log

(2π

τ2Ω

)+ τ2

Ω (I(s)− µΩ)2

]χΩ(s)

+1

2

[log

(2π

τ2∆

)+ τ2

∆ (I(s)− µ∆)2

](1− χΩ(s))

+τ2

2

J∑j=1

wjK(Ω(n),Ωj)∣∣φΩj (s)

∣∣α ∣∣χΩ(s)− χΩj (s)∣∣∑J

k=1 wkK(Ω(n),Ωk)

(19)

and

UE(s, u) =

πτ2

16wl(s, u)

J∑j=1

wjK(Ω(n),Ωj)∣∣φΩj

(s+u

2

)∣∣α∑Jk=1 wkK(Ω(n),Ωk)

×∣∣χΩj (s)

(1− χΩj (u)

)+ χΩj (u)

(1− χΩj (s)

)∣∣ + νwl(s, u) |χΩ(s)(1− χΩ(u)) + χΩ(u) (1− χΩ(s))| .

(20)

In Eq 20, wl(s, u) is an edge weighting that approximates theHausdorff measure of the boundary [49]. In the eight-neighborsystem we use, it takes values of wsuπ/8. Weighting of edgesin this manner helps enforce homogeneity of labeling betweenneighboring spatial points. It is of note that the energy dependsupon the distance functions for the kernel density referenceshapes, and not the evolving segmentation. If it were to dependon the signed distance function of the segmentation, it wouldnot be possible to write the energy in a form minimizablein a graph cuts framework, thus necessitating the use of anasymmetric shape distance like the one in Eq 13.

To minimize the energy, we augment our pixel lattice graphwith two special nodes. In the language of graph cuts, thesenodes are known as the source and sink. For our purposes, thelabeling of source and sink are arbitrary since we are dealingwith an undirected graph. We will call one of these verticesvΩ, and the other one v∆. The existence of an edge betweena pixel s and vΩ will represent the segmentation of s into Ω.We then connect each pixel node directly to both vΩ and v∆,




and weight these new edges as follows:

w(s, vΩ) =1

2

[log

(2π

τ2∆

)+ τ2

∆ (I(s)− µ∆)2

]+τ2

2

J∑j=1

wjK(Ω(n),Ωj)χΩj (s)∑Jk=1 wkK(Ω(n),Ωk)

∣∣φΩj (s)∣∣α , (21)

w(s, v∆) =1

2

[log

(2π

τ2Ω

)+ τ2

Ω (I(s)− µΩ)2

]+τ2

2

J∑j=1

wjK(Ω(n),Ωj)∣∣φΩj (s)

∣∣α (1− χΩj (s))∑Jk=1 wkK(Ω(n),Ωk)

. (22)

The cutting of the edge from an s to vΩ implies that s ∈ ∆,so it adds to the cost of the cut by the contribution of s intothe total energy as if s ∈ ∆. In other words, these weightscan be interpreted as a pixel’s strength of belonging to eachregion. If s is in a particular region, then neighbors of s aremore likely to be in the same region. This fact is representedby edge weights between neighboring pixels. Those weightsare

πτ2

16wl(s, u)

J∑j=1

wjK(Ω(n),Ωj)∣∣φΩj

(s+u

2

)∣∣∑Jk=1 wkK(Ω(n),Ωk)

α

+ νwl(s, u) (23)

where u is a grid-neighbor of s. Our surrogate energy isnow minimized by finding the minimum cut of the graph.For details on how to perform this optimization, we referthe reader to Boykov and Kolmogorov [52]. To minimizethe original energy function, one iteratively computes thegraph-cut minimum within the MM algorithm described insection II-C. It is of note that θ does not significantly change ifthe segmentation labels do not significantly change. Therefore,it is computationally beneficial to recompute θ after each ofthe first few MM iterations, and only recompute it if the labelsundergo further large changes.

E. Bayesian filtering

We now have all the pieces needed to perform sequentialBayesian estimation of the interface positions. In section II-A,we described our model for predicting the speed and henceposition of the interface. In section II-B, we showed howone can use a kernel density estimate of the position of theboundary in an image to craft an energy functional that whenminimized yields the position of the boundary. In sections II-Cand II-D, we provided an iterative method of minimizingthe energy functional. We now describe how to combinethese components together into a Bayesian filter. Let Ωk|k−1

be the random variable Ωk conditional on all images upto and including time tk−1. Then for each frame Ik in animage sequence, after initialization, our Bayesian filter iteratesbetween two steps, predict and update.

Initialization: To initialize our segmentation method, oneneeds an initial segmentation at the first image frame. Possess-ing prior shape knowledge, one may initialize the segmentationwith a set of prior shape templates [41], and then minimizeEq 16 directly. In the absence of shape information, one may

create a non-informational shape prior by defining a singlearbitrary reference shape, and setting τ2 → 0. This procedurereduces the energy function to pure intensity-based graph cuts.

Predict: We draw samples of segmentations Ωk|k from itsposterior, and infer the GMRF speed field associated with eachsegmentation. Then for each segmentation sample, propagatethe associated interface through samples from its speed fieldto generate an ensemble of predictive positions for Ωk+1|k.

The posterior of Ωk|k is defined by an energy

U(Ωk|k, θk) =

1

2

∑s∈Ωk|k

[τ2Ωk|k

(Ik(s)− µΩk|k

)2 − log(τ2Ωk|k

)]+

1

2

∑s∈∆k|k

[τ2∆k|k

(Ik(s)− µ∆k|k

)2 − log(τ2∆k|k

)]+ νH

(∂Ωk|k

)− log p

(θk|Ωk|k

)− log

J∑j=1

L∑l=1

WjK(

Ωk|k,Ω(j,l)k|k−1

), (24)

for some known weighting coefficients Wj and referenceshapes Ω

(j,l)k|k−1.

We sample segmentations from the posterior by usingimportance sampling [53]. Importance sampling obtains sam-ples from a difficult target distribution f(x) by samplingfrom an easier importance distribution g(x). With samplesXi ∼ g(x), one approximates expectations of a function h(x)under the distribution f(x) by first calculating the weightsWi = f(Xi)/g(Xi), and then approximating the expectationby E[h(x)] =

∑Wih(Xi)/

∑Wi. To approximate the target

distribution itself, one can combine kernel density estimationand importance sampling to deduce an approximating distribu-tion of the form f(x) =

∑WiK(x,Xi)/

∑Wi, where K(·, ·)

is a kernel function.

We sample a set of current interface positions

Ω(j)k|k

Jj=1

as the conditional maximum-a-posterior segmentations undermodified length penalties ν(j), where each ν(j) follows anexponential distribution with rate parameter 1/ν. With eachν(j), we modify the length penalty in Eq 24 by settingν → ν(j) and minimize the resulting energy to obtain a sampleinterface position Ω

(j)k|k with associated image intensity param-

eters θ(j)k . Conditional on each Ω

(j)k|k, we infer the associated

stochastic speed field V(j)k|k|Ω

(j)k|k, which follows a multivariate

t-distribution with scale matrix A(j)k|k. To infer this field we

first linearly interpolate arrival times of the interface fromthe set of wave positions. Then we calculate the past speedsaccording to the eikonal equation using second-order upwindfinite differences [42], giving us a vector of known speeds V0.Finally, we input the resulting speeds into the GMRF modelof section II-A, where Eqs 6, 7, 8, and 9 provide us with theparameters of the resulting multivariate t-distribution. Then,from each GMRF, we draw a fixed number of samples (weused 16). Thus, for each sampled interface position Ω

(j)k|k, we

have a collection of samples of the interface speed V(j,l)k|k

Ll=1.




With our states

Ω(j)k|k,

V

(j,l)k|k

Ll=1

Jj=1

at tk, and the

eikonal equation, we may now predict the location of theboundary at tk+1 by propagation of each pair of interface andspeed field through Eq 1 to calculate Ω

(j,l)k+1|k. This procedure is

accomplished by using the fast marching method [6], startingwith an initial position Ω

(j)k|k at tk, and solving with speeds

V(j,l)k|k until t = tk+1. The result is a set of predictive samplesΩ

(j,l)k+1|k

J,Lj=1,l=1

of the interface location at time tk+1, given

the information up to time tk.The weights in Eq 24 are then given by the importance

sampling weights

Wj = exp

[ν(j)

ν− U

(Ω

(j)k|k, θ

(j)k

)], (25)

normalized to sum to 1. Here, U is the energy function givenin Eq 24 with the original shape penalty ν. In Eq 16, we tookthe prior on Ω to be the kernel density estimate of referenceshapes. We can compute this representation of the prior by set-ting P (Ωk+1|k) ∝

∑j

∑lWjK

(Ωk+1|k,Ω

(j,l)k+1|k

). With our

prior constructed, we now have all the components necessaryto specify the posterior distribution for time tk+1.

Update: When the new observation at time tk+1 comes in,one updates his prediction of the state at tk+1 by minimizingthe posterior energy U(Ωk+1|k+1, θk+1). This update is assimple as relaxing the energy (Eq 24 with k incremented byone) by the MM procedure described in section II-C, iteratedwith estimation of the image intensity statistics. We initializethe MM algorithm to start at the state obtained by propagatingthe maximum-a-posterior Ωk|k according to its maximum-a-posterior speed field (Eq 8), and initially inferring the imageintensity statistics conditional on this state.

III. APPLICATION TO BIOMEDICAL IMAGES

We implemented the segmentation and speed interpolationmethod given in previous sections as a Java-based plug-in forImageJ, the image manipulation and analysis package fromthe NIH [54]. We chose UJMP [55], a LGPL licensed fastmatrix library, to perform all matrix operations. For graph-cuts optimization, we modified Fiji’s [56] implementation ofthe max-flow min-cut algorithm by Boykov and Kolmogorov[52] for our needs. In these examples, the shape mismatchpenalty was set to α = 2, and the length penalty was set toν = 20, unless otherwise stated. Convergence to a minimumenergy typically occurred within five to nine MM iterations.It is worthy to note that one need not recompute the imageintensity statistics θ at each step if only a small number oflabels have changed. We only recalculated the image intensitystatistics in the first three iterations, where convergence wasmost rapid. All segmentations were performed on sequencesof 320× 240 images.

A. Synthetic image sequence

To test the ability of our method to recover a knowninterface from images, we applied our method to a sequenceof synthetic images (Fig 4). We defined a “ground truth”

speed field within a 320 × 240 pixel spatial field using theUCLA logo (Fig 5, left), with speeds of 10 pixels/frame and6 pixels/frame inside and outside of the letters, respectively.Then, we advected a boundary traveling from the lower rightof the field until it crossed the entire field (total of 90frames). From this set of interface positions, we generated aseries of images, with mean intensities of 3 inside the regiontraversed by the boundary, and 0 outside, with a variance of9 (Fig 4). We used weakly informative image intensity anduninformative initial shape priors to segment the resultingimages. To estimate the speed, we used a weakly informationalprior speed centered at 8 pixels/frame (weakly informativeimplies that all Gamma hyperparameters are set to 0.001).

The boundary that we found is in good accord with theground truth. Our regularization is strong enough to providerobustness to noise, yet, not so strong that relatively large de-formations are not detected. The ground truth image undergoesrobust changes in topology that are faithfully reproduced in oursegmentation results. Fig 5 shows our reconstruction of theground truth interface speed, performed using second-orderupwind finite differences. The original UCLA logo is clearlyvisible in the reconstructed speed (right). There is noisiness inthe reconstruction due to the indeterminacy of the inversionof the eikonal equation (Eq 1), however we obtain an accuratereconstruction. The mean reconstructed speed is 9.8 ± 1.5pixels per frame within the UCLA lettering, and 6.0 ± 0.7pixels per frame outside of the UCLA lettering; these valuesare within 5% of the original speed field.

In Fig 6, we show that even large deformations far fromthe predicted mean shape are detected, typically in very fewiterations. Here, protrusions in the front are developing suchthat the shape of the front is significantly different fromthe mean predicted shape. After a single MM iteration, thealgorithm snaps to the protrusion. This behavior is a resultof the smoothness and large support of the shape divergencemeasure (Eq 13). Even protrusions that are improbable underthe motion model can be stabilized by the likelihood.

B. Cortical Spreading Depression (CSD)

Optical intrinsic signal (OIS) imaging is a simple method ofvisualizing physiological processes without the use of dyes ortracers. Instead, OIS captures changes in the sample’s intrinsicoptical reflectance. Because of its simplicity and versatility,OIS is used for in vivo imaging applications in neuroscience.During CSD, there are large hemodynamic or blood-related,changes in cortical tissue [57, 58]. Conveniently, in corticalbrain tissue under visible light, changes in blood volume andblood oxygen saturation constitute the majority of the OISsignal. The CSD wave appears as a low contrast brighteningof the tissue, visible in inter-frame difference images, thoughdue to the diffuse signal, is time-consuming for even a trainedpractitioner to trace.

Using the notation of section II-B, we treated the spatialextents of the CSD wave as foreground regions Ωk in an imagesequence OkKk=0. We modeled each inter-frame differenceIk = Ok+1 − Ok image as a conditional Gaussian mixturelike in section II-B. As prior information, the wavefront is




Fig. 4. Segmentation of a synthetic image sequence (top) Mask of ground truth wave sequence, with a growing interior region shown in black. The spatialfield is of size 320×240 pixels. The speed of the front varies between 6 and 10 pixels/frame (shown in fig 5), producing topological changes in the interface.(bottom) Segmentation of images where noise has been added to the ground truth. The image intensity was 0 ± 3 in the exterior and 3 ± 3 in the interior.δt = 7 frames.

Fig. 5. Recovery of interface speed (left) Ground truth speed field. (right)Reconstructed speed field. Reconstruction of synthetic wave speed using thesegmentations shown in figure 4, and second-order upwind finite differences.The ground-truth speed of the interface is 10 pixels/frame as it passes throughthe UCLA letters, and 6 pixels/frame outside of the letters. The reconstructedspeed field has an estimated speed of 9.8 ± 1.5 pixels/frame inside theletters, and 6.0 ± 0.7 pixels/frame outside. Velocity scale shown at right inpixels/frame.

known to travel at approximately 1−5 millimeters per minute.We incorporated this information by setting a prior meanwave speed of 3 millimeters per minute, with variance of 1millimeters2 per minute2 (we set aη = bη = 1). As in thesynthetic image sequence application, we used uninformativeinitial shape priors.

We tested our method on a real set of CSD images ac-quired from two separate experiments on separate C57Bl/6Jmice. The mice were approved for experiments in accordancewith University of California, Los Angeles Animal ResearchCommittee Guidelines. The mice had their skulls exposedunder anesthesia, and a rectangular section of the parietal bone(1 mm from the sagittal suture, temporal ridge, lambdoidalsuture and coronal suture) was thinned to transparency. Burr-holes were drilled proximal to the imaging field to allow forplacement of stimulating electrodes. After allowing the animalto rest, the experimenter induced CSD by passing electriccurrent through the stimulating electrodes. VGA-resolution(640 × 480) images were collected at a frequency of 1Hz,in 8-bit greyscale, under white light. The imaging field wasapproximately 3.2mm× 2.4mm, with each pixel representingapproximately a 5µm× 5µm square. Before analysis, to savecomputation time, we rescaled each image to a quarter of itsoriginal size by bilinear interpolation (to 320× 240).

Fig. 6. Updating predictions with new data (left) Past (yellow), current(green), and predicted future (red) interface positions drawn over meanpredicted speed field. Predictions of future interface positions, which act asshape priors, are made using samples from our stochastic speed model. Tothe left of the green boundary are speeds interpolated from the collectionof past interface positions. To the right are interface positions found bypropagating the green contour with speeds sampled from the GMRF model.Speed scale shown at left is pixels/frame. (middle) When a new noisy imageis acquired, the MM algorithm is initialized at the position obtained bypropagating the previous interface position according to its estimated meanspeed field. The true boundary deviates from the mean predicted boundarybecause of developing protrusions. The mean predicted boundary is calculatedby propagating the green boundary against the mean speeds given in Eq 8.(right) After a single MM iteration, the protrusions are found. Due to thecontinuous nature of the kernel density shape prior, our method is able toaccount for large deformations.

CSD segmentation results are shown in Fig 7. For compar-ison, we also provide the results of segmentation done withgraph cuts in the absence of shape prior [49]. Our trackingmethod is able to regularize against the noise and randomheterogeneity that is typical of these in-vivo experiments. Inthe absence of a prior, the interface location is not as welldefined, and graph cuts fails to select the entire CSD region.

In Fig 8, we examine the results of our model undernon-ideal conditions. In this experiment, biological movementhas caused the presence of a vascular distraction in thedifference images. The images in the top row show frame-by-frame segmentation in the absence of shape priors, where thevascular system is causing formidable interference. Using ourmethod, we achieve increasingly better segmentations whenincreasing the shape mismatch penalty α, though α = 2 still




Fig. 7. Segmentation of real in-vivo CSD data (Left→Right) CSD shown propagating, δt = 7s. Top: Original unsegmented inter-frame differencesshowing CSD-related changes in blood signal. Second row: Results from our segmentation method, where we track the moving front probabilistically. Thirdrow: Segmentation of the spreading region done without shape priors.

Fig. 8. Adjusting regularization by adjusting the shape penalty parame-ter (Left→Right) and (Top→ Bottom) Biological movement during imagingcauses artifacts in the difference image. Failure to adjust for the movementresults in less than ideal data. Increasing α, the shape mismatch penalty, cancompensate for poorly acquired image data. Our method is able to track themoving front even as it is partially occluded. Top: segmentation without shapeprior. Second row α = 2. Third row: α = 3. Fourth row: α = 4. δt = 2s.

works reasonably well. Increasing α increases the weight ofthe speed-based regularization relative to the likelihood. Thisparameter offers flexibility for tuning the method for data setswith increasing random heterogeneity.

C. Wound healing assays

Another application we explore is the tracking of a collaps-ing boundary. One specific example of this type of systemarises in in vitro assays of wound healing. In the typicalwound healing assay, a layer of cells is grown to confluencyon a substrate. A portion of the cell layer is then removed,either by scratching, lifting off a localized region of cells,or removing a constraint confining the monolayer [59]. The

dynamics of how the cells refill the bare substrate is thenstudied typically with bright or dark-field light microscopyunder a variety of physical and chemical conditions. While thebiological process is complex, involving chemical signalingpathways, mediated by mechanical interactions with the elasticsubstrate and neighboring cells [60], the main observable is themoving wound edge.

Both cell migration and cell proliferation can occur, [61],and as more quantitative studies of these types of assaysemerge [62], tracking the spatio-temporal dynamics of cellsand their proliferation will become critical. In response to awound or free space in which to migrate, the cells near theedge increase their motility to attempt to cover the wound.The motion of the wound boundary appears to be largelymonotonic. However, without additional experimental imagingmodalities such as fluorescent labeling of membranes, accuratetracking of the wound edge can be difficult due to low contrast,boundaries with other cells, and extraneous material (such asfloating dead cells) in the image field.

Our statistical approach for tracking the moving boundaryof a cell monolayer was tested on an in vitro wound heal-ing assay. Fig 9 shows segmentations on a series of sobelfiltered [54] images of a shrinking circular wound that wasinduced in a monolayer of epithelial cells. Pre-wounding,the epithelial cells were grown to confluency on extracellularmatrix substrate. Bright-field images were then taken of theepithelial cells migrating to fill the circular wound region.In these images, the wound region and the healed region aredistinguished by the lack and presence of cells. Cells in thefield appear dark at their boundaries, and bright in their bodies.In this application, we treat the wounded region as Ω. Asthe wound heals, it undergoes robust changes in shape. Oursegmentation method accurately tracked the boundary of thewound. For these segmentations, where we knew the locationof the initial wound circle, we used an informative shapeprior with the known reference shape, and set τ2 = 1. Weused a weakly informative prior mean speed centered at −4




Fig. 9. Segmentation of wound healing assays featuring robust shape changes (Left→Right) and (Top→ Bottom) Wound-healing time stills. Segmentationperformed on the sobel filter (shown) of the original image sequence, which is well-modeled by the Gaussian mixture of section II-B. In this application,the boundary is moving inward, while the shape of the inner region is undergoing large changes. The segmentation method works well even for non-convexshapes. δt = 30 min. We are grateful to Prof. C.-L. Guo, Caltech Bioengineering for these images. Resolution: 320× 240.

pixels/frame. In this application, the speed of the front V isstrictly negative.

D. Validation

We evaluated the accuracy of our segmentation methodby comparing results from our method against the resultsof human-assisted segmentations for the synthetic image se-quence (Fig 4) and the CSD image sequence (Fig 7). In bothcases, we compute the mismatch between two segmentations

Error =number of mismatching pixels

mean boundary length of segmentations. (26)

The resulting quantity has an interpretation as the averagethickness in pixels of the mismatched region.

Frame

Err

or

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 2 3 4 5 6

Segmentation

Human 1

Human 2

Human 3

Our Method

Fig. 10. Comparison of results against laborious manual human seg-mentation of synthetic images The accuracy of our method’s segmentationof the synthetic image sequence (for the six frames shown in Fig 4) comparedto the accuracy of three humans. The error plotted in the y-axis is the averagenumber of misclassified pixels per boundary-length, where boundary-length isthe average of the lengths of the ground truth and segmented boundary. Ourmethod differed from the ground truth by 0.30 ± 0.10 pixels. The humansperformed significantly worse with error of 0.61± 0.14 pixels.

For the synthetic image sequence, where the ground truthis available, we compared the error of segmentations madeusing our method to error from human-assisted segmentations.Fig 10 depicts the mismatch from ground truth for segmen-tations on the synthetic image sequence of Fig 4. For thisdata set, it is clear that our segmentation method outperformshuman segmentation, by approximately a factor of two.

For the cortical spreading depression images of Fig 7,where the ground truth is not available, we compared humansegmentation to our automated segmentations (Fig 11). Our re-sults are in good agreement with manually-segmented results,good to within 1.7 ± 0.5 pixels. By comparison, the humanssegmentations disagreed with each other by 1.4 ± 0.4 pixels.These data demonstrate the ability of our method to performas-well-as or better than manual segmentation.

Frame

Err

or

1.0

1.5

2.0

2.5

3.0

1 2 3 4 5 6

Segmentation

Human 1

Human 2

Human 3

Fig. 11. Comparison of results against laborious manual humansegmentation of in-vivo CSD images Deviation of human segmentationsfrom the results of our automated approach for the image sequence shownin Fig 7. The results from our segmentation method agreed with the resultsfrom the manual segmentations to within 1.7± 0.6 pixels.




IV. DISCUSSION

We have demonstrated a framework for simultaneous seg-mentation and inference of the dynamics of monotonicallytraveling boundaries in image sequences. Our method issimple, generalizable, and easily extended to a wide varietyof applications. The novelty of our method is in bringinglevel-set based interface modeling into a statistical estimationframework, where inference can occur using graph cuts. Inthe process, we have developed a theory for including shapepriors into the graph cuts method using an MM algorithm.Wehave demonstrated the efficacy of our method in solvingthe boundary tracking problem for two unrelated biomedicalapplications, and is able to recover a ground truth speed patternfor a synthetically generated image sequence.

Since we developed our method from the ground up withstatistical theory in mind, it can be easily extended or modifiedto suit a wide range of applications. Being modular in design,it is easy to alter particular components of our method whileleaving others unchanged. For instance, one may choose tomodify the shape distance we introduce in Eq 13, to suit aparticular application. There is much prior work on shape dis-tances in the level set literature, and the graph cut segmentationcommunity would greatly benefit from more research on howto incorporate shape knowledge.

Since our method is model-based, it is fairly easy to predictwhen performance suffers. In particular, when the image pixelintensities are poorly described by the likelihood model ofsection II-B, or when the speed of the interface is poorlymodeled as the smooth Gaussian process of section II-A. Inmany cases, success of the method depends on the choice ofshape penalty exponent α. In some sense, α controls the degreeof regularization done by the recursive shape priors. If α is settoo large, the segmenter will tend to favor the predicted inter-face positions over the data, ignoring real deformations in theinterface. Conversely, if α is set too small, the regularizationmay not be sufficient, and the segmenter is more likely to pickup noise in the image. For example, in Fig 8, increasing theshape penalty parameter to α = 4 improved the tracking ofthe interface in the presence of distractors. In general, α = 2worked well for our examples.

Perhaps the aspect of our method that would benefit themost from future research would be the spatial interpolationmethod we use to sample fluctuations in interface speed. TheGaussian process model, while theoretically clean, is compu-tationally complex. Yet, the Bayesian nature of this methodallows for the application of natural Bayesian model evaluationand inference methods. In particular, future development willfocus on Bayesian model selection of competing speed models.Finally, we would like to note that extension of our method tonon-monotonic movement is theoretically straightforward. Oneneed only implement the time-dependent Gaussian processinterpolation of Sahu et al. [46] to predict speeds, and use thelevel-set equation rather than the eikonal equation to describethe motion.

V. ACKNOWLEDGEMENTS

We thank Lydia L. Shook for performing the CSD exper-iments, Berta Sandberg for her input in the area of segmen-

tation, and Tanye Y. Tang for helping to adapt the Vaswaniet al. [23] Matlab code for use on the image sequences inthis manuscript. We are grateful to Prof. C.-L. Guo, CaltechBioengineering for the wound healing assay images. JC ac-knowledges support by grant mumber T32GM008185 fromthe National Institute of General Medical Sciences. JC andTC also acknowledge support from the the National ScienceFoundation through grants DMS-1032131 and DMS-1021818,and from the Army Research Office through grant 58386MA.KCB acknowledges support from the National Institutes ofHealth (NINDS NS059072 and NS070084) and the Depart-ment of Defense (CDMRP PR100060).

REFERENCES

[1] V. Fast and A. Kleber, “Role of wavefront curvaturein propagation of cardiac impulse,” Cardiovascular re-search, vol. 33, no. 2, p. 258, 1997.

[2] C. Hogea, B. Murray, and J. Sethian, “Simulating com-plex tumor dynamics from avascular to vascular growthusing a general level-set method,” journal of mathemat-ical biology, vol. 53, no. 1, pp. 86–134, 2006.

[3] D. Chien, K. Kwong, D. Gress, F. Buonanno, R. Buxton,and B. Rosen, “MR diffusion imaging of cerebral infarc-tion in humans,” American journal of neuroradiology,vol. 13, no. 4, p. 1097, 1992.

[4] A. Cornell-Bell, S. Finkbeiner, M. Cooper, and S. Smith,“Glutamate induces calcium waves in cultured astrocytes:long-range glial signaling,” Science, vol. 247, no. 4941,p. 470, 1990.

[5] A. Leao, “Spreading depression of activity in the cerebralcortex,” journal of Neurophysiology, vol. 7, no. 6, p. 359,1944.

[6] J. Sethian, “A fast marching level set method for mono-tonically advancing fronts,” Proceedings of the NationalAcademy of Sciences of the United States of America,vol. 93, no. 4, p. 1591, 1996.

[7] ——, Level set methods and fast marching methods:evolving interfaces in computational geometry, fluid me-chanics, computer vision, and materials science. Cam-bridge Univ Pr, 1999, no. 3.

[8] S. Osher and J. Sethian, “Fronts propagating withcurvature-dependent speed: algorithms based onHamilton-Jacobi formulations,” journal of computationalphysics, vol. 79, no. 1, pp. 12–49, 1988.

[9] R. Malladi, J. Sethian, and B. Vemuri, “Shape modelingwith front propagation: A level set approach,” PatternAnalysis and Machine Intelligence, IEEE Transactionson, vol. 17, no. 2, pp. 158–175, 1995.

[10] P. Macklin and J. Lowengrub, “An improved geometry-aware curvature discretization for level set methods:application to tumor growth,” journal of ComputationalPhysics, vol. 215, no. 2, pp. 392–401, 2006.

[11] X. Zheng, S. Wise, and V. Cristini, “Nonlinear simulationof tumor necrosis, neo-vascularization and tissue invasionvia an adaptive finite-element/level-set method,” Bulletinof mathematical biology, vol. 67, no. 2, pp. 211–259,2005.




[12] M. Sermesant, E. Konukoglu, H. Delingette, Y. Coudiere,P. Chinchapatnam, K. Rhode, R. Razavi, and N. Ayache,“An anisotropic multi-front fast marching method forreal-time simulation of cardiac electrophysiology,” Func-tional Imaging and Modeling of the Heart, pp. 160–169,2007.

[13] C. Wolgemuth and M. Zajac, “The moving boundarynode method: A level set-based, finite volume algorithmwith applications to cell motility,” journal of computa-tional physics, vol. 229, no. 19, pp. 7287–7308, 2010.

[14] N. Paragios and R. Deriche, “Geodesic active contoursand level sets for the detection and tracking of mov-ing objects,” Pattern Analysis and Machine Intelligence,IEEE Transactions on, vol. 22, no. 3, pp. 266–280, 2000.

[15] J. Tang and S. Acton, “Vessel boundary tracking forintravital microscopy via multiscale gradient vector flowsnakes,” Biomedical Engineering, IEEE Transactions on,vol. 51, no. 2, pp. 316–324, 2004.

[16] Y. Zhong, A. Jain, and M. Dubuisson-Jolly, “Objecttracking using deformable templates,” Pattern Analysisand Machine Intelligence, IEEE Transactions on, vol. 22,no. 5, pp. 544–549, 2000.

[17] A. Mansouri, “Region tracking via level set pdes with-out motion computation,” IEEE Transactions on PatternAnalysis and Machine Intelligence, pp. 947–961, 2002.

[18] A. Baumberg and D. Hogg, “An efficient method forcontour tracking using active shape models,” in Motion ofNon-Rigid and Articulated Objects, 1994., Proceedingsof the 1994 IEEE Workshop on. IEEE, 1994, pp. 194–199.

[19] Y. Chen, T. Huang, and Y. Rui, “Parametric contourtracking using unscented kalman filter,” in Image Pro-cessing. 2002. Proceedings. 2002 International Confer-ence on, vol. 3. Ieee, 2002, pp. 613–616.

[20] N. Peterfreund, “Robust tracking of position and velocitywith kalman snakes,” Pattern Analysis and MachineIntelligence, IEEE Transactions on, vol. 21, no. 6, pp.564–569, 1999.

[21] M. Isard and A. Blake, “Contour tracking by stochas-tic propagation of conditional density,” Computer Vi-sionECCV’96, pp. 343–356, 1996.

[22] P. Li, T. Zhang, and A. Pece, “Visual contour trackingbased on particle filters,” Image and Vision Computing,vol. 21, no. 1, pp. 111–123, 2003.

[23] N. Vaswani, Y. Rathi, A. Yezzi, and A. Tannenbaum,“Deform pf-mt: particle filter with mode tracker fortracking nonaffine contour deformations,” Image Pro-cessing, IEEE Transactions on, vol. 19, no. 4, pp. 841–857, 2010.

[24] A. Doucet, S. Godsill, and C. Andrieu, “On sequentialMonte Carlo sampling methods for Bayesian filtering,”Statistics and computing, vol. 10, no. 3, pp. 197–208,2000.

[25] J. Liu, Monte Carlo strategies in scientific computing.Springer Verlag, 2008.

[26] D. Mumford and J. Shah, “Boundary detection by mini-mizing functionals,” Image understanding, 1988.

[27] S. Zhu and A. Yuille, “Region competition: Unifying

snakes, region growing, and Bayes/MDL for multibandimage segmentation,” Pattern Analysis and Machine In-telligence, IEEE Transactions on, vol. 18, no. 9, pp. 884–900, 2002.

[28] T. Chan and L. Vese, “Active contours without edges,”IEEE Transactions on image processing, vol. 10, no. 2,pp. 266–277, 2001.

[29] L. Ford and D. Fulkerson, “Flows in networks,” 1962.[30] D. Freedman and P. Drineas, “Energy minimization via

graph cuts: Settling what is possible,” 2005.[31] V. Kolmogorov and R. Zabih, “What energy functions

can be minimized via graph cuts?” IEEE Transactionson Pattern Analysis and Machine Intelligence, pp. 147–159, 2004.

[32] O. Veksler, “Star shape prior for graph-cut image seg-mentation,” Computer Vision–ECCV 2008, pp. 454–467,2008.

[33] G. Slabaugh and G. Unal, “Graph cuts segmentationusing an elliptical shape prior,” in Image Processing,2005. ICIP 2005. IEEE International Conference on,vol. 2. IEEE, 2005.

[34] P. Das, O. Veksler, V. Zavadsky, and Y. Boykov, “Semiau-tomatic segmentation with compact shape prior,” Imageand Vision Computing, vol. 27, no. 1-2, pp. 206–219,2009.

[35] J. Malcolm, Y. Rathi, and A. Tannenbaum, “Graph cutsegmentation with nonlinear shape priors,” in Image Pro-cessing, 2007. ICIP 2007. IEEE International Conferenceon, vol. 4. IEEE, 2007.

[36] S. Dambreville, Y. Rathi, and A. Tannenbaum, “A frame-work for image segmentation using shape models andkernel space shape priors,” IEEE transactions on patternanalysis and machine intelligence, pp. 1385–1399, 2008.

[37] D. Freedman and T. Zhang, “Interactive graph cut basedsegmentation with shape priors,” in Computer Vision andPattern Recognition, 2005. CVPR 2005. IEEE ComputerSociety Conference on, vol. 1. IEEE, 2005, pp. 755–762.

[38] N. Vu and B. Manjunath, “Shape prior segmentation ofmultiple objects with graph cuts,” in Computer Visionand Pattern Recognition, 2008. CVPR 2008. IEEE Con-ference on. IEEE, 2008, pp. 1–8.

[39] H. Chang, Q. Yang, and B. Parvin, “A bayesian approachfor image segmentation with shape priors,” in ComputerVision and Pattern Recognition, 2008. CVPR 2008. IEEEConference on. IEEE, 2008, pp. 1–8.

[40] J. Zhu-Jacquot and R. Zabih, “Graph cuts segmentationwith statistical shape priors for medical images,” inSignal-Image Technologies and Internet-Based System,2007. SITIS’07. Third International IEEE Conference on.IEEE, 2007, pp. 631–635.

[41] D. Cremers, “Dynamical statistical shape priors for levelset-based tracking,” IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, pp. 1262–1273, 2006.

[42] D. Aldridge and D. Oldenburg, “Two-dimensional to-mographic inversion with finite-difference traveltimes,”Journal of Seismic Exploration, vol. 2, no. 25, pp. 7–274, 1993.

[43] H. Rue and H. Tjelmeland, “Fitting Gaussian Markov




random fields to Gaussian fields,” Scandinavian journalof Statistics, vol. 29, no. 1, pp. 31–49, 2002.

[44] L. Hartman and O. Hossjer, “Fast kriging of large datasets with Gaussian Markov random fields,” Computa-tional Statistics & Data Analysis, vol. 52, no. 5, pp.2331–2349, 2008.

[45] A. O’Hagan, J. Forster, and M. Kendall, Bayesian infer-ence. Arnold, 2004.

[46] S. Sahu, S. Yip, and D. Holland, “A fast Bayesian methodfor updating and forecasting hourly ozone levels,” Envi-ronmental and Ecological Statistics, pp. 1–23, 2009.

[47] N. Cressie, “Fitting variogram models by weighted leastsquares,” Mathematical geology, vol. 17, no. 5, pp. 563–586, 1985.

[48] A. Genz, “Numerical computation of multivariate normalprobabilities,” journal of computational and graphicalstatistics, vol. 1, no. 2, pp. 141–149, 1992.

[49] N. El Zehiry, S. Xu, P. Sahoo, and A. Elmaghraby,“Graph cut optimization for the Mumford-Shah model,”in The Seventh IASTED International Conference onVisualization, Imaging and Image Processing. ACTAPress, 2007, pp. 182–187.

[50] D. Cremers, S. Osher, and S. Soatto, “Kernel densityestimation and intrinsic alignment for shape priors inlevel set segmentation,” International journal of Com-puter Vision, vol. 69, no. 3, pp. 335–351, 2006.

[51] D. Hunter and K. Lange, “A tutorial on MM algorithms,”The American Statistician, vol. 58, no. 1, pp. 30–37,2004.

[52] Y. Boykov and V. Kolmogorov, “An experimental com-parison of min-cut/max-flow algorithms for energy mini-mization in vision,” Pattern Analysis and Machine Intel-ligence, IEEE Transactions on, vol. 26, no. 9, pp. 1124–1137, 2004.

[53] B. Ripley, Stochastic simulation. Wiley Online Library,1987, vol. 21.

[54] M. Abramoff, P. Magelhaes, and S. Ram, “Imageprocessing with ImageJ,” Biophotonics international,vol. 11, no. 7, pp. 36–42, 2004.

[55] H. Arndt, M. Bundschus, and A. Naegele, “Towards anext-generation matrix library for Java,” in 2009 33rdAnnual IEEE International Computer Software and Ap-plications Conference. IEEE, 2009, pp. 460–467.

[56] J. Schindelin, “Fiji is just ImageJbatteries included,” inProceedings of the ImageJ User and Developer Confer-ence, Luxembourg, 2008.

[57] A. Charles and K. Brennan, “Cortical spreading depres-sion new insights and persistent questions,” Cephalalgia,vol. 29, no. 10, p. 1115, 2009.

[58] J. Chang, L. Shook, J. Biag, E. Nguyen, A. Toga,A. Charles, and K. Brennan, “Biphasic direct currentshift, haemoglobin desaturation and neurovascular un-coupling in cortical spreading depression,” Brain, vol.133, no. 4, p. 996, 2010.

[59] D. L. Nikolic, A. N. Boettiger, D. Bar-Sagi, J. D.Carbeck, and S. Y. Shvartsman, “Role of boundaryconditions in an experimental model of epithelial woundhealing,” Am J Physiol Cell Physiol, vol. 291, pp. C68–

C75, 2006.[60] P. A. DiMilla, K. Barbee, and D. A. Lauffenburger,

“Mathematical model for the effects of adhesion andmechanics on cell migration speed,” Biophys. J., vol. 84,pp. 2907–2918, 1991.

[61] M. Poujade, E. Grasland-Mongrain, A. Hertzog, J. Jouan-neau, P. Chavrier, B. Ladoux, A. Buguin, and P. Sil-berzan, “Collective migration of an epithelial monolayerin response to a model wound,” Proc. Natl. Acad. Sci.USA, vol. 104, pp. 15 988–15 993, 2007.

[62] F. Posta and T. Chou, “A mathematical model of intercel-lular signaling during epithelial wound healing,” journalof Theoretical Biology, vol. 266, no. 1, pp. 70–78, 2010.

Documents

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XXX, NO. …IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. XXX, NO. YYY, MONTH 2011 3 reduced to their eigenspace, as indicator functions [37],