A New Adaptive Variational Model for Liver …wims.math.ecnu.edu.cn/ChunLi/Seminar_20110624/Used%20...2011/06/24 · Jialin Peng Joint Work With Dexing Kong and Fangfang Dong Zhejiang

A New Adaptive Variational Model for LiverSegmentation with Region Appearance

Propagation

Jialin Peng

Joint Work With Dexing Kong and Fangfang Dong

Zhejiang University

June 21, 2011

Jialin Peng Joint Work With Dexing Kong and Fangfang Dong Zhejiang University

A New Adaptive Variational Model for Liver Segmentation with Region Appearance Propagation

Outline

1 Backgrounds

2 Previous work and motivations

3 Proposed model

4 Numerical scheme and experimental results

5 Other applications



Backgrounds

I Liver cancer ranks at fourth place and is a rising cause of deathin the world.

I The treatment of malignant liver diseases targets at the completedestruction or removal of all tumors together with a sufficient safetyfree margin, at the same time life-critical anatomical structures mustbe saved.



Backgrounds: Live Donor Liver Transplantation

I Live Donor Liver Transplantation (LDLT) is a procedure in whicha living person donates a portion of his or her liver to another.

I It is one of the most expensive treatments in modern medicine.Numerous anastomoses and sutures, and many disconnections andreconnections of abdominal and hepatic tissue, must be made forthe transplant to succeed, requiring an eligible recipient and a well-calibrated live or cadaveric donor match.



Backgrounds

(a)

¥ Virtual resection planning¥ Quantitative assessment of treatment options



Figure: Contrast enhanced CT images, volume data 512× 512×400Jialin Peng Joint Work With Dexing Kong and Fangfang Dong Zhejiang University


Segmentation of the liver

¥ Manual segmentation of the liver is tedious, time consuming, andalso lacks of reproducibility.

¥ Manual segmentation on 2D slices generally neglects the potentialimpact of the third dimension on the results and often hard to geta regular liver surface.

¥ Highly robust and automated segmentation algorithms are favor-able.



Difficulties

¥ Low contrast and blurred edges often characterize the CTA im-ages.

¥ Complex context is often the case: adjacent organs (e.g. kidney,heart, spleen and stomach) share similar intensities, there are manyother abdominal organs.

¥ Large variability in appearance, size and shape of liver.



Computerized medical imaging analysis

¥ Fully automated and accurate segmentation is known to be anill-posed problem due to the fact that there is no clear definitionof a correct segmentation for the computer without high level priorinformation.

¥ It should be convenient to provide the radiologist with an automatic/semi-automatic segmentation system which takes few time, but segmentsthe liver accurately enough.

¥ The system should be robust to model parameters, the scanningprotocol and data quality.



Previous works for liver segmentation

Low level information based methods: intensity based region-growing, histogram processing, voxel-classification algorithms, thresh-olding with some pre- and post-processing steps.¥ Estimate the liver intensity range.

¥ A liver binary volume is created by using the threshold values.

¥ Some attached neighbor organs and irrelevant tissues are deletedby operations such as morphological operators, holefilling and con-nected component analysis.

¥ Refinement by active contours, e.g., Liu et al. [13] propose amethod which combined a GVF snake with edge detectors to re-fine the coarse liver surface obtained by intensity peak analysis andthresholding.



Drawback of previous low level information based methods

I They are sensitive to threshold values and optimal threshold valuedo not exist.

I It is difficult to isolate the liver effectively without including neigh-boring tissues with similar intensity.

I They segment the liver intuitively with several individual stepsand employ different cues in different steps.

I With only pixel level cues, e.g., intensities, it is impossible to ac-curately segment the liver from the rest of the anatomical structuresin the abdomen.



Previous works for liver segmentation

Prior level information based methods, such as shape model basedmethods, probabilistic atlas based methods and so on.

¥ Most of these methods treat segmentation firstly as a statisticalestimation problem, while the quality and the support of the train-ing set’s exemplars are often ignored. Building the training set isreally a hard work, which needs a lot of training data.

♣ Due to the large variability in appearances, size and shapeof liver, model based liver segmentation methods often fall short ofaccurate segmentation and still an interesting but challenging task.



Aim of our work

¥ What is the liver? Find the most reliable cues: prior? low level?

I Design an integrated model that takes use of multi-cuessimultaneously in a single variational model.

I The model should be robust to parameters and image qualities.

♣ Few variational models been designed to segment objectswith both blurred edges and complex backgrounds.



The Variational Framework and Motivations

Segmentation: edge extraction or image partition into disjoint,connected components that are homogeneous w.r.t. intensity,texture or certain probabilistic measures.

Three categories in variational framework: edge based, regionbased and hybrid models.



Edge based models

Edge based models: the use of edge detector to extract promi-nent edges, e.g., Geodesic Active Contour (GAC)(Caselles, Kimmel,Sapiro, 1997)

minCE (C ) =

∫ 1

0g(|∇I (C (p))|)|C ′(p)|dp,

g =1

1 + β|∇Iσ|2I Partial homogeneity is the underlying assumption for GAC .I Segment in a propagation process. Region growing is insimilar propagation style. Spatial orderI Drawbacks: Leaking out problem for blurred edges and beingsensitive to the initialization position and various image artifacts,e.g., spurious edges and noises.I Balloon force: sensitive to parameters.



Region based models

Region based models: relying on statistics of image intensity inthe philosophy of region competition. Segmentation is the processof finding a partition that provides the optimal fitting of the intensitydistribution.

I Strength: No edge information is required.

I Challenges: high level noise, intensity inhomogeneity, complexintensity distribution.



Region competition (S.Zhu-A.Yuille 96):

minC ,Θi

∑

i

∫

Ωi

−logp(I (x) | Θi ) + β | Γ |

CV: p(I (x) | Θi ) ∼ Gaussian distribution with the same variance.

I Piecewise homogeneity is the underlying assumption and at thesame time there is little intensity overlapping between them.

I The segmentation is taking with no spatial order.

I For multi-phase segmentation, phase number is predefined be-forehand and the computation complexity grows drastically as phasenumber grows.



Models that are close related to our proposed model:

♣ Region growing: discrete, hard thresholds and non-regular surface

I Problem: discrete → continues?∫

Ωf

H(I > µ)dΩ

♣ Zhang’s model [4] Hybrid:

∫ 1

0g(|∇I (C (p))|)|C ′(p)|dp+

∫

Ωf

(I − µ)dΩ

I Problems: hard thresholds, poor balance, weak GAC term.

The underlying assumption is that the surround context must haveobviously lower intensity values than the foreground object.



Three key observations

Three key observations are vital for the liver segmentation and arethe foundation of our model:

I The boundary is the most reliable cue to delineate the liver region,while it may faint.

I The liver image meets partial homogeneity. The intensities andregion appearance are homogeneous in main organs such as liver,spleen, bone and so on.

I While it is often confused to recognize the liver from other organs,the region intensity and appearance information of the initial regioncan be token as the most reliable information.



The philosophy of our proposed method

For this liver segmentation problem from complex context, we willnot follow the philosophy of region competition.I The most important cue for correct segmentation in CTA imagesis the edge information.

I Segment the liver in a process of multi-cues information propa-gation.

I Edge information, intensity range information and region appear-ance, are integrated and leaking out problem are greatly alleviate byregion appearance information.

I Adaptively balance of different terms of the model is also thefocus of us. However the image features, e.g., edge information, arenot uniformly strong.



Proposed model

As we have observed that the intensity of the liver region, in thecontrast enhanced CTA image, roughly lie in a range. Suppose theintensity range is [µ, η]. A rough estimate of [µ, η] can be obtainedfrom the initial region and in experiments we set the intensity rangeas [µ − 3σ, µ + 3.5σ], which is kept for most of our data sets andresult in stable results.



Proposed model I: integrating intensity information andedge information.

Model I : Hybrid model with Local Information adaptive Thresh-olding

minCER(C ) =

∫ 1

0g(|∇I |)|C ′(p)|dp+

∫

Ωin

w(x)(I − µ)(I − η)

(η − µ)2dΩ,

Ωin is the foreground region enclosed by the contour C and w(x) isthe weight to spatially balance the region and edge term accordingto the local image property.Let

R = w(x)f , f (x) =(I − µ)(I − η)

(η − µ)2,

f < 0, when f (x) ∈ [µ, η]Jialin Peng Joint Work With Dexing Kong and Fangfang Dong Zhejiang University


Proposed model：Model I

The optimal segmentation minimizing ER(C ) is computed throughevolving an active contour.

∂C

∂t= gκ− 〈∇g ,

−→N 〉+ w(x)

(I − µ)(η − I )

(η − µ)2−→N , (1)

where−→N and κ are the inward normal and the curvature of the

contour C respectively. Let

υ = gκ −〈∇g ,−→N 〉︸︷︷︸

bi−direction−force

+w(x)(I − µ)(η − I )

(η − µ)2︸︷︷︸bi−direction−force

.

The first two terms in υ correspond the first term in energy functionalER(C ) and the third term corresponds to our new region based term.




I To avoid directly tackling complex backgrounds, a contour orsurface will be firstly initialized inside the liver region. Then theactive contour evolves in speed υ and classifies pixels on it as theforeground or backgrounds in each step.I It is obvious that only minimizing the second region based termencourages the contour to enclose the regions with gray-levels lyingin [µ, η]. As a result,when setting w(x) = Constant, the secondterm amounts to region growing in continuous from with predefinedaccept range [µ, η].

∂C

∂t= α

(I − µ)(η − I )

(η − µ)2−→N




I With the spatially weight w(x), the pixel with intensity lying in[µ, η], may be also classified as background when w(x) ≈ 0.

I The spatially weight can be set simply as

w(x) = αg(x) =α

1 + β|∇I |2 ,

Balanced with the GAC term in ER , our new region term can beregarded as a Gradient-adaptive Thresholding term.

♣ Other choice can be local variance information



Proposed model: Model I

∂C

∂t= gκ− 〈∇g ,

−→N 〉+ w(x)

(I − µ)(η − I )

(η − µ)2−→N

Set R = w(x) (I−µ)(I−η)(η−µ)2

I R is a bi-direction force.I R is aware of edge.I R is able to treat blurred edges.I The region based term and edge based term. dominate in differentregions and sometimes are combined.I With w(x) = Constant, the model will sensitive to parameters,e.g., µ and η and lead to leaking out problems.



Some segmentation results of Model I



The effect of w(x)

(a) (b) (c)

Figure: The the effect of w(x) to segmentation results. (a) The originalimage. (b) Segmentation results with w(x) = α with [m− 3σ,m + 3.5σ].(c) Segmentation results by our model, with intensity range [m− 3σ,m +3.5σ].



Model II: Enhanced Model with Region Appearance(RAP)

♣ Only pixel level information such as gray level intensity informationand gradient information can not well discriminate the liver regionfrom other tissues.

(a) (b) (c)

Figure: (a) The original image. (b) Segmentation results by Model 1 withintensity range [m − 3σ,m + 3.5σ]. (c) Segmentation results with ourenhanced model with intensity range [m − 3σ,m + 3.5σ].




♣ Model 1 propagates out intensity and edge information♣ Propagate out region appearance information? ! Different or-gans with different properties such as intensity statistics, texturedistribution and so on.♣ The local property of the boundary is quite different.♣ What is region appearance? How to keep the region appearanceof the liver consistent?




♠ Region appearance is modeled by the distribution of some keyfeatures which can be pointwise or patchwise .♠ The consistency is measured by some distribution metric W (P1,P2).♠ Probability density function (PDF) can be estimated by paramet-ric and nonparametric method. Histogram can be used as a roughapproximate of probability density functionIn this Region Appearance Propagation framework, we have greatfreedom to choose feature descriptors, i.e., features that describeslocal structures of the image , and distribution metrics.




Feature mapping:

FI : x ∈ Ω 7→ FI (x) ∈ Rb

which maps an image I (x) : Ω 7→ RN×N to its feature space. b isthe dimensional of the feature vector.The simplest choice of FI is the identity mapping I and b = 1:

FI (x) := I(I (x)) = I (x).

Other choice can be : FI (x) := f (x), where b = 1; FI (x) :=∇I (x), in which case b = 2. Other proper choices such as LocalBinary Pattern (LBP) can also be made.I The distribution of the feature ”signatures” FI (x) over an imageregion provides a discriminative tool to both segment the liver fromadjacent organs and control the region appearance consistency ofthe foreground region




ITo keep consistency of the region appearance of the foregroundliver region, we keep to measure the similarity of the local region ap-pearance and the region appearance learned from the initial region.I Each pixel is initially assigned a local estimated histogram, i.e. anormalized histogram of the feature ”signatures” in a neighborhoodof that pixel.I We select the distance measure introduced by Chan et al [3], i.e.,the particular Wasserstein distance as the distance measure of twoPDFs P and Q.

W (P,Q) = W (F ,G ) =

∫ L

0|F (y)− G (y)|dy ,

where L is the number of the gray levels. F (y) and G (y) are thecumulative distribution functions of P and Q respectively.




LetP(x) = W (F0,Fx).

Our enhanced model energy function reads

minC∫ 1

0g |C ′(p)|dp+

∫

Ωin

w(x)(I − µ)(I − η)

(η − µ)2dΩ+γ

∫

ΩinP(x)dΩ,

where F0 is the cumulative histogram of the liver region and is of-ten estimated from the initial region histogram and Fx is the localhistogram at point x .




The corresponding gradient flow equation is

∂C

∂t= gκ− 〈∇g ,

−→N 〉+ w(x)

(I − µ)(η − I )

(η − µ)2− γP(x)−→N . (2)

I P(x) is a constraining field.I Blurred edges are enhanced by local region appearance and thesegmentation process is more stable.I Adjacent organs are them further.I The model is more stable to intensity range [µ, η].I Vessels can be segmented with precise.



Numerical scheme and experimental results

I Our model can be solved with an Operator Splitting Scheme(AOS), which is unconditional stable and allows the decompositionof the multidimensional problem into several one-dimensional ones.

∂φ

∂t= |∇φ|

[∇·

(g∇φ

|∇φ|)

+ w(x)(I − µ)(η − I )

(η − µ)2− γP(x)

](3)



Blurred edges

(a) (b) (c)

Figure: With the enhanced model, a.e equation 6, we get more reliableresults. (a) The original image; (b) Segmentation results by model I withintensity range [m − 3σ,m + 3.5σ]; (c) Segmentation results with ourenhanced model with [m − 3σ,m + 3.5σ].



Comparison results

Figure: Comparison of Segmentation results with equ. The β in g andw(x) is set the same (a) Original image. (b) Segmentation results withballoon force GAC model. (c) Segmentation results with our model 1. (d)Segmentation results with our enhanced model.



Robustness of our model to parameters

(a) (b) (c) (d)

(e) (f) (g) (h)

Figure: The estimated σ is 15. The results show that our model isinsensitive intensity range In the first column (a) the original image.(e) The ground truth. In the second column, results by model whenw(x) = Constant with intensity range (b) [m − 3.0σ,m + 3.5σ] and (f)[m − 4.5σ,m + 3.5σ]. In the third column, results by model 1 with in-tensity range (c) [m − 3.0σ,m + 3.5σ] and (g) [m − 4.5σ,m + 3.5σ]. Inthe forth column, results by our enhanced model with intensity range (d)[m − 2.5σ,m + 3.5σ] and (h) [m − 7.5σ,m + 3.5σ].



Segmentation of vessels

(a) (b) (c)

Figure: Comparison of segmentation results of liver vessels More reliableresults can be get by our enhanced model. (a) The original image. (b) Seg-mentations with model GAC with balloon force model. (c) Segmentationswith our model 1. (d) Segmentations with our enhanced model.



Other applications

Figure: 2D segmentation results of the enhanced model with the fist andsecond column for liver and third and fourth for spleen. Segmenting thespleen on this case is very convolved for there boundary is two weak. Forthe enhanced model, W (F0,Fx)is higher near the weak boundary and theintensity distribution of liver is slightly different from the spleen.



The segmentation of the cholecyst

(a) (b)

Figure: The segmentation results of the cholecyst.



V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active contours.Intl. J. of Computer Vision,1997.

T. Chan and L. Vese. Active contours without edges. IEEETrans. Image Processing10(2):266-277, February 2001.

K. Ni , X. Bresson , T. Chan,S.Esedoglu. Local Histogram basedSegmentation using the Wasserstein Distance

Y.Zhang. , B.J. Matuszewski, L.-K. Shark, C.J Moore.: Medi-cal Image Segmentation Using New Hybrid Level-Set Method.In: Proceedings of the 2008 Fifth International ConferenceBioMedical Visualization: Information Visualization in Medicaland Biomedical Informatics 2008.

S.C.Zhu and A.Yuille.Region competition: Unifying snakes, re-giongrowing, and Bayes/MDL for multiband image segmenta-tion. IEEE Transactionson Pattern Analysis and Machine Intel-ligence, 1996. ...



Thanks！



Documents

A New Adaptive Variational Model for Liver …wims.math.ecnu.edu.cn/ChunLi/Seminar_20110624/Used%20...2011/06/24 · Jialin Peng Joint Work With Dexing Kong and Fangfang Dong Zhejiang