Pattern Recognition 76 (2018) 367–379
Contents lists available at ScienceDirect
Pattern Recognition
journal homepage: www.elsevier.com/locate/patcog
Weighted variational model for selective image segmentation with
application to medical images
Chunxiao Liu
a , Michael Kwok-Po Ng
b , Tieyong Zeng
b , c , ∗
a Department of Mathematics, Hangzhou Normal University, Hangzhou 310036, PR China b Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong c HKBU Institute of Research and Continuing Education Shenzhen Virtual University Park, Hong Kong
a r t i c l e i n f o
Article history:
Received 18 January 2017
Revised 1 November 2017
Accepted 16 November 2017
Available online 20 November 2017
Keywords:
Selective segmentation
Mumford-Shah model
Thresholding
Medical images
Iterative algorithm
a b s t r a c t
Selective image segmentation is an important topic in medical imaging and real applications. In this pa-
per, we propose a weighted variational selective image segmentation model which contains two steps.
The first stage is to obtain a smooth approximation related to Mumford-Shah model to the target region
in the input image. Using weighted function, the approximation provides a larger value for the target
region and smaller values for other regions. In the second stage, we make use of this approximation and
perform a thresholding procedure to obtain the object of interest. The approximation can be obtained
by the alternating direction method of multipliers and the convergence analysis of the method can be
established. Experimental results for medical image selective segmentation are given to demonstrate the
usefulness of the proposed method. We also do some comparisons and show that the performance of the
proposed method is more competitive than other testing methods.
© 2017 Elsevier Ltd. All rights reserved.
1
i
t
c
b
[
s
o
fl
T
s
n
t
a
i
e
t
o
t
s
m
a
s
t
d
s
a
g
s
p
t
s
M
p
f
c
m
t
h
0
. Introduction
Image segmentation is a fundamental task in image process-
ng. In this field, models based on variational methods and par-
ial differential equations are popular due to their flexibility and
omputational convenience. Variational segmentation models can
e classified into edge based models [1–3] and region based ones
4] . Edge based methods utilize the local properties of the first and
econd order derivatives of the image. The main weakness is a lack
f robustness in dealing with image noise. The well-known and in-
uential region based model is the Mumford–Shah (MS) model [4] .
he variational problem leads to a piecewise smooth solution with
mooth boundaries. In numerical computation, parameterization is
eeded and topological changes are not allowed during the itera-
ions.
In some applications, such as surgery simulation, medical di-
gnosis, object tracking, etc, people aim to extract only objects of
nterest from an image. In such cases, selective segmentation mod-
ls are more appropriate. There is relatively less work done for this
opic. Gout et al. [5] introduced a geometry constraint using a set
f marker points on the contour of interest to the geodesic ac-
∗ Corresponding Author.
E-mail addresses: [email protected] (M.K.-P. Ng), [email protected]
(T. Zeng).
p
n
p
t
l
ttps://doi.org/10.1016/j.patcog.2017.11.019
031-3203/© 2017 Elsevier Ltd. All rights reserved.
ive contour model. Later, Badshah and Chen [6] improved the re-
ults by combining a region term. With the region information, the
odel became more robust especially for noisy images. In [7] , the
uthors proposed an adaptively varying narrow band algorithm to
egment a larger class of image segmentation problems and cap-
ure more complex features. A new variational model was intro-
uced in [8] with two level set functions which can do two tasks
imultaneously. One is to find the segmentation of all boundaries
nd the other focuses on the selected object that is close to the
eometry constraints. An intensity term was proposed for 3D liver
egmentation in [9] . In [10] , a selective segmentation model was
roposed by utilizing the average image of channels, which can ex-
ract textural and inhomogeneous objects.
The main purpose of this paper is to propose a new two-
tage weighted selective image segmentation model based on the
umford–Shah model. In the first stage, we choose several marker
oints around the objects of interest and construct a weighted
unction by using an edge function and a distance function. By in-
orporating this weighted function into a convex second-order seg-
entation model, we can obtain a smooth approximation to the
arget region. Existence and uniqueness of the minimizer of the
roposed segmentation model can be guaranteed. Many effective
umerical algorithms can be used by virtue of the convexity of the
roposed model. We illustrate clearly how to use alternating direc-
ion method to get a suitable numerical algorithm and prove the
inear convergence under mild conditions. In the second stage, we
368 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
2
i
s
i
p
d
d
w
z
f
a
d
r
d
g
w
m
w
t
�
w
fi
C
t
p
m
3
m
t
m
s
3
m
t
u
w
i
ω
make use of this approximation function and perform a threshold-
ing procedure to obtain the object of interest.
This paper is organized as follows. In Section 2 , we briefly re-
view the related segmentation models. In Section 3 , we present
our model and develop the related basic mathematical properties.
The numerical algorithm and its convergence analysis are given
in Section 4 . Numerical experiments are then listed in Section 5 ,
which clearly show the good performance of our approach. Finally,
we conclude the paper in Section 6 .
2. Related works
2.1. Global segmentation models
The level set method [11] is a classical method to solve curve
evolution problems. The interface is represented implicitly by the
zero level set of a Lipschitz continuous level set function (LSF) and
topological changes are allowed. Discretization schemes can be im-
plemented on fixed mesh grids. For images with piecewise con-
stant intensities, Chan and Vese proposed the two-phase MS model
[12] and the multiphase case [13] . Piecewise constant level set
methods [14–16] use discontinuous functions to represent different
phases. Methods of fuzzy membership function [17,18] use contin-
uous functions ranging from 0 to 1 to represent the probability of
belonging to some region. Convex relaxation methods [19,20] and
graph cut method [21] were proposed to overcome the local min-
imum problem of the MS model. Fast continuous max-flow meth-
ods were proposed in [22] . Recently, Cai et al. [23] proposed a two-
stage segmentation model using a convex variant of the MS model,
which can be regarded as a unification of image restoration and
image segmentation. They first found a unique smooth minimizer
by the split-Bregman algorithm [24,25] and then segmented the
image by a thresholding strategy. Furthermore, they introduced a
K-means method to choose the thresholds automatically. In [26] ,
Cai proposed a new multiphase segmentation model by com-
bining image restoration and image segmentation models. Multi-
atlas segmentation methods have been proposed in [27,28] and
proved to be useful. In [29,30] , nonparametric statistical segmenta-
tion methods were proposed to handle texture features. A wavelet
method with shape prior was proposed in [31] for individual image
and video segmentations. For more about segmentation methods,
please refer to [32–40] and the references therein.
Let � ⊂ R
2 be a bounded connected open set with Lipschitz
boundary. Let � be a compact curve in � and f : � → R be a
given image. The MS model aims to solve the following minimiza-
tion problem [4] :
inf u, �
{H
1 (�) +
α
2
∫ �\ �
|∇u | 2 dx +
β
2
∫ �
| u − f | 2 dx
},
where α and β are positive constants and H
1 is the one-
dimensional Hausdorff measure. For regular curves, H
1 (�) denotes
the length of �.
In [23] , a two-stage variational image segmentation was pro-
posed. The first stage is to find a smooth approximation to the
original image by solving the following convex minimization prob-
lem:
min
u ∈ W
1 , 2 (�)
{∫ �
|∇ u | dx +
α
2
∫ �
|∇ u | 2 dx +
β
2
∫ �
| u − f | 2 dx
}, (1)
where the Sobolev space W
1 , 2 (�) = { v ∈ L 2 (�) | ∂ i v ∈ L 2 (�) , i =1 , 2 } with L 2 (�) = { f (x )
∣∣( ∫ � f 2 (x ) dx
)1 / 2 < ∞} . Once u is obtained,
then in the second stage, the segmentation is obtained by thresh-
olding u properly. The thresholds can be provided by the users or
obtained automatically by any clustering methods such as K-means
or convex K-means methods [41,42] .
.2. Selective segmentation models
Selective segmentations that extract objects of interest from an
mage are important and challenging in a number of areas, such as
ecurity monitoring and medical diagnosis. Assume that K points
nside the target are available on the image. Using these marker
oints X = { x 1 , x 2 , · · · , x K } , a distance function d(x ) : � → R can be
efined as [6] ,
(x ) =
K ∏
i =1
(1 − e −
| x −x i | 2 2 h 2
), x ∈ �, (2)
here h is a positive constant. The distance function is close to
ero near the marker points and approximates to one far away
rom the marker points. There are other ways to define d ( x ), such
s
(x ) = min
y ∈ X | x − y | . (3)
To use the edge information, an edge detector is often incorpo-
ated into segmentation models. A popular way to define an edge
etection function is
(x ) =
1
1 + β|∇ f (x ) | 2 , (4)
here β > 0 adjusts the strength of edge.
In [5] , an edge-based model was proposed as follows:
in
�
∫ �
d · gds,
ith d defined by (3) .
Badshah and Chen [6] improved the model by introducing in-
ensity fitting terms similar to Chan-Vese model [12] as follows:
min
,C 1 ,C 2
{
α
∫ �
d · gds + λ1
∫ �in
| f − C 1 | 2 dx + λ2
∫ �out
| f − C 2 | 2 dx
}
,
(5)
here α, λ1 and λ2 are constants to adjust the smoothing and the
delity term, � is the boundary between �in and �out and C 1 and
2 are constants to be optimized. Under the level set framework,
hey need to solve the Euler-Lagrange equation of the LSF.
Later in [7] , a new adaptive local band level set method was
roposed. Nguyen [43] combined a marker set and another anti-
arker set to obtain a selective segmentation model.
. The proposed weighted model
In this section, we propose our new model and present the
athematical analysis. Our method consists of two stages. In
he first stage, we solve a minimization problem based on some
arker points and obtain a smooth minimizer. Then in the second
tage, segmentation can be carried out by simple thresholding.
.1. Proposed model
Inspired by the convex model (1) and the above selective seg-
entation models, we propose the following weighted model for
he task of selective image segmentation:
min
∈ W
1 , 2 (�)
{
E(u ) :=
∫ �
|∇ u | dx+
α
2
∫ �
|∇ u | 2 dx +
β
2
∫ �
ω
2 | u − f | 2 dx
}
,
(6)
here ω is a weight function to adjust the fidelity and the smooth-
ng terms. In this paper, we use
2 (x ) = 1 − d(x ) g(x ) , ω(x ) ∈ (0 , 1] , (7)
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 369
Fig. 1. First row: segmentation results of [5] ; Second row: segmentation results of [10] with initial LSF Fig. 1 e; Third row: segmentation results of [10] with another initial
LSF Fig. 1 i; Fourth row: segmentation results of our model.
370 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
Fig. 2. || u k +1 −u k || 2 || u k || 2 (left) and || v k +1 −v k || 2
|| v k || 2 (right) vs. the iteration number.
T
|
a
|
M
f
|
C
w
v
e
E
P
m
P
b
t
where d ( x ) and g ( x ) are defined by (2) and (4) , respectively.
In practice, d ( x ) is small around the marker points, while g ( x ) is
small around the boundaries. Since the marker points are chosen
near the boundaries, the weight function ω( x ) in (7) is large near
the boundaries and becomes smaller far away from the boundaries.
Therefore, we conclude that:
(I) Near the boundaries, the third term in (6) , which is the fi-
delity term, plays an important and details are remained.
(II) Far away from the boundaries, smoothing plays more and
more important roles. As we can seen from the following
numerical results, complicated structures far away from the
edge are smoothed out and only the outline of the target
object remains.
We make some comparisons between our model and other ex-
isting selective segmentation models here:
• The existing selective models need to use level set methods
or membership functions in numerical computation, which can
bring additional computational efforts. • In the performed experiments, references [5,10] show sensitiv-
ity to the initialization, differently from the proposed model
whose convexity is demonstrated in Section 3.2 . For our model,
existence and uniqueness of the minimizer can be guaranteed.
Many stable and effective numerical algorithms can be devel-
oped.
Once we obtain the solution of (6) , we can obtain the segmen-
tation by thresholding in the second stage.
3.2. Mathematical analysis
Proposition 1. Suppose f ∈ L 2 ( �) and inf x ∈ �ω( x ) > 0 . Then model
(6) is strictly convex and there exists a unique minimizer
u ( x ) ∈ W
1, 2 ( �) .
Proof. By the definition of ω( x ) in (7) and the condition
inf x ∈ �ω( x ) > 0, we have the boundness of ω( x ). We suppose
M 1 ≤ω( x ) ≤ M 2 , where M 1 and M 2 are two positive constants.
Choose u = 0 , we have
00 ≤ inf u ∈ W
1 , 2 (�) E(u )
≤ E(u 0 ) =
β
2
∫ �
ω
2 f 2 dx
≤ M
2 2 β
2
|| f || 2 L 2 (�)
< + ∞ .
hus inf u ∈ W
1 , 2 (�) E(u ) must exist.
We now prove that E ( u ) is coercive. It is obvious that
|∇u || L 2 (�) ≤√
2
αE(u ) (8)
nd
| u || L 2 (�) ≤ || u − f || L 2 (�) + || f || L 2 (�) . (9)
eanwhile,
0 ≤ M
2 1 β
2
∫ �
| u − f | 2 dx
≤ β
2
∫ �
ω
2 | u − f | 2 dx ≤ E(u ) ,
rom which we obtain
| u − f || L 2 (�) ≤√
2
M
2 1 β
E(u ) . (10)
ombining (8), (9) and (10) , we have
|| u || W
1 , 2 (�) ≤ || u || L 2 (�) + ||∇u || L 2 (�)
≤( √
2
α+
√
2
M
2 1 β
) √
E(u ) + || f || L 2 (�) ,
hich means that E ( u ) is coercive.
Note that W
1, 2 ( �) is a reflective Banach space, and E ( u ) is con-
ex, lower semicontinuous (l.s.c.) and coercive. We conclude the
xistence of the minimizer of E ( u ) in W
1, 2 ( �) [23,44] .
Finally, uniqueness can be guaranteed by the strict convexity of
( u ). �
roposition 2. Suppose f ∈ L 2 ( �), inf x ∈ �ω( x ) > 0, the unique mini-
izer u ∗( x ) of model (6) satisfies inf x ∈ � f (x ) ≤ u ∗(x ) ≤ sup x ∈ � f (x ) .
roof. From Proposition 1 , it is clear that E ( u ) is proper. Let { u n }
e a minimizing sequence. Then there exists a constant M > 0 such
hat E ( u n ) ≤ M for all n ∈ N . Therefore, we have ||∇u n || L 2 (�) are
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 371
Fig. 3. First row: segmentation results of [5] ; Second row: segmentation results of [10] ; Third row: segmentation results of our model.
u
f
|i
W
s
E
u
d
w
ω
L
∫
niformly bounded. Moreover,
βM
2 1
2
|| u n − f || 2 L 2 (�) ≤β
2
∫ �
ω
2 | u n − f | 2 dx ≤ M, for all n ∈ N ,
rom which we conclude that || u n − f || L 2 (�) is uniformly bounded.
Then we have
| u n || L 2 (�) ≤ || u n − f || L 2 (�) + || f || L 2 (�)
s uniformly bounded.
Therefore, up to a subsequence, u n converges strongly in
1, 2 ( �) to some u ∗ and ∇u n converges weakly as a mea-
ure to ∇u ∗. Since E ( u ) is lower semicontinuous, we have
( lim inf n →∞
u n ) ≤ lim inf n →∞
E(u n ) , which implies u ∗ is the
nique solution to (6) .
Let α = inf f and β = sup f . We remark that x → ω
2 | x − f | 2 is
ecreasing in (0, f ) and increasing in ( f, + ∞ ) . Therefore, if C ≥ f ,
e have
2 | min (x, C) − f | 2 ≤ ω
2 | x − f | 2 .
et C = β = sup f, we obtain
�ω
2 | min (u
∗, β) − f | 2 dx ≤∫ �
ω
2 | u
∗ − f | 2 dx. (11)
372 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
Fig. 4. (a) The original liver CT image with six marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.
Fig. 5. (a) The original CT image with eight marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.
Fig. 6. (a) The original image with two marker points. (b) The smoothed solution u . (c) Thresholding of u . (d) Segmentation.
4
g
m
w
a
i
c
m
W
s
m
In the same way we can prove that ∫ �
ω
2 | sup (u
∗, α) − f | 2 dx ≤∫ �
ω
2 | u
∗ − f | 2 dx. (12)
On the other hand, from Proposition 15 in [45] , both min ( u ∗, β)
and sup (u ∗, α) are members of W
1, 2 ( �) and
|∇ ( min (u
∗, β)) | ≤ |∇ u
∗| , |∇ ( sup (u
∗, α)) | ≤ |∇u
∗| . (13)
Combing (11), (12) and (13) , we have
E( min (u
∗, β)) ≤ E(u
∗) , E( sup (u
∗, α)) ≤ E(u
∗) ,
which implies that α ≤ u ∗ ≤β . �
4. Numerical optimization
Since model (6) is convex, there can be many efficient numeri-
cal methods to solve it [17,24,46–50] . In this section, we present an
alternating direction method (ADM) [17,51] to solve it and provide
convergence analysis of the algorithm.
.1. Algorithm
Rewriting the minimization problem (6) into a matrix form
ives
in
u
{|| Au || 1 +
α
2
|| Au || 2 2 +
β
2
|| ω(u − f ) || 2 2
}, (14)
here A denotes the matrix of the gradient operator ∇ and u is
n n 2 -by-1 vector corresponding to an n -by- n image. By introduc-
ng an auxiliary variable v = Au , (14) is equivalent to the following
onstrained optimization problem:
in
u , v
{|| v || 1 +
α
2
|| v || 2 2 +
β
2
|| ω(u − f ) || 2 2
}subject to v = Au .
e apply the quadratic penalty method [4 8,4 9] to enforce the con-
traint and solve the following unconstrained problem:
in
u , v
{|| v || 1 +
α
2
|| v || 2 2 +
β
2
|| ω(u − f ) || 2 2 +
μ
2
|| v − Au || 2 2
}, (15)
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 373
Fig. 7. First row: segmentation results of [23,26] ; Second row: segmentation of the triangle using our method; Fourth row: segmentation of the ring using our method. Third
row: segmentation of the triangle and the ring together.
w
u
s
v
v
here μ> 0 is a penalty parameter.To solve (15) , we use the ADM and minimize with respect to
and v alternatingly. From an initial guess u
0 for u , we obtain a
equence
1 , u
1 , v 2 , u
2 , . . . , v k , u
k , v k +1 , u
k +1 , . . .
ia the following two subproblems:
v k +1 = R (u
k ) := arg min
v
{
|| v || 1 +
α
2
|| v || 2 2 +
μ
2
|| v − Au
k || 2 2
}
(16)
374 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
Fig. 8. (a) The original image f with eight marker points. (b) The image (1 − f ) with eight marker points. (c) The smoothed solution u . (d) Segmentation.
S
i
m
p
F
f
p
A
4
r
(
v
W
b
L
a
H
s
|
and
u
k +1 = S(v k +1 ) := arg min
u
{β
2
|| ω(u − f ) || 2 2 +
μ
2
|| v k +1 − Au || 2 2
},
(17)
for k = 0 , 1 , 2 , . . . , which we call the iterations the outer-ADM-
iterations. We now solve the subproblems separately.
4.1.1. v -Subproblem
Minimization problem (16) can be reformulated as
v k +1 = R (u
k ) = arg min
v
{
|| v || 1 +
α + μ
2
|| v − μ
α + μAu
k || 2 2
}
.
(18)
Suppose v = (v 1 , v 2 ) ,
y 1 =
μ
α + μ∇ x u
k , y 2 =
μ
α + μ∇ y u
k , | y | =
√
y 2 1
+ y 2 2 ,
then (18) can be solved explicitly using a generalized shrinkage op-
erator [24] as follows:
v k +1 1 = max
(| y | − 1
α + μ, 0
)y 1 | y | ,
v k +1 2 = max
(| y | − 1
α + μ, 0
)y 2 | y | .
(19)
4.1.2. u -Subproblem
In order to solve (17) efficiently, we introduce another auxiliary
variable p = u − f . We can use the split-Bregman method to en-
force the constraint strictly. With b
0 = 0 , we have at step (n + 1)
(p
n +1 , u
n +1 ) = arg min
p , u
{
β
2
|| ωp || 2 2 +
λ
2
|| p − u + f − b
n || 2 2
+
μ
2
|| v k +1 − Au || 2 2
}
,
b
n +1 = b
n − p
n +1 + u
n +1 − f . (20)
The minimization problem can be solved effectively by minimiza-
tion with respect to u and p alternatingly. Firstly, the minimization
with respect to u reads
u
n +1 = arg min
u
{λ
2
|| p
n − u + f − b
n || 2 2 +
μ
2
|| v k +1 − Au || 2 2
},
which gives
(λI + A
T A ) u
n +1 = λ(p
n + f − b
n ) + μA
T v k +1 . (21)
ince the matrix (λI + A
T A ) is positive definite and hence invert-
ble, we can use the fast Fourier transforms to solve (21) . Then, the
inimization for p is
n +1 = arg min
p
{β
2
|| ωp || 2 2 +
λ
2
|| p − u
n +1 + f − b
n || 2 2
}.
rom its Euler-Lagrange equation we have an explicit expression
or p :
n +1 =
λ(u
n +1 − f − b
n )
βω
2 + λ. (22)
Unifying all the above steps, we obtain Algorithm 1 .
lgorithm 1 Solving (15) by ADM.
Choose μ and λ; Initialize u
0 = 0 . Choose the tolerance ε. Set
k = 1 ;
• Step 1. Update v k by (19).
• Step 2. Set b
0 = 0 and p
0 =0.
Do n = 0 , 1 , 2 , . . . , N until || u k,n +1 −u k,n || 2
|| u k,n +1 || 2 < ε.
(i) Update u
k,n by (21).
(ii) Update p
n by (22).
(iii) Update b
n by (20).
When iterations (i)-(iii) terminate, we set u
k = u
k,N .
• Step 3. Test convergence. If the stopping criterionis satisfied,
the algorithm stops. Otherwise, set k = k + 1 andgo to Step 1.
.2. Convergence analysis
In the following, we will investigate the convergence of Algo-
ithm 1. Recall that the relationship between v k and v k +1 from
16) and (17) can be expressed as
k +1 = R (S(v k )) := T (v k ) .
e will analyze the convergence of the sequence { v k } generated
y T .
emma 1. [48] If φ is a convex, l.s.c. proper function, then the oper-
tor
A φ : x → arg min
y
{φ(y ) +
1
2
|| x − Ay || 2 2
}atisfies:
| A H
A φ(x 1 ) − A H
A φ(x 2 ) || 2 ≤ || x 1 − x 2 || 2 .
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 375
Fig. 9. Influence of the number of marker points. Blue lines: ground truth drawn by an expert; Red lines: segmentations of our model.
D
i
〈w
L
T
|P
q
P
t
|
S
w
|
w
(
S
a
T
f
l
i
t
efinition 1. A convex, l.s.c. proper function φ is called c-strongly
f there exist a positive constant c such that
∂φ(x 1 ) − ∂φ(x 2 ) , x 1 − x 2 〉 ≥ c|| x 1 − x 2 || 2 2 , ∀ x 1 , x 2 ,
here ∂ denotes the partial derivative.
emma 2. [48] Let φ be a convex, l.s.c. proper c-strongly function.
hen the operator H φ : x → arg min y { φ(y ) +
1 2 || y − x || 2 2 } satisfies:
| H φ(x 1 ) − H φ(x 2 ) || 2 ≤ 1
1 + c || x 1 − x 2 || 2 .
roposition 3. If the subproblem (17) is solved exactly, then the se-
uence { v k } converges to the limit point v ∗.
roof. For (17) , it is obviously φ1 (y ) :=
β2 μ || ω(y − f ) || 2
2 satisfies
he conditions of Lemma 1 . Therefore, we obtain immediately that
| A S(v 1 ) − A S(v 2 ) || 2 ≤ || v 1 − v 2 || 2 . (23)
uppose φ2 (y ) :=
1 μ || y || 1 +
α2 μ || y || 2
2 in (16) , then φ2 is c -strongly
ith c =
αμ . Applying Lemma 2 to (16) , we have
|R (u 1 ) − R (u 2 ) || 2 ≤ 1
1 + q || Au 1 − Au 2 || 2 , (24)
here q =
αμ > 0 .
Let T = RS, u 1 = S(v 1 ) and u 2 = S(v 2 ) . Combining (23) and
24) , we have
||T (v 1 ) − T (v 2 ) || 2 = ||RS(v 1 ) − RS(v 2 ) || 2 = ||R (u 1 ) − R (u 2 ) || 2 ≤ 1
1 + q || Au 1 − Au 2 || 2
=
1
1 + q || A S(v 1 ) − A S(v 2 ) || 2
≤ 1
1 + q || v 1 − v 2 || 2 .
ince 1 / (1 + q ) < 1 , the operator T is a contract operator and has
unique fixed point. Denote the fixed point of T by v ∗, we have
(v ∗) = v ∗ and
|| v k +1 − v ∗|| 2 || v k − v ∗|| 2 =
||T (v k ) − T (v ∗) || 2 || v k − v ∗|| 2 ≤ 1
1 + q ,
rom which we conclude that the algorithm converges at least Q-
inearly. �
Once we obtain the smoothed solution u of (6) , the second step
s to segment the target object by thresholding of u . We will illus-
rate the thresholding methods for our model in Section 5 .
376 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
Fig. 10. Influence of the position of marker points. (a) Three marker points are far away from the boundary; SA = 84.96%. (b) Four marker points are outside but near the
boundary; SA = 95.86%. (c) All marker points are inside the boundary; SA = 96.05%.
Table 1
CPU time in seconds for Figs. 1 and 3 .
[5] [10] Our method
Fig. 1 4.40 3.93 3.52
Fig. 3 4.56 4.17 3.18
f
t
s
t
m
(
p
a
|
w
i
s
F
i
f
T
l
t
i
T
b
r
m
5
i
[
H
s
o
t
f
5. Numerical results
We provide several numerical results in MATLAB to demon-
strate the stability and efficiency of our algorithm. Some medical
images are chosen as test images. All the images are scaled into
the interval [0, 1] by the linear-stretch formula:
ˆ f =
f − f min
f max − f min
,
where f max and f min are the maximum and minimum of the pixel
values of f , respectively.
We set α = 1 , μ = 15 and λ = 10 in all experiments. We use
green stars to indicate the marker points. The parameter h in
(2) should be slightly changed according to the size of the target.
Generally, large objects require large values of h and small objects
require smaller ones.
The threshold parameter in our method is similar to that of
[23] . The user can try different thresholds by try and error. Note
that we need not to recompute our convex model (6) when the
thresholds change. This virtue of our method can save much more
time compared with other methods which have to solve the min-
imization problems again if the threshold is changed [12,52] . Be-
sides, the values of threshold are relatively stable for the same tis-
sue, which can help us to choose suitable threshold parameters.
5.1. Experiment 1
In this subsection, we segment some pictures with the property
that the average intensity of the target area is larger than that of
the surrounding area. Results are presented in Figs. 1 to 6 . In Fig. 1 ,
we present the segmentation results of a bone image. We compare
our method with the selective segmentation models of [5,10] . For
comparison, we choose the same three marker points for different
methods. The first row presents the segmentation results of the
method in [5] . We can see that the model of Gout et al. is not able
to capture the object of interest. The model in [10] can successfully
segment the object, the results of which are listed in the second
row of Fig. 1 . But the method of [10] depends on the initialization
of the LSF. We take another different initialization for the LSF and
obtain a different segmentation result, which is listed in the third
row. We can see the segmentation results are largely dependent on
the initialization.
Segmentation results for our method are presented in the
fourth row. We set β = 2 and h = 20 . We choose three markers
points near the edge. We can observe that all the details far away
rom the edge are filtered out by virtue of the weighting func-
ion. Meanwhile, the image are smoothed selectively and the main
tructures are retained. Therefore, it is easy to obtain the segmen-
ation by simply thresholding the solution u . For this example, seg-
entation is obtained by u > 0.75, which is shown in Fig. 1 (o) and
p). For illustration, we display the convergence curves of the pro-
osed numerical method as shown in Proposition 3 . Since the ex-
ct solution is unknown, we provide the convergence curves of
| u
k +1 − u
k || 2 / || u
k || 2 and || v k +1 − v k || 2 / || v k || 2 in Fig. 2 .
In Fig. 3 , the tumor in the brain has complicated details. Again,
e use same marker points to compare our model with methods
n [5,10] . For our method, we set β = 2 and h = 20 . The tumor is
uccessfully segmented. We provide the CPU time comparison of
igs. 1 and 3 in Table 1 , from which we can see that our algorithm
s faster than the two level set based algorithms.
In Figs. 4 and 5 , we show the segmentation results of two dif-
erent organs in a CT image. In the two tests, β = 2 and h = 15 .
he numbers of marker points are six and eight respectively for the
eft and right organ. The CT image includes many complicated de-
ails, some of which have similar intensity distributions. However,
n the solution image, all details are hid except the target object.
herefore, the targets are highlighted and can be easily extracted
y thresholding.
To test the model for small targets, we provide segmentation
esults for a small vessel in Fig. 6 . We set β = 10 and h = 10 . Two
arker points are chosen. The vessel is easily segmented.
.2. Experiment 2
In this subsection, we provide some experiments and compar-
sons with [23,26] in Fig. 7 . In the first row, we can see that both
23,26] can provide good segmentation results for noisy images.
owever, their methods segment the three objects simultaneously
ince their methods provide a two-phase segmentation. In the sec-
nd and the third rows, we present respectively segmentations of
he triangle and the ring using our method. It is shown in the
ourth row that our model can also segment the two parts simulta-
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 377
Fig. 11. Influence of noise; Blue lines: ground truth; Red lines: our segmentation results. (a) σ= 0.5; SA = 95.13%. (b) σ= 1; SA = 94.80%. (c) σ= 2; SA = 94.01%.
n
a
5
p
m
s
p
t
p
q
b
S
e
o
a
l
i
t
p
W
t
c
t
p
p
F
t
t
a
s
i
e
g
t
b
b
w
p
m
n
σ
f
a
t
w
6
o
e
t
t
t
t
i
i
m
f
A
a
t
t
1
G
f
C
i
q
f
R
eously. In a word, our method can segment any part of the image
ccording to different objectives.
.3. Experiment 3
We emphasize that we should use (1 − f ) instead of the in-
ut normalized image f in model (6) when the target has lower
ean intensity than its neighborhood. The image (1 − f ) has the
ame structure as the original image f . Segmentation results are
resented in Fig. 8 .
Now we investigate the influences of three factors on segmen-
ation results: the number of marker points, the position of marker
oints and different noise levels. To compare segmentation results
uantitatively, we consider the Segmentation Accuracy (SA) defined
y:
A =
correctly classified pixels
all pixels of the region of interest × 100% .
(i) Number of marker points. Firstly, we investigate the influ-
nce of the number of marker points. In Fig. 9 , different quantity
f marker points are used to segment the same ultrasound im-
ge. Ground truth is given by an expert, which is drawn in blue
ines. Our segmentations are drawn in red lines. The correspond-
ng SA values are given. The number of marker points is related
o the size of the object to be segment. Small targets need less
oints while big objects require more points, which is reasonable.
e can see that when the number of marker points is from three
o fifteen, the SA value becomes larger and larger. Meanwhile, we
an observe that the segmentation results are improved little when
he marker points are more than nine. In a word, too less marker
oints is infeasible, while too many is unnecessary.
(ii) Position of marker points. Secondly, we choose ten marker
oints but in different positions to segment the same image as in
ig. 9 . First of all, we emphasize that it is definitely not reasonable
o choose marker points that are far away from the target object. In
his situation, the surrounding objects may be wrongly segmented
s can be seen from Fig. 10 (a). In Fig. 10 (b), segmentation is still
atisfactory even four marker points are outside of the object of
nterest because the marker points are near the boundary. Differ-
ntly, all marker points are inside the boundary in Fig. 10 (c), which
ives similar result as Fig. 10 (b). Thus we conclude that there are
wo rules to choose the marker points for better performance:
(I) If the points are outside the object, they should be near the
oundary.
(II) If the points are inside the object, they can be near the
oundary or in the interior. To get better segmentation results,
e can choose relatively more points near the boundary and less
oints in the interior.
(iii) Noise. Finally, we investigate the influence of noise for our
odel in Fig. 11 . The original image is contaminated by Gaussian
oise with different noise levels. Zero mean noise with variance
is added to the image Fig. 9 (a). Ten marker points are chosen
or different experiments. When σ= 0.5, 1 and 2, our method can
chieve good performances. Besides, we also get good segmenta-
ion results for noisy images in Fig. 7 . Therefore, from Figs. 7 and 11
e can see that our method is robust against noise.
. Conclusion
We proposed a two-stage selective segmentation model based
n the MS model. By selecting several marker points around the
dge of the object, a weight function was constructed to adjust
he smoothing term and the fidelity term. Solving the model by
he ADM produced a smooth solution, which filtered out the de-
ails far away from the edge and preserved the main structure of
he target object. Then segmentation can be obtained by threshold-
ng. Numerical experiments performed on some challenging med-
cal images demonstrated the efficiency and effectiveness of our
ethod. We will apply our method to 3D segmentation in the near
uture.
cknowledgment
The first author is supported by National NSFC (No. 11301129 )
nd Zhejiang Provincial NSFC (No. LQ13A010025 ). The second au-
hor is supported in part by Hong Kong RGC GRF 12302715 . The
hird author is supported in part by Hong Kong RGC 211911 ,
2302714 ; the FRGs of Hong Kong Baptist University and NSFC
rant No. 11271049 . The authors would like to thank Prof. Ke Chen
rom University of Liverpool, Dr. Xiaohao Cai from University of
ambridge and Dr. Haider Ali from University of Peshawar for shar-
ng their Matlab codes. They also would like to thank Prof. Xiao-
un Zhang and Dr. Jiulong Liu from Shanghai Jiaotong University
or providing some experimental data.
eferences
[1] M. Kass , A. Witkin , D. Terzopoulos , Snakes: active contour models, Int. J. Com-
put. Vis. 1 (1988) 321–331 . [2] V. Caselles , R. Kimmel , G. Sapiro , Geodesic active contours, Int. J. Comput. Vis.
22 (1997) 61–79 . [3] Y. Xiang , A.C.S. Chung , J. Ye , An active contour model for image segmentation
based on elastic interaction, J. Comput. Phys. 219 (2006) 455–476 .
378 C. Liu et al. / Pattern Recognition 76 (2018) 367–379
[
[4] D. Mumford , J. Shah , Optimal approximations by piecewise smooth functionsand associated variational problems, Commun. Pure Appl. Math. 42 (1989)
577–685 . [5] C. Gout , C.L. Guyader , L. Vese , Segmentation under geometrical conditions with
geodesic active contour and interpolation using level set methods, Numer. Al-gorithms 39 (2005) 155–173 .
[6] N. Badshah , K. Chen , Image selctive segmentation under geometry constrainesusing an active contour approach, Commun. Comput. Phys. 7 (2009) 759–778 .
[7] J. Zhang , K. Chen , B. Yu , D.A. Gould , A local information based variational
model for selective image segmentation, Inverse Prob. Imaging 8 (2014)293–320 .
[8] L. Rada , K. Chen , A new variational model with dual level set functions forselective segmentation, Commun. Comput. Phys. 12 (2012) 261–283 .
[9] J. Peng , F. Dong , Y. Chen , D. Kong , A region-appearance-based adaptive varia-tional model for 3d liver segmentation, Med. Phys. 41 (4) (2014) . 043502.
[10] L. Mabood , H. Ali , N. Badshah , K. Chen , G.A. Khan , Active contours textural and
inhomogeneous object extraction, Pattern Recognit. 55 (2016) 87–99 . [11] S. Osher , J.A. Sethian , Fronts propagating with curvature-dependent speed: al-
gorithms based on Hamilton-Jacobi formulations, J. Comput. Phys. 79 (1988)12–49 .
[12] T.F. Chan , L.A. Vese , Active contour without edges, IEEE Trans. Image Process.10 (2001) 266–277 .
[13] L.A. Vese , T.F. Chan , A multiphase level set framework for image segmetation
using the Mumford and Shah model, Int. J. Comput. Vis. 50 (2002) 271–293 . [14] J. Lie , M. Lysaker , X.-C. Tai , A variant of the level set method and applications
to image segmentation, Math. Comput. 75 (2006) 1155–1174 . [15] S. Zhu , Q. Wu , C. Liu , Shape and topology optimization for elliptic boundary
value problems using a piecewise constant level set method, Appl. Numer.Math 61 (2011) 752–767 .
[16] S. Zhu , C. Liu , Q. Wu , Binary level set methods for topology and shape op-
timization of a two-density inhomogeneous drum, Comput. Methods Appl.Mech. Eng. 199 (2010) 2970–2986 .
[17] F. Li , M. Ng , T. Zeng , C. Shen , A multiphase image segmentation method basedon fuzzy region competition, SIAM J. Imaging Sci. 3 (2010) 277–299 .
[18] B. Mory , R. Ardon , Fuzzy region competition: a convex two-phase segmenta-tion framework, in: Scale Space and Variational Methods in Computer Vision,
2007, pp. 214–226 .
[19] T.F. Chan , S. Esedoglu , M. Nikolova , Algorithms for finding global minimizersof image segmentation and denoising models, SIAM J. Appl. Math. 66 (2006)
1632–1648 . [20] T. Pock , A. Chambolle , D. Cremers , H. Bischof , A convex relaxation approach
for computing minimal partitions, in: Processings of the IEEE Computer So-ciety Conference on Computer Vision and Pattern Recognition (CVPR), 2009,
pp. 810–817 .
[21] L. Grady , C. Alvino , Reformulating and optimizing the Mumford–Shah func-tional on a graph- a faster, lower energy solution, in: Proceedings of the Euro-
pean Conference on Computer Vision (ECCV), 2008, pp. 248–261 . [22] J. Yuan , E. Bae , X.-C. Tai , Y. Boykov , A continuous max-flow approach to Potts
model, in: ECCV 2010: Proceedings of the 11th European Conference on Com-puter Vision, Springer, Berlin, 2010 . 379-39.
[23] X. Cai , R. Chan , T. Zeng , A two-stage image segmentation method using a con-vex variant of the Mumford-Shah model and thresholding, SIAM J. Imaging Sci.
6 (2013) 368–390 .
[24] T. Goldstein , S. Osher , The split Bregman method for l1-regularized problems,SIAM J. Imaging Sci. 2 (2009) 323–343 .
[25] X. Zhang , M. Burger , S. Osher , A unified primal-dual algorithm frameworkbased on Bregman iteration, J. Sci. Comput. 46 (2011) 20–46 .
[26] X. Cai , Variational image segmentation model coupled with image restorationachievements, Pattern Recognit. 48 (2015) 2029–2042 .
[27] S.P. Awate , R.T. Whitaker , Multiatlas segmentation as nonparametric regression,
IEEE Trans. Med. Imaging 33 (2014) 1803–1817 .
[28] E.B. Dam , M. Lillhol , J. Marques , M. Nielsen , Automatic segmentation of high--and low-field knee MRIs using knee image quantification with data from the
osteoarthritis initiative, J. Med. Imaging 2 (2015) . 024001. [29] S.P. Awate , T. Tasdizen , R.T. Whitaker , Unsupervised texture segmentation
with nonparametric neighborhood statistics, Eur. Conf. Comput. Vis. (2006)494–507 .
[30] J. Kim , A. Yezzi , M. Cetin , A.S. Willsky , A nonparametric statistical method forimage segmentation using information theory and curve evolution, IEEE Trans.
Image Process. 14 (2005) 1486–1502 .
[31] J. Liu , X. Zhang , B. Dong , Z. Shen , L. Gu , A wavelet frame method withshape prior for ultrasound video segmentation, SIAM J. Imaging Sci. 9 (2015)
495–519 . [32] Z. Hui , H. Trevor , Regularization and variable selection via elastic net, J. R. Stat.
Soc. B 67 (2005) 301–320 . [33] A. Chambolle , Image segmentation by variational methods: Mumford and Shah
functional and the discrete approximations, SIAM J. Appl. Math. 55 (1995)
827–863 . [34] C. Liu , F. Dong , S. Zhu , D. Kong , K. Liu , New variational formulations for level
set evolution without reinitialization with applications to image segmentation,J. Math. Imaging Vis. 41 (2011) 194–209 .
[35] S. Esedoglu , Y. Tsai , Threshold dynamics for the piecewise constant Mum-ford-Shah functional, J. Comput. Phys. 211 (2006) 367–384 .
[36] Y. Zhao , L. Rada , K. Chen , S.P. Harding , Y. Zheng , Automated vessel segmenta-
tion using infinite perimeter active contour model with hybrid region infor-mation with application to retinal images, IEEE Trans. Med. Imaging 34 (2015)
1797–1807 . [37] F.A. Limberger , M.M. Oliveira , Real-time detection of planar regions in unorga-
nized point clouds, Pattern Recognit. 48 (2015) 2043–2053 . [38] H. Min , W. Jia , X.F. Wang , Y. Zhao , R.X. Hu , Y.T. Luo , F. Xue , J.T. Lu , An in-
tensity-texture model based level set method for image segmentation, Pattern
Recognit. 48 (2015) 1547–1562 . [39] H.K. Ji , Q.S. Sun , Z.X. Ji , Y.H. Yuan , G.Q. Zhang , Collaborative probabilistic labels
for face recognition from single sample per person, Pattern Recognit. 62 (2016)125–134 .
[40] X. Cai , Variational image segmentation model coupled with image restorationachievements, Pattern Recognit. 48 (2015) 2029–2042 .
[41] J. Hartigan , M. Wang , A k-means clustering algorithm: algorithm AS 136, Appl.
Stat. 28 (1979) 100–108 . [42] F. Lindsten , H. Ohlsson , L. Ljung , Just relax and come clustering! a convexifica-
tion of k-means clustering, Technical Report 2992, 2011 . [43] T. Nguyen , J. Cai , J. Zhang , J. Zheng , Robust interactive image segmentation us-
ing convex active contours, IEEE Trans. Image Process. 21 (2012) 3734–3743 . 44] I. Ekeland , R. Temam , Convex analysis and variational problems, Classics Appl.
Math. 28 (1999) . SIAM.
[45] G. David , Sigular sets of minimizers for the Mumford-Shah functional, Pro-gressin Mathematics 233, Birkhäuser, 2005 .
[46] D. Han , X. Yuan , W. Zhang , An augmented Lagrangian based parallel splittingmethod for separable convex minimization with applications to image process-
ing, Math. Comput. 83 (2014) 2263–2291 . [47] Y. Zhang , B. Dong , Z. Lu , l 0 minimization for wavelet frame based image
restoration, Math. Comput. 82 (2013) 995–1015 . [48] Y. Huang , D. Lu , T. Zeng , Two-step approach for the restoration of images cor-
rupted by multiplicative noise, SIAM J. Sci. Comput. 35 (2013) 2856–2873 .
[49] Y. Huang , M. Ng , Y. Wen , A new total variation method for multiplicative noiseremoval, SIAM J. Imaging Sci. 2 (2009) 20–40 .
[50] P. Getreuer , Rudin-osher-fatemi total variation denoising using split Bregman,Imaging Process. on Line 2 (2012) 74–95 .
[51] M. Nikolova , R. Chan , The equivalence of half-quadratic minimization andthe gradient linearization iteration, IEEE Trans. Image Process. 16 (2007)
1623–1627 .
[52] B. Dong , A. Chien , Z. Shen , Frame based segmentation for medical images,Commun. Math. Sci. 32 (2010) 1724–1739 .
C. Liu et al. / Pattern Recognition 76 (2018) 367–379 379
C , in 2006, and the Ph.D. degree in mathematics from Zhejiang University, China, in 2011. S . Her research interests include image processing, the variational methods, and scientific
c
M the University of Hong Kong, and the Ph.D. degree in 1995 from the Chinese University o Kong Baptist University. His research interests include bioinformatics, image processing,
s
T M.S. degree from Ecole Polytechnique, Palaiseau, France, and the Ph.D. degree from the U ining Hong Kong Baptist University as an Assistant Professor, he worked as a postdoctoral
r sing, statistical learning, and scientific computing.
hunxiao Liu received the B.S. degree in mathematics from Linyi University, Chinahe is a lecturer in department of mathematics at the Hangzhou Normal University
omputing.
ichael K. Ng received the B.Sc. degree in 1990 and the M.Phil. degree in 1992 fromf Hong Kong. He is a Professor in the Department of Mathematics at the Hong
cientific computing and data mining.
ieyong Zeng received the B.S. degree from Peking University, Beijing, China, the niversity of Paris XIII, Paris, France, in 20 0 0, 20 04, and 20 07, respectively. Before jo
esearcher in CMLA, ENS de Cachan, France. His research interests are image proces