Shape Deformation: SVM Regression and Application to Medical ImageSegmentation

SongWang,Weiyu ZhuandZhi-PeiLiangDepartmentof ECEandBeckmanInstitute

Universityof Illinois at Urbana-Champaign,IL 61801,U.S.A.songwang@uiuc.edu, weiyuzhu@uiuc.edu, z-liang@uiuc.edu

Abstract

This paperpresentsa novel landmark-basedshapede-formationmethod.Thismethodeffectivelysolvestwo prob-lemsinherent in landmark-basedshapedeformation: (a)identificationof landmarkpointsfroma giveninput image,and (b) regularizeddeformationof the shapeof an objectdefinedin a template. Thesecondproblemis solvedusinganew constrainedsupportvectormachine(SVM)regressiontechnique, in which a thin-platekernelis utilizedto providenon-rigid shapedeformations.This methodoffers severaladvantagesover existing landmark-basedmethods.First,it hasa uniquecapability to detectand usemultiple can-didatelandmarkpointsin an input image to improveland-markdetection.Second,it can handlethe caseof missinglandmarks,which oftenarisesin dealingwith occludedim-ages. We haveappliedthe proposedmethodto extract thescalpcontours frombrain cryosectionimageswith veryen-couragingresults.

1 Introduction

Imagesegmentationis an importanttask in imageun-derstandingandcomputervision. Traditionalmethodsareusuallybasedon edgedetection,region merging/splitting,statisticalpixel classificationor hybrid methods[19]. Al-thoughthesemethodsarecapableof partitioningan imageinto regionsof connectedpixelsaccordingto imageinten-sity distributions, more often than not, the segmentedre-gionsdonotcorrespondwell to trueobjects.Thislimitationis particularlyseriousfor medicalimagesegmentationbe-causemedicalimagesareoftenverynoisyandthestructuresto beidentifiedhave large“internal” intensityvariations.

An effective way to solve this problemis to incorporatethe known shapeinformation of the desiredobjectsintothe segmentationprocess.Generally, the shapeof an ob-ject is describedby theboundarycontour(or severalclosedcontours)of theobjects,which canberepresentedby a se-quenceof sampledlandmarkpoints.

A numberof shape-basedsegmentationmethodshave

beenproposed.For example,theactivecontour(or snakes)[8, 1, 17, 4] modelconstrainsthe desiredshapeto be suf-ficiently continuous,smoothanddifferentiableduring thedeformationprocess.Although the original active contourcannotprocessmultiplecontours,theproblemcanbesolvedusingtherecentgeodesiccontourmethod[3].

More advancedshapemodelsinclude the point distri-bution model (PDM) by Cooteset al.[5], which can learnshapevariationsfrom a setof segmentedtraining images,eachcontaininga set of landmarksto define the shape.PDM calculatesthe covariancematrix of theselandmarksandthenrepresentstheshapevariationsalongthedirectionsof the mostsignificanteigenvectorsof the covariancema-trix. However, it is a tediousjob to build the training setmanuallyandin somecasesit is alsodifficult to identify thesetof landmarkpointsfor all theshapesof interest.

Similarly to PDM, Staib et al. [13] decomposetheshapeinto itemswith differentfrequencies.Thentheshapeknowledgeis learnedby studyingthe distribution of eachFouriercoefficient. Theresultis usedto constraintheshapedeformation.Jain[7] presentsasimilarapproachwhichpa-rameterizesthe shapeandusesthe distribution of the pa-rametersastheconstraintfor shapedeformation.Along thesameline, Leventonet al. [10] incorporatestatisticalshapeinformationinto geodesicactivecontoursto segmentmedi-cal images.

Thework presentedin thepaperis morecloselyrelatedto thedeformablemodelsdevelopedby Rueckertetal. [12]andZhong et al. [18]. Similar to the active contour, theenergy function of thosemodelsconsistsof two termsac-countingfor externalandinternalenergy respectively. Theexternalenergy is a potentialfield describingtheedgeandregion matchinginformationbetweenthe templateandtheinput image. The internalenergy representsthe shapede-viation from theprior. However, it is not easyto determinetheappropriateweight for the internalandexternalenergy.Additionally, this energy functionis nonlinearandincludestoo many unknowns to be well handledby gradient-basedalgorithms.

This paperpresentsa new landmark-basedshapedefor-mationmethodwhich caneffectively avoid theaboveques-

tions. At first, edge-or/andregion-basedmethodsareap-pliedto detectcandidatelandmarkpointsin theinput imagewithin the neighboringareaaroundeachlandmarkin thetemplate.Multiple candidatesmaybedetectedfor oneland-markwhenit is necessary. Thenthe templateis deformedusinga specialsupportvectormachine(SVM) regressiontechnique.Basedon the fitting result,a setof betterland-marksis identifiedfrom thedetectedcandidates.This pro-cessis repeateduntil thefinal resultis obtained.Usually, acoupleof iterationsis sufficient.

The rest of the paperis organizedas follows. In Sec-tion 2, we describethe proposedshape-baseddeformationmethod.Section3 discussestheselectionof theregulariza-tion parameterandhow to handlemissinglandmarks.Sec-tion 4 presentsrepresentativeresultsfor medicalimageseg-mentation,followedby theconclusionin Section5.

2 The Proposed Method

2.1 Problem formulation

For simplicity, considerthe casein which the templatecontainsonly a single closedcontour. Extensionto thecaseof multiple contoursis straightforward. The con-tour is representedasa seriesof orderedlandmarkpoints����������� �������� �� ���

, where� ����������������

arethecoordi-natesof the -th landmark.For eachlandmark

� �, thepro-

posedmethodfirst identifiesasetof possiblecorrespondinglandmarkpoints ! � �"���$#&%�'� )(*�,+-/.0��������1 � �

in the in-

put image,where� #&%�'� �2��� #&%�'� �� #&%�'� �

. Thentheproblemissolvedin two steps:

1. Identify the best landmarkpoint��3� �4�5��3� ��63� �

fromthelandmarkset ! � suchthat

�738�9���83� ��83� ������:��83� �locatedin or neartherealobjectboundaryin theinputimage;

2. Deformtheprior shape�

to match� 3

while keepingthegeneralshapecharacteristicsof

�.

The templatedeformationis describedas a regressionproblemof finding a transform; �<�>=��?0�A@�B �7C B �

thatminimizes +1 �D �FE��8G �5� 3� ; ��� � ���8HJI�KML ;ON (1)

where G �QPF�P&�ARTS is a lossfunction,IURVS

is a regulariza-tion parameterand

KML ;�N R9S is a regularizationfunctional.In this setting, ; ���W�X�<� ; �5� � �Y �2+�Z.0������[�1\� is thede-siredobjectshapein theinput image.

Generally, it is acombinatorialproblemto determinethebest

�73. In thispaper, weuseanEM-like(ExpectationMin-

imization)two-stepalgorithmto solve it iteratively. Specif-ically, we performtheshape-regularizedfitting to aninitial�73

to get an optimal ; and thenupdate�73

from detected

! � basedon the acquired; . This processis repeateduntilconvergence.

Therearethreemainsubproblemsin this approach:de-tectionof landmarkcandidates! � in the input image,de-terminationof theregressionmodelandupdating

� 3based

ontheregularizedfitting results.Solutionto theseproblemsarediscussedbelow.

2.2 Detection of landmark candidates

Given�

, we assumethat�83�

in aninput imagefalls in acircular areaof radius ] � centeredat

� �. Accordingto the

principle of anisotropicdiffusions[11], non-rigid biologicshapedeformationbetweenthe templateandthe input im-agecanbe doneeffectively by moving eachlandmark

� �along the normaldirectionof the shape. This meansthatwecansearchfor thecorresponding

�73only alongthenor-

mal directionsof�

asshown in Fig.1. Denote ^ � as the

2

n1

v

n2v2

vn

nn

n

v1

v1

v

Figure 1. Search for landmarks along the nor -mal direction of the initial shape .

unit vector along the normal direction at� �

. Combiningtheaboveassumptions,thecorrespondinglandmark

��3�can

be obtainedfrom the line segment _ � �`��� � H ]a^ � ]cbLed ] � ] � N � .Becauseof noiseandstructuralcomplexity, sometimes

it is difficult to accuratelyidentify�83�

from _ � . In this

case,we canextractseveralcandidatesin ! � �f��� #&%�'� �(W�+�Z.0������[�1 � �andtheproposedalgorithmwill determinean

optimal�83�

from ! � .For differentapplicationproblems,variousmethodscan

be usedto locate ! �� �g+�Z.0������:�1. Generally, they can

becategorizedinto two classes.

1. Edgedetectionmethods.Sincethedesiredshapeis thecontourof an object in the input image,edgepointsin _ � will be collectedto construct! � . Any edgede-tectionalgorithmcanbeappliedhere. A simplestap-proachis to calculatethegradientvectorfor eachpixelalong _ � . If its amplitudeis very large(accordingto apre-selectedthreshold),it will beincludedin ! � .

2. Region matchingmethods. For any pixel along _ � ,if its neighboringareahas similar featureswith the

neighboringareacenteredat� �

in thetemplateimage,it will be includedin ! � . A numberof matchingmet-rics canbe usedfor this problem,which includenor-malizedcross-correlation,squareddifferencesin pixelintensities,measuresbasedonopticalflow modelsandmutualinformationmetrics[14].

Generally, region matchingis oftenmorerobust thanedgedetectionto processnoisy imagescontainingcomplex ob-jects. However, region matchingmethodsusually requirethetemplatelandmarks

�to beveryaccurate,which is not

necessaryfor edgedetectionmethods.Selectionof ] � is relatedto the similarity betweenthe

templateandthe input image.For segmenting3D medicalimagessliceby slice,this is usuallydeterminedby thesam-pling interval betweentwo neighboringslices. The largertheinterval, thelargerthe ] � . In practice,] � shouldalsobeadaptively modifiedbasedon thesizeof ! � . If therearetoomany candidatesin ! � , then] � needsto bereduced,andviceversa.

It is alsopossiblethatnolandmarkcandidateis identifiedfor some

� �. Thisproblemwill bediscussedin Section3.2.

2.3 SVM regression model for deformation

Given a set of estimatedlandmarks�73

from� ! �Q �+�/.h������[�1\�

, this sectiondescribesa regressionmodel for(1) andthecorrespondingalgorithm. Generally, we expectthe regressionmodel to have the following properties:(a); �5�i� is a goodapproximationfor

� 3; (b) ; ��P � is a geomet-

rically homologousmapping(an importantfeaturein bio-logic deformation);(c)

K�L ;ON representswell thebiologicde-formation between

�and ; �5�i� ; (d)

KML ;�N is invariantun-dertheaffine or rigid transforms;and(e) ThelossfunctionG �QPF�P&� is robustagainstnoiseandoutliers;

Fromregularizationtheory,KML ;�N canbedefinedasanorm

in a reproducingkernelHilbert space(or subspace)whichcanbe uniquelyrepresentedby a positive definite(or con-ditionally positive definite)kernel function j �5�k��83l� . Formedicalimagesegmentation,we proposeto usethe thin-platekernel[2] j �5�k��83l�m�onY�pdq�83rn �[sFt�u nO�vdi�83rn , whereK�L ;ON , alsonamedthebendingenergy, is givenbyKML ;�N �Vw$wyxz x �>{|��=:� H*{}�l?h���Q~-��~-� (2)

where{}�QP&�m���k������ � � � HJ.h����������� � � H��k������ � � � .

A popularlossfunctionis G �5�83� ; �5� �)���k�9nY�83� d ; �5� �)��n � .In this case,the model becomesthe traditional thin-platesplines[2]. However, it is well known that thesquareder-ror loss function is not robust againstoutliers. To avoidthis problem,variousrobust loss functionshave beende-veloped. Two examplesare the modulo function and theHuber’sfunction[6]. Hereweadoptthelinear � -insensitivefunctionusedin SVM [15] for thefollowing reasons:(a) Itis morerobustthanthesquarederrorlossfunction;(b) With

theinsensitive margin parameter� , landmarkpointscanbeidentifiedassupportvectorsor non-supportvectors. Thisresultcanbeusedto improve

�W3. This stepis discussedin

detail in Section2.4.The � -insensitive functionis definedbynO� 3� d ; �5� �)��n��� � S if

nO�83� d ; �5� �r��nA� �nY��3� d ; �5� �r��n�d � else�

When � �cS , this is just themodulolossfunction.Theonly remainingcomponentof the regressionmodel

is theadditionalconstraintsthatshouldbeplacedonthede-sireddeformation; �QP&� . Obviously, we hopethatthedesired; ��� ��� lie in thenormaldirectionof theshapecenteredat

� �,

i.e., ; �5� � �m��� � H�� � ^ � ��+�Z.0������[�1 (3)

where�0� b B .

Since�W3

is detectedalongthe normaldirectionsof�

,wehave � 3� ��� ��HJ��� ^ �Q ��+-/.0������:�1 (4)

where� � b B� ��+�/.h������[�1

are known. Noting that; ����=�Q?h�and

n ^ �/nv��+- �"+�Z.0������:�1, the regression

problemcanberewrittenasProblem 1:������/� ��� � ; ��$��� +1 �D �FE���� � � d�� � � � HJI�K�L ;ON

subjectto constraints(3), where���9�l���Q�0�������� Q�0�h�Q�

.The thin-plate kernel is conditionally positive definite

and the linear functionsform the null spaceof the result-ing ; �9�>=��?0� , which mustbeof theform [2]� =\�����k���0�kH*�6�O��H*�6 O�XHJ¡ ��FE��[¢ � j ���m�� �)�?:�5�M�k��£��$Hy£Y���¤HJ£Y O�XH ¡ ���E�� ~-� j ���m�� �)�Y� (5)

To solveProblem+, we first fix

�andminimize � � ; ��m�

with respectto ; underconstraints(3). This is equivalenttoseeking; minimizing ¥�¥ xz x ��{}�>=:�\H�{|�5?0���Q~-��~-� subjecttotheconstraintthat ; mustmap

� �to��¦���/6¦���>�m��� �)H|�0� ^ � , �+�Z.0������[�1

. It is a typical thin-plateinterpolationproblemandthe parameters§ �`���h�a/�6����6 ��Q�

, ¨ �`�>£��a/£Y�aZ£Y ����,© �ª� ¢ � ¢ �������� ¢ �h� � and « �ª��~0�/~-�-������:/~-�h� �

in (5)canbecalculatedby solvingthefollowing equation:¬®­ ¯¯ � °<± ¬ © «§ ¨ ± � ¬<²³ ²´° °�± (6)

where µ � % � j �5� � �� % � , �(¶�·+�/.h������[�1are the ele-

mentsof matrix­

and¯ �·�Q+� ³ ´ � . Note that ³ ��5�:�a����������� ����h�Q�

, ´ �g���¸������a������ ����h���,� �X�g����������)�

,²³ �9��¦�:�a6¦���-������:¸¦���h�Q� and²´ �9��¦�¸�a6¦����������:6¦���h��� .

In theabove thin-plateinterpolation,theresultingmini-malbendingenergy is givenbyKML ;ON � © � ­ © H « � ­ « � +¹�º � ²³ �M» ²³ H ²´ ��» ²´ �O (7)

where»

is the1�¼71

upperleft sub-matrixof¬ ­ ¯¯ � °<± z � (8)

which is positivesemi-definite.Substituting(7) into Problem

+yields thefollowing un-

constrainedproblem.Problem 2:������ � ���k�m� +1 �D �FE�� � ���:d��6� � � H I¹�º � ²³ � » ²³ H ²´ � » ²´ �Using the standardSVM technique,we can formulate

theproblemasfollows.Problem 3:���F��[� ½��F¾½ � �����¿ ¦¿��·� +1 �D ��E�� ��À � H ¦À � �H I¹�ºÂÁ � ³ Hyà � �m� �M» � ³ H*à � �$�H¤� ´ Hyà � �$� � » � ´ Hyà � �k��Ä

subjectto � � dq� � � � H ¦À � ��+�Z.0������[�1� � d�� � � � H*À � ��+�Z.0������[�1À � ¦À � RÅS ��+�Z.0������[�1where

¿4� �5À����À��������� �À��h� �,¦¿4� � ¦À�a ¦À���������8 ¦À��h� �

,à � � diag��Ƥ� � /ÆÇ� � ������:/ÆÇ� � � and

à � � diag�>Ƥ� � ÆÇ� � ������ �ÆÂ� � � with ^ �8���>ÆÇ� � �ÆÂ� � � .

Introducethe Lagrangemultipliers È �4�>É\���ÉM��������:É��h� �,¦È �Ê��¦É$�a�¦É��������� �¦É��h� �

, Ë �Ì�5Í:��Í���������8�Í��h� �and

¦Ë �®� ¦Í:� ¦Í���������: ¦Í��h���, the Lagrangianfunction for

ProblemÎ isÏÐ���ÑQ¿: ¦¿ È ¦È Ë ¦Ë �m�Ò �D ��E�� ��À���H ¦À��)�8H +. Á � ³ Hyà � �m� � » � ³ Hyà � �m�H¤� ´ H*à � �m� � » � ´ Hyà � �k� Äd �D �FE�� É��/� � H*À��:d��6��HJ�����d �D �FE�� ¦É��/� � H ¦À���HU�6�:dU�����Md �D �FE�� �5Í���À��[H ¦Í�� ¦À��)�

whereÒ ��ÓZÔ��Õ .

Imposingthefirst-ordernecessaryconditions(settingthederivativesof theLagrangian

Ïwith respectto

�,¿

and¦¿

to zero)yields � � ÖU� ¦È d È �MdØ× (9)

and É � HUÍ � � Ò ��+�Z.0������[�1¦É � H ¦Í � � Ò ��+�Z.0������[�1É � �¦É � �Í � ¦Í � RJS ��+�Z.0������[�1where Ö � ��à �� » à � Hyà �� » à � � z �× � ÖU��à �� » ³ H*à �� » ´ �O�

Substitutingtheseresultsinto Problem Î givesthe fol-lowing equivalentformulation

Problem 4:�����Ù � ¾Ù ¦ÏÐ� È ¦È �k� +. � È d ¦È � � ÖU� È d ¦È �H¤�5×WH*Ú\� � � È d ¦È �8HyÛ � � È H ¦È � (10)

subjectto S��JÉ � �¦É � � Ò �Ü+-/.h������[�1where

����� � /� � ������:Z� � ���and

Û¤�Ü� � � ������: � ��� .Problem Ý is a simplequadraticprogrammingproblem

which canbe solved usingstandardalgorithms. After theoptimal È ¦È areobtained,thedesired

�in ProblemÎ can

becalculatedfrom (9). Finally, we apply(6) and(5) to getthedesired; .2.4 Updating

�73Thenext stepin our iterativealgorithmis to improvethe

estimationof�W3

from ! �Q �Þ+�/.h������[�1 basedon there-sults from the shapefitting step. To do so, we invoke theKuhn-Tuckerconditionfor ProblemÝ , which statesÉ � �5� � d�� � d � dØÀ � �m��S ��+�Z.0������:�1¦É � �5� � d�� � H � H ¦À � �m��S ��+�Z.0������:�1� Ò d�É��)�rÀ��M��S ��+�Z.0������:�1� Ò d¶¦É��)� ¦À��M��S ��+�Z.0������[�1$�It is obvious that

É � Pm¦É � �2Sh �"+-/.h������[�1. The input

data� �

for whichÉ �

or¦É �

have nonzerovaluesarecalledsupportvectors. Otherwise,

� �is a non-supportvector.

As shown in Fig.2, if� �

is a non-supportvector, thenthe current

�83�lies insidethe � -insensitive margin centered

at the resulting ; �5�i� , i.e., � � � dV� � � � � . From (10), italso easyto seethat

�83�will have no effects on the final

regressionandhasnocontributionto thebendingenergy. In

ε-insensitive margint( )

t( )

v

j

i

v

v

jv

j

i

invi jn

v

Figure 2. Suppor t vector� % and non-suppor t

vector� �

.

this case,the current��3�

canbe regardedasbeingaccurateenoughwith the permissibleerror margin � . On the otherhand,any supportvector lying outsidethe margin needtobe improved if possible. Basedon this consideration,thefollowing simpleexchangingstrategy is adoptedto update� 3

.

1. For non-supportvector� �

, we leave thecurrent��3�

un-changed;

2. For supportvector� �

, if anothercandidatein ! � liesin the � -insensitivemargin, thenweadoptit asthenew�83�

.

Thisstrategy will reducesthebendingenergy if � is fixed.Theremainingproblemis theselectionof � . Inspiredby

thearea-shrinkingtechniqueusedin SOM (self-organizingmapping),wereduce� graduallyin theiterationprocessun-til it approacheszero.This is quitereasonablesincetheac-quired ; �5�i� is usually inaccurateat the beginning of theiterationfor somemis-selected

� 3�. Thusa large � may in-

creasethechanceof includingthetrue�W3

insidetheinsen-sitive margin while eliminatingbadoutliers. As ; �5�i� be-comesmoreandmoreaccurate,wecanreduce� to improvetheresolutionof thefinal results,which is similar to whatisdonein multiscaleimageanalysis[9].

2.5 Summary of the algorithm

Theproposedalgorithmis summarizedbelow.

1. Basedonthetemplateshape�

, detectalandmarkcan-didateset ! � �o+�Z.0������[�1 in theinput imagealongthenormaldirectionof

�.

2. Randomlyselect� 3

from ! � �Ð+-/.h������[�1 andini-tialize � ;

3. Perform SVM regressionusing a thin-plate kernel.Find the deformedshape; �5�i� and label the supportvectorsandnon-supportvectors;

4. Update�W3

by replacingthesupportvectorswith bettercandidatesin ! �Q ��+-/.0������:�1 . If nooneis updated,reduce� . Returnto step Π. If no oneis updatedand

� �2S , stopthe algorithmandreturncurrent ; �5�i� asthedesiredshape.

3 Discussion

This sectiondiscussestwo relatedissuesin practicalap-plication of the proposedalgorithm: (a) how to selecttheregularizationparameter

I(orÒ

which boundsthe È and¦È ), and (b) how to deal with the casewhen some ! � isempty.

3.1 Selection ofI

TheregularizationparameterI

playsanimportantroleinthedeformation.Theoptimal

Iis chosenfor our problem

by minimizing the ordinary cross-validation (OCV) func-tion ß�à ��I��k� +1 �D �FE�� nO� 3� d ;�á â/ãÕ �5� �)��n��where;�á âZãÕ is theminimizerof Problem

+exceptthatthe µ th

landmarkpoint is left out, i.e., ;�á â/ãÕ minimizes+1 �D��E�� � �räE â nY� 3� d ; �5� � ��n � HJI�K�L ;ON (11)

subjectto constraints(3). This is obtainedusingtheprinci-ple of “leave-out-one”cross-valuesfor crossvalidation. Inthiscase,the

�83â is alsosetasanunknown which is allowedto vary freely alongthenormaldirection ^ � . Similar to theanalysisin Section2.3,solutionto this problemcanbeob-tainedby solvingthequadraticprogrammingin ProblemÝtogetherwith two additionalconstraintsÉ â �¦É â ��S��In this case,

� â arefixedto bea non-supportvectorand� â(so

��3â ) will not affect theresults.Basedon theacquiredÈand

¦È , we canapply(9), (6),and(5) to get�

and ; .Wahba[16] hasconducteda thoroughstudyon how to

estimateI

for the scalar function approximationsplinesandalsoextendedtheOCV to generalizedcrossvalidation(GCV). The main principle can also be adaptedto selectoptimal

Ifor ourSVM-basedshapedeformation.

3.2 No landmark candidate

It is possiblethatno landmarkcandidatescanbe identi-fied for some

� �. Assumeå �g� � +q� �<1$ ! �X�çæ6�

.In this case,we canonly usethe remaininglandmarksde-tected.Accordingto theresultingfunction ; , theundetectedlandmarkscanbedirectlycalculatedfrom ; �5� �r� . Theprob-lemcanbeformulatedasminimizing+1 �D��E�� � �räè�é nY� 3� d ; ��� � ��n � HJI�K�L ;ON

subjectto constraints(3).This can be regardedas the “leave-several-out” test

which is a generalizationof (11)andhasasimilar solution.At first, we solveProblemÝ with additionalconstraintsÉ % �¦É % ��ShêÊë0( bpå whichdisable

� % �( b7å . ThenbasedontheacquiredÈ and¦È , we canuse(9), (6) and(5) to gettheoptimal�

and ; .This propertyis usefulfor two reasons:(a) We canpur-

poselyexclude“bad” landmarkcandidatesets! � in thefit-ting step. (b) When the object to be segmentedis partlyoccluded,this methodcancombinetheobjectshapein thetemplateto geta reasonableandaccuratesegmentation.

However, it should be noted that the numberof non-empty ! � ’smustbesufficient largeto capturetheshapede-formationbetweenthetemplateandtheinput image.Deter-mining the minimum numberof landmarkpointsrequiredfor a givenshapehasnot beensolved.

4 Experiments

We have usedthe proposedmethodto extract the scalpcontourfrom a seriesof braincryosectionimagesfrom theVisual HumanProject. Fig.3(a)shows the initial templateslice with the manuallyextractedscalpcontourcomposedof+�S�S

landmarkpointsdistributeduniformly.

(d)(c)

correct one(a) (b)

Figure 3. (a) Initial template image with themanuall y extracted scalp contour; (b) The in-put image with the unpr ocessed scalp con-tour from the template overlaid; (c) Detected! � based on one special

� �; (d) Detailed il-

lustration sho ws there are Î landmark candi-dates in ! � .We usedthe edgedetectionmethodto extract the land-

mark candidates! �� �ì+�/.h������[�1. Noting that the in-

tensity of pixels just outsidethe scalpcontour is smaller

thanthat of thosejust insidethe contour, we extracted ! �by searchinga rising-edge(from outsideto inside) along^ � near

� �. An exampleis illustratedin Figs.3(c)and(d)

whereÎ landmarkcandidatesaredetectedcorrespondingtoone

� �.

Our first experimentis to segmentan image(shown inFig.3(b))

+�Sslicesaway from the template.Theshapede-

formationprocessis shown in Fig.4. At thebeginning(leftcolumnof Fig.4), some

��3�aremis-selected.However, the

regularizedshapecanmatchthedesiredcontourmuchbet-teralthoughsomeerrorstill existsin somelocalareas.Afterupdating

�W3�correspondingto the supportvectors,we can

seethatmany outliersareremoved.The secondexperimentis to verify the effectivenessof

theproposedalgorithmwhenpartof theinput imageis oc-cluded. Fig.5(a)is a simulated“occluded” imagewith thetemplateshapeoverlapped.Usingtheproposedmethodto-getherwith the additionalconstraintsdiscussedin Section3.2,quiteaccuratesegmentationwasacquired,asshown inFig.5(b). In this experiment,only

.-íout of

+�S-Slandmarks

areallowedto slidefreely alongtheir normaldirections.Finally, we usedthe proposeddeformationmethodto

segmentthe cryosectionbrain imagesslice by slice. Be-ginning from the manual segmentedtemplateshown inFig.3(a), the shapein the templateis deformedto matchits neighbors.Thenthesegmentedslicesaretreatedasnewtemplatesto processtheirunsegmentedneighbors.Thefirstand last imagesin Fig.6 are Î + slicesaway from the ini-tial templateimage. Only the sliceswith oddnumbersareshown here.

5 Conclusion

A novel landmark-basedshapedeformationmethodhasbeendescribedin thispaper. Thismethodprovideseffectivesolutionto two problemsinherentin landmark-basedshapedeformation: (a) identificationof landmarkpoints from agiven input image,and(b) regularizeddeformationof ob-ject shapeembeddedin a template. The first problemissolvedusinga combinationof edgedetectionandshapefit-ting. Thesecondproblemis solvedusinganew constrainedSVM regressiontechnique,in which a thin-plate kernelis utilized to provide non-rigid shapedeformations. Thismethodoffers several advantagesover existing landmark-basedmethods. First, it hasa uniquecapability to detectandusemultiple candidatelandmarkpointsin aninput im-ageto improve landmarkdetection.Second,it canhandlethecaseof missinglandmarks,whichoftenarisesin dealingwith occludedimages.

Theproposedmethodis especiallysuitablefor segment-ing 3D imagesslice by slice, wherethereare only smallshapevariationsacrosstheneighboringslices. In this typeof applications,the segmentedshapein one slice can beusedasa templatefor its unsegmentedneighborings.Wehave appliedtheproposedmethodto extract thescalpcon-

Figure 4. Segmentation to the input image sho wn in (Fig.3). The top row is the updated�W3

(connectedas a pol ygon). The mid dle row sho ws the acquired ; �5�i� . The bottom row is the local enlar gement ofthe mid dle row. From the left to right column sho ws the intermediate results with � of Î , . , + and

S,

respectivel y.

(b)(a)

Figure 5. Segmentation to a sim ulated oc-cluded image.

toursin cryosectionheadimageswith veryencouragingre-sults.

Acknowledgements

We would like to thankProf. TamerBasarfor invalu-able suggestionson solving the optimization algorithm.This work wassupported,in part, by NSF-BES-î í - S-.h+�.h+(Z.P.L) and a BeckmanGraduateFellowship

.�S�S-S-.�S�S�+

(S.W.).

References

[1] A. A. Amini, S. Tehrani,andT. E. Weymouth. Using dy-namic programmingfor minimizing the energy of activecontoursin the presenceof hard constraints. In Proc. ofICCV, pages95–99,1988.

[2] F. L. Bookstein.Principalwarps:Thin-platesplinesandthedecompositionof deformations.IEEETrans.PAMI, 11:567–585,June1989.

[3] V. Casselles,F. Catte,T. Coll, andF. Dibos. A geometricmodelfor active contours. Numerische Mathematik, 66:1–31,Jan.1993.

[4] L. D. CohenandI. Cohen.A finite elementmethodappliedto new active contourmodelsand 3d reconstructionfromcrosssections.In Proc.of ICCV, pages587–591,1990.

[5] T. F. Cootes,C. J.Taylor, D. H. Cooper, andJ.Graham.Ac-tiveshapemodels- their trainingandapplication.ComputerVisionandImage Understanding, 61:38–59,Jan.1995.

[6] P. Huber. RobustStatistics. New York: Wiley, 1981.[7] A. K. Jain,Y. Zhong,andS.Lakshmanan.Objectmatching

using deformabletemplates. IEEE Trans.PAMI, 18:267–278,March1996.

Figure 6. Segmentation of the Î D brain image slice by slice . The top-left and bottom-right slices areÎ + slices away from the template .

[8] M. Kass,A. Witkin, andD. Terzopoulos.Snakes: Activecontourmodels.Int. Journalof ComputerVision, 1(4):321–331,1987.

[9] J. J. Koenderink. The structureof images. Biological Cy-cbernetics, 50:363–370,1984.

[10] M. E. Leventon,E. L. Grimson,andO. Faugeras.Statisti-cal shapeinfluencein geodesicactive contours.In Proc.ofCVPR, pages316–323,2000.

[11] P. Perona,T. Shiota,andJ.Malik. Anisotropicdiffusion. InGeometry-DrivenDiffusionin ComputerVision, pages73–92.Dordrecht:Kluwer, 1994.

[12] D. Rueckert andP. Burger. Geometricallydeformabletem-platesfor shape-basedsegmentationandtrackingin cardiacmr image.In EnergyMinimizationMethodsin Comp.VisionandPatternRecog.: Int. Workshop, pages83–98,1997.

[13] L. H. StaibandJ.S.Ducan.Boundaryfindingwith paramet-rically deformablemodels. IEEE Trans.PAMI, 17:1061–1075,Nov. 1992.

[14] A. W. Toga.Brain Warping. AcademicPress,1999.[15] V. N. Vapnik. The Nature of StatisticalLearningTheory.

New York: Springer-Verlag,2000.[16] G. Wahba.SplineModelsfor ObservationalData. Society

IndustrialandApplied Mathematics,1990.[17] D. J. Williams and M. Shah. A fast algorithm for active

contours.In Proc.of ICCV, pages592–595,1990.[18] Y. Zhong, A. K. Jain, andM. P. Dubuisson-Jolly. Object

tracking using deformabletemplate. IEEE Trans. PAMI,22:544–549,May 2000.

[19] S. C. Zhu and A. Yuille. Region competition: Unifyingsnakes,regiongrowing, andbayes/mdlfor multi-bandimagesegmentation.IEEETrans.PAMI, 18(9):884–900,1996.