1

PartsandWholesDonaldD.Hoffman

Visionstartswithashowerofphotonsfocusedbythelensoftheeyeontotheretina.Eachretinahas

roughly125millionphotoreceptors,andeachphotoreceptorsignalsroughlyinproportiontothe

numberofphotonsitcatches(Alexiades&Khanal,2007).Onecanthinkofthephotoreceptormosaic

asreporting125millionnumbers,oneforthequantityofphotonscaughtateachphotoreceptor.This

massivearrayofnumbersisthestartingpointofvision.Ithasnoobjects,shapes,parts,colors,

texturesormotions.Theobjectswesee,andalltheirvisualproperties,mustbeconstructedbythe

visualsystemoutofthisbewilderingtorrentofnumbers.Inparticular,thepartsandwholesofvisual

objectsarenotgiven,theymustbeconstructed.

Thisconstructiveprocesshasbeenshapedbynaturalselectiontoguideadaptivebehaviorin

ourniche.Aeonsofrandommutations,togetherwiththecullingofmutationsthatarelessfit,have

ledtotheconstructiveprocessesthatcreatethepartsandwholesofobjectsthatweseetoday.This

raisesthequestion:Canwecharacterize,withmathematicalprecision,theprocessbywhichparts

andwholesareconstructed?

Weconstructthevisualworldtohavethreespatialdimensions,andweconstructwhole

objectstobecompactregionswithinthatthree-dimensional(3D)space,eachobjecttypicallyhaving

awell-definedtwo-dimensional(2D)surfacethatboundsitscompactregion.

Mostobjectsarenotundifferentiatedwholes,butinsteadareseenashavinganatural

decompositionintoparts.Severalmathematicallypreciserulescanaccountforthepartsweseein

manyobjects.Hereweexploretheminimarule,theshort-cutrule,andpartsaliencerules.Webegin

withtheminimarule(Hoffman&Richards,1984).

MinimaRule:Divide3Dshapesintopartsalongconcavecuspsandalongnegativeminimaofthe

principalcurvatures,alongtheirassociatedlinesofcurvature.

2

AnexampleofthepartboundariesdefinedbythisruleisgiveninFigure1.In(a),whenthethree

dotsappeartolieonthefacesofasinglecube,thenthelinesthatboundthatcubeareconcavecusps,

andthereforearepartboundariesaccordingtotheminimarule.In(b)thecirculardashedcontours

arenegativeminimaoftheprincipalcurvaturesalongtheirassociatedlinesofcurvature,andare

thereforepartboundariesaccordingtotheminimarule.

Figure1.Illustrationofpartboundariesdefinedbytheminimarule.(a)Partboundariesdefinedby

concavecusps.(b)Partboundariesdefinedbynegativeminimaoftheprincipalcurvaturesalong

theirassociatedlinesofcurvature.

Tounderstandthestatementoftheminimarule,recallthatmost2Dsurfacesembeddedin

3Dspaceareorientable,i.e.,theyadmittwodistinctfieldsofsurfacenormals.Anexceptionare

unusualsurfacessuchastheMoebiusstriporKleinbottle.Formanyorientablesurfaces,human

visionassignsonesideofthesurfacetobe“figure,”i.e.,theobject,andtheothersidetobe“ground,”

i.e.,thespacearoundtheobject(Rubin,1915/1958).Weadopttheconventionthatsurfacenormals

pointtowardsthefiguresideofthesurface.

Withthisconvention,concavecuspspointtowardthefiguresideofthesurface,andconvextoits

groundside.Theconcavecuspsarepartboundariesaccordingtotheminimarule.Ifhumanvision

reversesfigureandground,thenconcaveandconvexcuspsreverserolesand,accordingtothe

minimarule,newpartboundariesshouldbeseen.ThiscanbecheckedinFigure1(a).Atfirstthe

3

threedotsappeartobeonsinglecube,andtheconcavecuspsaroundthatcubeareitspart

boundaries.Butafterviewingtheimageforawhile,figureandgroundwillappeartoreverseandthe

threedotswillsuddenlyappeartolieonthreedifferentcubes.Thesenewcubesaredefinedbynew

concavecuspsthatusedtobeconvexcuspswhenthethreedotswereseentobeonasinglecube.

Recallthatateverysmoothpointofa2Dsurfaceembeddedin3Dspace,therearetwo

“principaldirections,”onedirectioninwhichthesurfacecurvesmost,andanorthogonaldirectionin

whichthesurfacecurvesleast(doCarmo,1976).Linesofgreatestcurvaturearecurvesonthe

surfacewhosetangentsalwayspointsinadirectionofgreatestcurvature;linesofleastcurvatureare

curveswhosetangentsalwayspointsinadirectionofleastcurvature.InFigure1(b)thecosine-like

curvesradiatedfromthecenterarelinesofcurvature.Ifweadopttheconventionthatcurvatureis

negativeforregionsofcurvesonthesurfacethatareconcave,andpositiveforregionsthatare

convex,thenthenegativeminimaofcurvaturealonglinesofcurvatureareindicatedbythedashed

circularcontoursinFigure1(b).Thesearethepartboundariesdefinedbytheminimarule.Onesees

partsthatlooklikehillsdividedfromeachotherbythesedashedcircularcontours,inaccordwith

theminimarule.Ifoneturnsthepageupsidedown,thenfigureandgroundwillappeartoreverse,

andthedashedcircularcontourswillnolongerbenegativeminimaofcurvature.Insteadtheyare

positivemaximaofcurvature,andappeartolieontopofanewsetofhillsthataredefinedbythe

newnegativeminimaofcurvature.

Theminimaruleforpartboundariesonthesurfacesof3Dobjectsleadsnaturallytoa

minimaruleforpartboundariesofsilhouettes(Hoffman&Richards.1984).

Minimaruleforsilhouettes.Divideeachsilhouetteintopartsatconcavecuspsandnegative

minimaofitsboundingcontour.

Thisruleisillustratedbythewell-knownface-gobletillusionshowninFigure2(a).It

sometimesappearstobeasinglegoblet.Butatothertimes,duetoafigure-groundreversal,it

appearstobetwofaces.InFigure2(b)areshownthetwosetsofpartboundariesdefinedbythe

minimaruleforsilhouettes.Ifthebowlisseenasthefigure,thenthepartboundariesareatthe

4

pointsindicatedbytheshortlinesegmentsontheleftsideofthedrawing.Thesepartboundaries

dividethesilhouetteintopartsthatcorrespondtothelip,bowl,stemandbaseofthegoblet.Ifthe

facesareseenasthefigure,thentheregionsofpositivecurvaturebecomeregionsofnegative

curvature,andviceversa.Thereforethepartsdefinedbytheminimaruleforsilhouetteschange.The

newpartboundariesareatthepointsindicatedbytheshortlinesegmentsontherightsideofthe

drawing.Thesepartboundariesdividethesilhouetteintopartsthatcorrespondtotheforehead,

nose,lipsandchinofthefaces.Noticethatthepartsdefinedbytheminimarulecorrespondto

regionsoftheshapethathavesinglewordlabels.Forinstance,ifthegobletisfigure,thenonepart

definedbytheminimaruleiscalledthebowl.Thissameregion,ifthefaceisfigure,wouldrequirea

morecomplicatedphrasetodescribeit,suchas“lowerhalfoftheforeheadandupperhalfofthe

nose.”Thissuggeststhatthepartsdefinedbytheminimarulearenaturalunitsofthevisual

representationofshape,andareamongtheunitsthatarenamedbysubsequentlinguisticprocessing

inthebrain.

Figure2.Theminimaruleforsilhouettes,illustratedonthefacegobletillusion.(a)Theface-goblet

illusion.(b)Thetwosetsofpartboundariesdefinedbytheminimaruleforsilhouettes,oneforthe

facesseenasfigureandtheotherforthegobletseenasfigure.

Theminimaruledefinespartboundaries,butinsomecasesdoesnotdefineauniquepart

cut.Forinstance,Figure3(a)showsashapewiththepartboundariesdefinedbytheminimarulefor

5

silhouettes.Howeverthesepartboundariescouldinprinciplebejoinedbyapartcuteitherasshown

inFigure3(b)or(c).ExperimentsbySingh,SeyranianandHoffman(1999)demonstratehuman

visionpreferscutsthatareshorter.This“short-cutrule”leadsthevisualsystemtopreferthecutin

(c)overthecutin(b).

Figure3.Anillustrationoftheshort-cutrule.(a)Asilhouettewithpartboundariesdefinedbythe

minimaruleforsilhouettes.(b)Partcutsthatarenotpreferredbytheshort-cutrule.(c)Partcuts

thatarepreferredbytheshort-cutrule.

Partboundariesdefinedbytheminimarulecanvaryintheirperceptualsalience(Hoffman&

Singh,1997).InFigure4(a)therearetwopartboundaries,oneatthesharpconcavecuspinthe

middleoftheshape,andoneattheconcavecuspdirectlybelowit.Theangleofthetopboundaryis

moreacutethanthatofthelowerboundary,andthismakesitperceptuallymoresalient.InFigure4

(b)thereareagaintwopartboundaries,onewithhighmagnitudeofcurvature,andonedirectly

belowitwithlowermagnitudeofcurvature.Theboundarywithhighermagnitudeofcurvatureis

perceptuallymoresalient.Becausecurvatureisnotascaleinvariantquantity,curvaturesat

boundariesmustbenormalizedinacanonicalfashionbeforetheirsalienceiscompared(see

Hoffman&Singh,1997,fortechnicaldetails).

6

Figure4.Salienceofpartboundaries.(a)Theconcavecuspontophasamoreacuteanglethanthe

concavecuspdirectlybelowit,andisthereforemoresalient.(b)Thenegativeminimaboundaryon

tophasgreater(normalized)curvaturethantheonedirectlybelowit,andthereforeismoresalient.

Thesalienceofapartboundaryinfluencestheperceptionoffigureandground(Hoffman&

Singh,1997).InFigure5(a)thestandardface-gobletillusionhasbeenalteredsothatthepart

boundariesassociatedwiththegobletaremoresalient.InFigure5(b)thepartboundaries

associatedwiththefacesaremoresalient.Whensubjectswereshowntheseimagesfor250ms,they

wereabout3timesaslikelytoseeFigure5(a)asagobletthanFigure5(b).

Figure5.Salienceofpartboundariesaffectsperceptionoffigureandground.In(a)theboundaries

associatedwiththegobletaremoresalient,andin(b)theboundariesassociatedwiththefacesare

moresalient.Humansubjectsaremorelikelytosee(a)asbeingagobletand(b)asbeingtwofaces.

7

Thesalienceofapartisinfluencedbythesalienceofitsboundaries(Hoffman&Singh,

1997).Itisalsoinfluencedbytheprotrusionofthepart—whichisroughlyhowfarthepartsticks

out—andbythevolumeofthepartrelativetothevolumeofthewholeobject.Theseinformal

descriptionscanbegivenprecisegeometricdefinitions(Hoffman&Singh,1997).

Ifpartsdefinedbytheminimarulecorrespondtonaturalunitsofthevisualrepresentation

ofshape,thenonewouldexpectthatsuchpartswouldaffectjudgmentsofshapesimilarity.InFigure

6,whichofthetwohalfmoonsontherightlooksmostsimilartothesinglehalfmoonontheleft?Ina

controlledexperiment,subjectsfoundthehalfmoononthebottomtobemoresimilar(Hoffman

1983a,1983b).Notethattheboundingcurveofthehalfmoonontopispoint-for-pointidenticalto

thehalfmoonontheleft.Indeed,theyfitlikepuzzlepieces.Thehalfmoononthebottomhasbeen

figure-groundreversedfromthehalfmoonontheleft,andthepositionoftwopartshasbeen

switched.Ifourvisualjudgmentsofshapesimilaritywerebasedonapoint-for-pointcomparison,

thenthehalfmoononthetopwouldbejudgedmoresimilar.However,humanvisionappearsto

judgeshapesimilaritypartbypart,ratherthanpointbypoint.

Figure6.Ademonstrationthatminimapartsinfluencejudgmentsofshapesimilarity.

8

Ourperceptionsofsymmetryandrepetitionprovideanotherdemonstrationthathuman

visionanalyzesshapespartbypartratherthanpointbypoint.Mach(1885)notedthatitisoften

easiertodetectsymmetryinashapethantodetectrepetition.Forinstance,inFigure7therepetition

in(a)isdifficulttodetectwhereasthesymmetryin(b)iseasytodetect.Whatissurprisingaboutthis

fromamathematicalpointofviewisthatsymmetryinvolvesrepetitionplusmirrorreversal.Sothe

numberofmathematicaltransformationsrequiredtocreatesymmetryismorethanthatrequiredto

createrepetition.

Figure7.Detectingsymmetryandrepetition.In(a)therepetitionisdifficulttodetect.In(b)the

symmetryiseasytodetect.

BaylisandDriver(1994,1995a,1995b)foundthatrepetitioncanbemademoreeasyto

detectthansymmetryifonereversesfigureandground.Forinstance,inFigure8thesymmetryin(a)

isnoteasilydetectedwhereasin(b)therepetitioniseasilydetected.ThedifferencebetweenFigure

8and7ishowfigureandgroundareseenacrossthecurves.InFigure7theperceptionoffigureand

groundmakesthetwosidesofthesymmetricfigurehavethesameminima-ruleparts(mirror

reversed),makingsymmetryeasiertodetect.InFigure8theperceptionoffigureandgroundmakes

thetwosidesoftherepetitionfigurehavethesameminima-ruleparts,makingrepetitioneasierto

detect.

9

Figure8.Symmetryandrepetitionrevisited.In(a)thesymmetryisnoteasilydetectedandin(b)the

repetitioniseasilydetected.

Partboundariesdefinedbytheminimaruleforsilhouettesappeartobecomputedquickly

andearlyinthestreamofvisualprocessing.Figure9illustratesthiswithadisplaysimilartothat

usedbyHulleman,teWinkel&Boselie(2000).InFigure9(a)theobjectwithapartboundarypops

outamongconvexdistracters.InFigure9(b)theconvexobjectdoesnotpopoutamongobjectswith

partboundaries.VisualsearchexperimentsbyXuandSingh(2002)indicatethatparsingobjectsat

negativeminimaofcurvatureoccursobligatorily.

10

Figure9.DemonstrationsimilartostimulifromHullemanetal.(2000)showingthatpartboundaries

arecomputedquicklyandearlyinvisualprocessing.In(a)theobjectwithapartboundarypopsout.

In(b)theconvexobjectdoesnotpopout.

Partsdefinedbytheminimaruleaffecttheperceptionoftransparency(Singh&Hoffman,

1998).Figure10(a)showsastandardexampleoftransparency.InFigure10(b)minimapart

boundarieshavebeenaddedatpreciselythelocationwhereluminancechanges,andthisresultsina

greatlyreducedperceptionoftransparency.Ifthepartboundaryisnotalignedwiththeluminance

change,asinFigure10(c),thentheperceptionoftransparencyreturns.Thelossoftransparencyin

Figure10(b)appearstohavetwocontributions,oneduetopartsandoneduetogenericity.Different

partscanhavedifferentproperties,andthevisualsystemisthereforeinclinedtoattributethechange

inluminanceinFigure10(b)toadifferenceinpartpropertiesratherthantotransparency.Ifthe

cuspsinFigure10(b)aremadeconvexratherthanconcave,thereisagainareductioninperceived

transparency,butnotasmuchasforthepartboundaries.Thisindicatesthatthenongeneric

alignmentofcusps(concaveorconvex)alsoinclinesthevisualsystemtorejecttransparency.

Figure10.Minimapartboundariesandperceivedtransparency.Thetransparencyseenin(a)

disappearsin(b)whereapartboundaryalignswiththeluminancechange.Transparencyisseen

againin(c)wherethepartboundaryisnotalignedwiththeluminancechange.

11

Partsdefinedbytheminimaruleaffectrecognitionmemory.Braunstein,Hoffman

andSaidpour(1989)showedsurfacesofrevolutiontohumanobservers.Aftereachsurface

waspresented,observerswereshownfourseparatepartsandhadtochoosewhichpart

wasinthesurfacejustseen.Twoofthepartsweredistracters,buttwowereactualpartsof

thesurface,onedefinedbynegativeminimaofcurvatureandonedefinedbypositive

maximaofcurvature.Observersweretwiceaslikelytochoosetheminimapartratherthan

themaximapart,suggestingthatthesepartsarenaturalunitsforrecognitionmemory.

Minimapartsaffectvisualattention.SinghandSchollfoundthatattentioncanshift

morequicklywithinapartthanacrossparts,andthatgreatersalienceofapartboundary

makestheshiftacrosspartsevenslower(Scholl,2000;SinghandScholl,2000).

Figure11.Transversalintersectionandconcavecusp.

Whydoesthevisualsystemuseconcavecuspsandnegativeminimaofcurvatureto

definepartboundaries?Considertwoseparateobjects,asshownontheleftinFigure11.If

thosetwoobjectsarejoinedtoformasingleobject,asshownontherightinFigure11,then

thetwooriginalobjectsaregoodcandidatesfornaturalpartsofthenewobject.Inthe

12

genericcase,thetwoobjectsintersecttransversally(Guillemin&Pollack,1974),leadingtothe

concavecuspindicatedbydashedcontours.Ifthiscuspissmoothed,itleavesanegative

minimumofcurvature.Thusgenericityandtransversalityappeartobethemathematical

principalsunderlyingtheminimarule.

Itremainsanopenproblemtopreciselycharacterizeallthequalitativelydifferent

kindsofpartsthatcanresultfromapplyingtheminimaandshort-cutrulesto3Dshapes.

Priortheoriesofshapeperceptionhaveusedvariousprimitiveshapesasparts,including

polyhedra(Roberts,1965;Waltz,1975;Winston,1975),generalizedconesandcylinders(Binford,

1971;Brooks,1981;MarrandNishihara,1978),superquadrics(Pentland,1986)andgeons

(Biederman,1987).Althoughthereissubstantialevidencefortheimportanceofpartsinthevisual

perceptionofshapes,thereisnoexperimentalevidencethatanyoftheseparticularprimitivesplaya

roleinthehumanvisualperceptionofparts.Thesetofgeons,forinstance,istoorestrictive.Geons

canhavetruncatedorpointedtips,butcannothaveroundedtips,eventhoughthedistinction

betweenrounded,pointedandtruncatedtipsisanonaccidentalproperty(Binford,1981;Lowe,

1987;Witkin&Tenenbaum,1983)thatsurvivesprojectionandiseasilyseenbyhumanvision.No

geoncanhaveacrosssectionthatchangesshapeasitsweepsalongthegeon’saxis,butsuch

changingcrosssectionsposenoproblemforhumanvision.

Insummary,humanvisionappearstodecomposeshapesintopartsusingtheminimaand

short-cutrules,bothfor3Dshapesandfor2Dsilhouettes.Theresultingpartsappeartobecomputed

quicklyandearlyinvisualprocessing,toaffectourperceptionsofshapesimilarity,toinfluenceour

recognitionmemoryforshapes,tointeractwithourperceptionsofsymmetryandrepetition,toalter

ourperceptionsoftransparency,andtoinfluenceourdeploymentofvisualattention.Partsarea

powerfulcomponentintheanalysisandrepresentationofvisualshapes.

13

References

Alexiades,V.,&Khanal,H.(2007).Multiphotonresponseofretinalrodphotoreceptors.Sixth

MississippiStateConferenceonDifferentialEquationsandComputationalSimulations,

ElectronicJournalofDifferentialEquations,Conference15,1-9.

Baylis,G.C.,&Driver,J.(1994).Parallelcomputationofsymmetrybutnotrepetitioninsingle

visualobjects.VisualCognition,1,377-400.

Baylis,G.C.,&Driver,J.(1995a).One-sidededgeasignmentinvision.1.Figure-ground

segmentationandattentiontoobjects.CurrentDirectionsinPsychologicalScience,4,

140-146.

Baylis,G.C.,&Driver,J.(1995b).Obligatoryedgeassignmentinvision-Theroleoffigureand

partsegmentationinsymmetrydetection.JournalofExperimentalPsychology:Human

PerceptionandPerformance,21,1323-1342.

Biederman,I.(1987).Recognition-by-components:Atheoryofhumanimageunderstanding.

PsychologicalReview,94,115-147.

Binford,T.O.(1971,December).Visualperceptionbycomputer.IEEESystemsScienceandCybernetics

Conference,Miami,FL.

Binford,T.O.(1981).Inferringsurfacesfromimages.ArtificialIntelligence,17,205-244.

Braunstein,M.L.,Hoffman,D.D.,&Saidpour,A.(1989).Partsofvisualobjects:an

experimentaltestoftheminimarule.Perception,18,817-826.

Brooks,R.A.(1981).Symbolicreasoningamong3-Dmodelsand2-Dimages.ArtificialIntelligence,

17,205–244.

DoCarmo,M.(1976).Differentialgeometryofcurvesandsurfaces.EnglewoodCliffs,NJ,Prentice-Hall.

Guillemin,V.&Pollack,A.1974.Differentialtopology.EnglewoodCliffs,NewJersey:Prentice-Hall.

Hoffman,D.D.(1983a).Representingshapesforvisualrecognition.PhDThesis,MIT.Hoffman,D.D.(1983b).Theinterpretationofvisualillusions.ScientificAmerican,249,6,154-162.

Hoffman,D.D.(1998).VisualIntelligence:Howwecreatewhatwesee.NewYork:Norton.

Hoffman,D.D.,&Richards,W.A.(1984).Partsofrecognition.Cognition,18,65-96.

Hoffman,D.D.&Singh,M.(1997).Salienceofvisualparts.Cognition,63,29-78.

14

Hulleman,J.,teWinkel,W.,&Boselie,F.(2000).Concavitiesasbasicfeaturesinvisualsearch:

Evidencefromsearchasymmetries.Perception&Psychophysics,62,162-174.

Lowe,D.(1985).Perceptualorganizationandvisualrecognition.Amsterdam:Kluwer.

Mach,E.1885/1959.Theanalysisofsensations,andtherelationofthephysicaltothepsychical.New

York:Dover.(TranslatedbyC.M.Williams.)

Marr,D.,&Nishihara,H.K.(1978).Representationandrecognitionofthree-dimensionalshapes.

ProceedingsoftheRoyalSocietyofLondon,SeriesB,200,269–294.

Pentland,A.P.(1986).Perceptualorganizationandtherepresentationofnaturalform.Artificial

Intelligence,28,293–331.

Roberts,L.G.(1965).Machineperceptionofthree-dimensionalsolids.InJ.T.Tippettetal.(Eds.),

Opticalandelectroopticalinformationprocessing(pp.211–277).Cambridge,MA:MITPress.

Rubin,E.(1958).Figureandground.InD.C.Beardslee(Ed),Readingsinperception,Princeton,New

Jersey:VanNostrand,194-203.(ReprintedfromVisuellwahrgenommenefiguren,1915,

Copenhagen:GyldenalskeBoghandel.)

Scholl,B.(2001).Objectsandattention:Thestateoftheart.Cognition,80,1-46.

Singh,M.&Hoffman,D.D.(1998).Partboundariesaltertheperceptionoftransparency.

PsychologicalScience,9,370-378.

Singh,M,&Scholl,B.(2000).Usingattentionalcueingtoexplorepartstructure.Posterpresentedat

theAnnualSymposiumofObjectPerceptionandMemory,NewOrleans,Louisiana(November

2000).

Singh,M.,Seyraninan,G.D.,&Hoffman(1999).Parsingsilhouettes:Theshort-cutrule.

PerceptionandPsychophysics,61,636-660.

Waltz,D.(1975).Generatingsemanticdescriptionsfromdrawingsofsceneswithshadows.InP.

Winston(Ed.),Thepsychologyofcomputervision(pp.19–91).NewYork:McGraw-Hill.

Winston,P.A.(1975).Learningstructuraldescriptionsfromexamples.InP.H.Winston(Ed.),The

psychologyofcomputervision(pp.157–209).NewYork:McGraw-Hill.

Witkin,A.P.,&Tenenbaum,J.M.(1983).Ontheroleofstructureinvision.InJ.Beck,B.Hope,

&A.Rosenfeld(Eds.),Humanandmachinevision(pp.481-543).NewYork:Academic

15

Press.

Xu,Y.&Singh,M.(2002).Earlycomputationofpartstructure:Evidencefromvisualsearch.

Perception&Psychophysics,64,1039-1054.