Automatic Synthesis of Isotr opic Textures on Subdivision Surfaces fromSample Images

JoelCastellanosUniversityof New Mexico

CurtisSirl-Moore†

Universityof New MexicoLanceWilliams‡

Universityof New Mexico

Abstract

Wedescribeafully automaticmethodof synthesizingisotropictex-tureson subdivision surfacesfrom sampleimages.We restrictour-selvesto isotropic texturesbecauseonly isotropictexturescanbeautomaticallygeneratedon an arbitrarysurfacein the absenceofa parametrization.Unlike previous approaches,texture synthesisis accomplishedin a coarse-to-finefashionby constructingbothGaussianand Laplacianpyramid representationsof the synthetictexture.TheinverseLaplacianpyramidtransformis usedto gener-atefirst approximationsto thetextureateachlevel of theassociatedGaussianpyramid.Theseapproximationsarerefinedusingamodi-fiednearestneighborsearchprocesswhichpreservesthefirst-orderstatisticsof thesampletexture. This searchprocesscanbeconsid-eredto be a samplingprocedurefor an implicitly definedMarkovrandomfield. The resulting texture is generateddirectly on thesubdivision surface. Within the domainof isotropic textures,theproposedmethodoffersimprovementsin faithful reproductionof asample’s appearanceover a wide rangeof scales.

Keywords: texturesynthesis,laplacianpyramid,subdivision sur-faces

1 Intr oduction

A texturecanbe thoughtof asa visualpatternthat is repeatedac-cordingto ruleswhichareappliedeitherdeterministicallyor atran-dom.Thepatternmaybecompletelydeterministic,suchasa chessboard,completelyrandom,suchas snow on a television screen,or exhibit intermediatedegreesof randomness,suchastreebark,clouds, or marble. Textures can also be classifiedon a contin-uum from isotropic (having no preferredorientation)to stronglyanisotropic.An exampleof the formerwould bepolishedgranite,while horizontallayersof sandstonewould be an exampleof thelatter. Thereis a strongconnectionbetweenisotropy andrandom-ness: With the trivial exceptionof uniform colors, deterministictexturesmustbeanisotropic.Finally, weobserve thatatexturemayexhibit different degreesof randomnessand isotropy at differentscales.SeeFigure 1.

e-mail: joel@cs.unm.edu†e-mail: joel@cs.unm.edu‡e-mail:williams@cs.unm.edu

(a) (b)

(c) (d)

Figure1: (a) A floral print fabric is deterministicandhencenon-isotropic. (b) The large scalestructureof a brick wall is non-isotropic.However, atsmallscales,thestructurebecomesisotropic.(c) Texturesformed by the randomarrangementof identical ele-mentsexhibit aspectsof both determinismandrandomness.Suchtexturesareisotropicat all scales.(d) Sometexturescanonly becharacterizedstatistically. Burlwood is anexampleof a textureofthissortwhich is isotropicat all scales.

Texturesynthesisis thecreationof a new imagethatis differentfrom the original, yet canbe saidto have the samevisual appear-anceastheoriginal. Texturesynthesishasbeenanactive researchtopic in bothcomputervision andgraphics.Themostcommonap-plication is to synthesizethe textureon thesurfaceof anobjectinacomputergeneratedscenein orderto increaseits degreeof visualrealism[Ebertetal. 1994].Other, image-basedapplicationsincludeholefilling[Efros andLeung1999],imageandvideocompression,andanimation[Bar-Josephetal. 2001].

2 Previous Work

2.1 Efros and Leung

Oneof thesimplestandmostsuccessfultexturesynthesismethodsis that of [Efros andLeung1999],which wasinspiredby an ideaof Shannon’s for producingEnglishsoundingtext by modelinglan-guageasageneralizedMarkov chain.Theirmethodis basedonthesimpleassumptionthat theprobabilitydistribution of RGB valuesfor agivenpixel onlydependsontheRGBvaluesof pixelsin alocalneighbourhoodof thegiven pixel, i.e., the texturecanbemodeledasa Markov randomfield. Using this assumption,thetexturesyn-thesisprocessgrows a new texture outwardsfrom an initial seed,onepixel at a time. Thevalueof thenext pixel to besynthesizedischosenat randomfrom thevaluesof samplepixelswith neighbour-hoodsresemblingthe neighborhoodof the given pixel, i.e., thosewith similar spatialcontexts. Theonly userspecifiedparameteristhesizeof theneighbourhoodof a pixel, which controlsthedegreeof randomnessof thesynthesizedtexture.

2.2 Wei and Levoy

Our work builds most directly upon the multi-resolution,neighborhood-search-basedmethoddevelopedby [Wei andLevoy2001].We describetheir processin detailbecauseourprocesspar-allelsit. [WeiandLevoy 2001]describemethodsfor texturesynthe-sis on both parameterized,andnon-parameterizedsurfaces.How-ever, in this paper, we areprimarily concernedwith their work onnon-parameterizedsurfaces.For non-parameterizedsurfaces,theirmethodtakestwo inputs: (1) A sampleimage;and(2) A triangu-lation of the surfaceon which the texture is to be synthesized.AGaussianpyramid with K levels is built from the sampleimage.The triangulationof the surfaceis thenrecursively subdivided sothat all the trianglesareapproximatelyequilateralandcongruent.Eachfaceof theremeshedsurfaceis subdividedK 1 times.Eachsubdivision of the remeshedsurfacecorrespondsto a level of theGaussianpyramidof thesampleimage.Theoriginal remeshedsur-facecorrespondsto thelowestlevel of thesample’s Gaussianpyra-mid andthefinestsubdivision is thetriangulationonwhichthefinalsynthetictexturewill berepresented.Thetexture is synthesizedina coarse-to-finemanner. After initialization, thesynthesisprocesstakesplacein two distinct matchingphasesperGaussianpyramidlevel:

1. Initialization. To begin, level k is populatedwith randomlychosenRGB valuesfrom level k of the analysisGaussianpyramid.

2. Multi-level neighborhood matching. In the first matchingphase,eachfaceat level k is visited in randomorder. Like[Efros andLeung1999],a face’s RGB valueis replacedwiththeRGB valueof the facefrom level k of theanalysisGaus-sianpyramid with mostsimilar neighborhood.However, inorderto propagateinformationfrom coarse-to-finescales,thefirst phaseneighborhoodspanstwo levels of the Gaussianpyramid, i.e., it consistsof neighboringfacesat level k andlevel k 1.

3. Single-level neighborhood matching. In thesecondmatch-ing phase,thefinal valueof thesynthesisfaceat level k is de-terminedby repeatingthenon-parametricsamplingprocedureusedin thefirst phase.However, unlike thefirst phase,whichwasusedto propagateinformationfrom coarse-to-finescales,theneighborhoodwhich is usedin thesecondphaseconsistssolelyof facesat level k in theanalysisGaussianpyramid.

Finally, in order to compareneighborhoodsof the triangulatedsurface(on thesynthesisside),with imageneighborhoods(on theanalysisside)they projecttheneighborhoodof thetriangulatedsur-faceontoa tangentplaneandresampleit usinga rectangulargrid.

2.3 Quilting

Quilting methodsstitch togetherrepresentative patchesof a sam-ple texture image. Quilting methodssuchas thosedescribedby[Praunet al. 2000;Efros andFreeman2001]canbevery fastandproduceexcellentresultsfor awiderangeof texturesonparameter-izedsurfaces.They areespeciallygoodatsynthesizingdeterminis-tic textures. Thereare,however, at leasttwo disadvantagesinher-ent to any quilting process.First, quilting methodscannotrepro-ducestructureatscaleslargerthanthesizeof thequilting elements.Second,unlike themethodof [Wei andLevoy 2001](andour ownmethod),quilting cannotbeusedto synthesizeisotropicvariantsofnon-isotropictextures. This limits their utility on surfaceslackingan explicit two dimensionalparameterization.The reasonis thaterror dueto misalignmentof quilting elementscanonly be mini-mizedby exercisingcomparitively few degreesof freedomwhencomparedto methodswhichsynthesizea texurepixel by pixel.

3 Our Appr oach

Our methodfor texturesynthesison surfaceshasbeeninspiredbyother multi-resolution,texture synthesismethodswhich usenon-parametricmethodstosampleanimplicitly definedMarkov randomfield. Althoughourprimaryinspirationwas[Wei andLevoy 2001],wehavealsobeeninfluencedby [EfrosandLeung1999],[Debonet1997],and[Bar-Josephet al. 2001]). Like [Wei andLevoy 2001]we synthesizetexture in a coarse-to-finemannerby searchingforthefacein theanalysisGaussianpyramidwith mostsimilar neigh-borhood.However, our methoddiffersfrom [Wei andLevoy 2001]in threesignificantways. First, our methodpropagatesinforma-tion from coarse-to-finein a moreprincipledmanner, namely, byincrementallyinverting a Laplacianpyramid transform. Second,thegreedymethodof non-paramatricsamplingdescribedby [WeiandLevoy 2001] hasbeenreplacedwith a samplingprocedureofourown designwhichdoesabetterjob of preservingthefirst-orderstatisticsof the sampledtexture. Finally, unlike [Wei andLevoy2001],asa preprocesssingstep,we resamplethe input textureus-ing a triangulargrid so that its geometrymoreclosely resemblesthatof thesubdivision surfaceon which thetexturewill besynthe-sized.Theconsequenceis that in our system,themulti-resolutionpyramiddatastructuresonboththeanalysisandsynthesissidesarebasedon triangulations.This considerablysimplifiestheneighbor-hoodmatchingprocedurewhichis theheartof thetexturesynthesisprocess.

3.1 Coarse-to-fine Information Flow

Whereas[Wei andLevoy 2001]computea Gaussianpyramid rep-resentationof the input textureanduseit to synthesizea Gaussianpyramidrepresentationof theoutputtexture,ourmethodcomputesbothGaussianandLaplacianpyramid representationsof the inputtexturein orderto synthesizebothGaussianandLaplacianpyramid

representationsof theoutputtexture.To this end,we have general-izedthe

Laplacianpyramidtransformsothatit canbeusedto repre-

sentfunctionsdefinedontriangulations.Onepropertyof theLapla-cianpyramid which distinguishesit from theGaussianpyramid isthat,likethewavelettransformsto whichit is intimatelyrelated,theLaplacianpyramid transformcanbe inverted. That is, theoriginalfunction canbe recoveredfrom the Laplacianpyramid anda sin-gle residualGaussianvalue.Becausewe synthesizebothGaussianandLaplacianpyramids,wecanincrementallyinvert theLaplacianpyramid transformin order to propagateinformation from coarsescalesto fine scalesduringthesynthesisprocess.This mechanismavoidstheawkwardmulti-level neighborhoodsandmultiplematch-ing phaseswhich [Wei andLevoy 2001]useto propagateinforma-tion from coarse-to-finescales.

The samplingprocedurewe useto samplethe Markov randomfield at level k of thesynthesisGaussianpyramiddoesnot just re-turn ananalysisGaussianpyramidvalue,it alsoreturnsthevaluesof the analysisLaplacianchildren at level k 1. Oncethe sam-pling processfor level k of thesynthesisGaussianpyramidhascon-verged,weinitialize level k 1 by upsamplinglevel k, convolving itwith aGaussiankernel,andaddingthis to level k 1 of thesynthe-sisLaplacianpyramid. This incrementalinversionof thesynthesisLaplacianpyramid transformgeneratesthe first approximationofthetextureat level k 1 of thesynthesisGaussianpyramid.

In summary, [Wei andLevoy 2001] usea two phaseprocessateachlevel. Theirprocessconsistsof randominitialization; followedby (1) onepassof multi-level neighborhoodmatching;and(2) onepassof single-level neighborhoodmatching. We usea consider-ably simplerprocessconsistingof initialization using the inverseLaplacianpyramid transformfollowed by onepassof single-levelmatching.

3.2 Sampling Procedure

Thesecondmajordifferencebetweenour methodandthatof [WeiandLevoy 2001] is the non-parametricsamplingprocedure.Oursamplingprocedurefindsareasonablygoodmatchbasedonneigh-borhoodsimilarity while simultaneouslypreservingthe first-orderstatisticsof the sampletexture. This is done by using a non-parametricsamplingmethodwhich approximatessamplingwith-out replacement.An exampleof samplingwithout replacementisa child repeatedlyselectingandremoving jellybeansfrom a jar. Insamplingwith replacement,theselectedjellybeanis returnedto thejar beforea subsequentselectionis made. For finite populations,thesetwo samplingmethodsgive differentresults.

We will explain our samplingprocedureusinga somewhat far-fetchedanalogy. We hopethe readerwill forgive us! A groupofchildren are allowed to choosejellybeansfrom a jar containingmany differentcolors. However, after choosinga jellybean,theymustcut it in half and returnhalf to the jar. A child approachesthejellybeanjar with theideaof finding his/herfavorite color. Shesearchesfor andselectstheclosestmatch,andthenreturnshalf ofthe jellybeanto the jar. Subsequentselectionsaremadein turn bydifferentchildreneachsearchingfor their favorite colors.It is con-ceivablethatfor somemix of childrenandjellybeansthataconflictnever arises:Every child alwaysfinds a whole beanof his/herfa-voritecolor. A morelikely scenariois thatachild occasionallyonlyfindshalf beansor quarterbeansof thatcolor. In sucha situation,thechild maybewilling to settlefor awholebeanof asimilarcolorif all beansof her favorite color have alreadybeenhalvedor quar-tered,i.e., if toomany childrenchoosepink,eventuallytheredbeanwill startlookingquitetasty.

Notethatthissamplingpolicy (unlikethesamplingwith replace-mentpolicy usedby [Efros andLeung1999]and[Wei andLevoy2001]) tendsto insurethat the distribution of colorsin the jar andthedistribution of colorswhich arechosenwill be thesame.Fur-

thermore,unlike samplingwithout replacement,theproblemof re-spectingthe choicesof children who chooselast is not overcon-strained,i.e., No child needever chooselicorice if he/shereallydislikesit!

In our algorithm, this is implementedby scalingthe error dis-tancebetweenthepair of neighborhoodsbeingcompared,i.e., onein the analysisGaussianpyramid andone in the synthesisGaus-sianpyramid,by a factorof 2 c j cmin , wherec j is thenumberoftimesthatface j in theanalysisGaussianpyramidhasalreadybeenselected,andcmin is theminimumnumbertimesthatany analysisGaussianpyramidfacehasbeenselected.Whencombinedwith therandomorderin whichudatesareperformed,thispolicy effectivelytransformsthegreedyheuristicusedby [Wei andLevoy 2001]intoagenuinesamplingprocedure.Furthermore,thisprocedurehastheadditionalbenefitof preservingthefirst-orderstatisticsof thesam-ple texture.

4 Anal ysis

4.1 Laplacian Pyramid Transf orm

TheLaplacianpyramid is a linear transformthat representsanim-ageby decomposingit into a setof bandpassimages,anda singlelowpassimagefrom which theoriginal imagecanbeperfectlyre-constructed.It is a historical precursorof the fastwavelet trans-form.

In the first stepof the Laplacianpyramid transform,the imageis lowpassfilteredby convolving it with a Gaussiankernel,andtheresult is subtractedfrom the original image. The high-frequencydetail in the imageis retainedin this differenceimage. The low-passfilteredimage(calledtheGaussianpyramidlevel) canthenbedownsampledby a factorof two in eachdimensionwithout lossofinformation. Let GK be the original image,and let g be a Gaus-siankernel of size2X 1 by 2X 1 whereX is a positive inte-ger. Then,at eachstepof the encodingprocess,the imageis de-composedinto ahalf-resolutionlow-frequency component,Gk, anda full-resolution,high-frequency component(calledthe LaplacianLevel), Lk, wherek is thepyramidlevel. More formally,

Gk 1 i j Gk g 2i 2 j (1)

Lk 1 i j Gk i j Gk g i j (2)

Givenanm m image,thenafterK iterations,wherem 2 K 1 ,theresiduallowpassGaussianlevel,G0, isasinglepixel. Theimagerepresentationconsistsof thesetof K 1 Laplacianpyramidlevels,Lk, andthefinal Gaussianpyramidlevel, G0.

Imagedecodingis donein thereverseorder. Upsampling is theprocessof insertingzerosbetweensamplepoints in both dimen-sions.EachGaussianpyramidlevel, Gk, beginningwith thelowestresolution,G0, is upsampledandinterpolatedby convolution withtheGaussiankernel,g. Thentheresultis addedto thecorrespond-ing Laplacianpyramidlevel, Lk 1 andtheprocessis repeated.Thisprocessreconstructsthe original imagewithout error. Laplacianpyramid level k canbethoughtof asbeingthedifferencebetweenGaussianpyramidlevelsk 1 andk.

For notionalconveniencewe definetwo operations:reduce andproject. The reduceoperationcorrespondsto convolution with aGaussiankernel followed by downsamplingby a factorof two ineachdimension. The project operationupsamplesby a factor oftwo in eachdimensionand then interpolatesby convolution withthesameGaussiankernel.Therefore:

Gk 1 reduce Gk (3)

Lk 1 Gk pro ject Gk (4)

4.2 Surface Remeshing

Facesof the triangulationsareusedin a way which is analogousto the pixels of an image. Therefore,it is desirablethat all faceshave approximatelyequalarea,andbe approximatelyequilateral.Thedownsamplingandupsamplingprocesseswhich we defineforsubdivision surfacesadditionally assumethat the triangulationistwo-colorable. A triangulationof a surfacewithout boundaryistwo-colorableif andonly if all of its verticesareof even valence.Computingthe error distancebetweenneighbohoodsof the inputandoutputGaussianpyramids,asrequiredby our non-parametricsamplingprocedure,is easiestwhenall of the verticeshave a va-lenceof six. Thiswill bethecasefor verticesassociatedwith facesproducedby meansof recursive subdivision of an input geometryrepresentedasa triangulation.

It follows that in order to make useof existing meshlibrariesit is necessaryto have a methodof remeshingthat satisfiestheseconstraintswhile preservingtheoriginal topology. [Turk 1992]de-scribesjust sucha method. His methodcanbe usedto convert apolygonalmeshof a surfaceinto a semi-regular triangulationwithauserspecifiednumberof vertices.Furthermore,it canbearrangedsothatall verticesareeven,andthatmosthaveavalenceof six. Fi-nally, thisprocessisguaranteedto preservethetopology. Theinitialpolygonalsurfacemaybeeitherconcaveor convex, andit mayhaveholes. Themajor restrictionis that themeshmustbe conforming,i.e., eachedgeis sharedby no morethantwo polygons.

In our implementation,thegiventriangulationis remeshedusingTurk’s method. The facesof the remeshedtriangulationform thegeometryof the residuallowpasslevel, G0, of the Gaussianpyra-mid.

4.3 Mesh Subdivision

Butterfly subdivision[Zorin et al. 1996] is usedrecursively to gen-eratethe geometryof eachsubsequentlevel of the pyramid, G1throughGK 1. ButterflySubdivisionaddsavertex to everyedge,intheprocess,subdividing every faceinto four faces.Verticesaddedto interior edgeswill always have a valenceof six, and verticesaddedto boundaryedgeswill alwayshave a valenceof four. Theschemeis interpolating,sinceits neitherdeletesnor movesexist-ing vertices. The schemeis triangularasit operateson triangularcontrol netsand producestriangular limit surfaces. It is station-ary asit usesthesamesetof rulesevery time it subdividesthenet,anduniformbecauseeverysectionof thenetis subdividedwith thesamesetof rules.Therulesfor addingverticesaresimpleandhavecompactsupport.

The butterfly schemeproducessurfaceswith continuousfirstderivatives. A drawback of the schemeis that, like many inter-polatingschemes,it producesslight ripplesandundulationsin thesurface,especiallyneartightly joint areas.Theseripplesshow upmostclearlyon monochromaticsurfaces.They arenot usuallyno-ticeableon texturedsurfaces.

4.4 Downsampling and Upsampling

In orderto constructa Laplacianpyramidof a functiondefinedona triangulation,thebasicoperationsof downsampling,upsampling,andconvolutionmustberedefined.As mentionedbefore,thegivenremeshedtriangulationbecomesthegeometryfor theresiduallow-pass,G0, of the Gaussianpyramid. Afterwards,recursive subdi-vision is usedto generatethe geometryof eachsubsequentlevelof thepyramid,G1 throughGK 1. Since,by construction,all ver-ticesof the triangulationareeven, the residuallowpasslevel, G0,is two-colorable. The butterfly subdivision processneitheraddsnor removesedgesfrom existing verticesandall new verticesareeven, therefore,all subsequentlevels of thepyramidarealsotwo-colorable.

Figure2: Color inheritance:Gk 1 (left) andGk (right).

Eachfaceof the residuallowpassgeometryis marked aseitherblack or white so that no two adjacentfaceshave the samecolor.When a faceis subdivided by the butterfly scheme,eachfaceissubdivided into four faces. The centerfaceinherits the color ofits parent,andthe threeouterfacesinherit the color which is theoppositeof its parent’s. SeeFigure 4.4.

Downsampling,populateseachcell of Gk with thevalueof thecenterchild of Gk in Gk 1. Upsampling,populateseachwhite cellin Gk with thevalueof its parentin Gk 1, andeachblackcell in Gkwith zeros.

4.5 Gaussian Convolution

In the caseof a two-dimensionalimage,the value of a pixel af-ter discreteconvolution with a given kernel can be computedasa weightedsumof the valuesof the pixel’s neighbors,wherethekerneldefinestheweightingfunction.Thecomputationis straight-forwardbecauseall of thepixelsarethesamesizeandtheirspacingis uniform. In thecaseof asurface,in placeof pixelswehavefaces,andconvolutionmustberedefinedto takeinto accountboththesizeandspacingof neighboringtriangularfaces.In theimplementationdescribedin this paper, the areaof a faceis usedasa measureofits size,andthe distancebetweena pair of facesis definedasthestraight-lineEuclideandistancebetweenthecentroidof eachface.Given that this distanceis only usedfor convolution with a Gaus-sian, which hascompactsupport,this definition of distanceis areasonableapproximation.

In orderto computethevalueof face,i, afterconvolution with aGaussiankernel,we mustdefinethekernelweightsof facesin itsneighborhood,Ω i . Thekernelweightof face j relative to faceifor j Ω i is:

Wi j g

d i j

σ a j (5)

whereg x e x2 2, d i j is theEuclideandistancebetweenfacesi and j, andσ is themaximumEuclideandistancebetweenall pairsof adjacentfaces.

In theexpressionfor Wi j , dividing by σ dilatesg, a Gaussianofunit standarddeviation, so that it is (in effect) a Gaussianof stan-darddeviation σ . The standarddeviation of the Gaussianat eachlevel of theLaplacianpyramid thereforeequalsthemaximumdis-tancebetweenany two adjacentfacesof the triangulationat thatlevel. This is a conservative designchoice(comparedto thealter-native of settingσ equalto theaveragedistancebetweenadjacentfaces),which minimizesthealiasingintroducedby downsampling,yet remainsfaithful to Burt’s formulation[Burt 1981]. As in thecaseof a two-dimensionalimage,we want the sumof the kernelweights to be unity, so we divide the convolution by the factor,∑ j ! Ω i Wi j . Thevalueof facei afterconvolution is therefore:

v "i ∑ j ! Ω i Wi jv j

∑ j ! Ω i Wi j(6)

Figure3: Geometryof G0, G1, andG2 for a 12 12 sampleimagewith K 3.

wherev j is thevalueof face j beforeconvolution.We define the neighborhood,Ω i , as i, all facesadjacentto

i, andall facesadjacentto thosefacesrecursively with the recur-sionendingin eachdirectionwhenthedistancebetweenthefaceinquestionandi, is greaterthan3σ . Theweightsfor any facesfartherthanthis distanceveryquickly approachzero.

4.6 Mapping Rectangular to Triangular Grids

Prior to building the analysispyramid, we mapthe sampleimagefrom the given rectangulararray of pixels to a pair of triangulargrids. Eachtriangulargrid is a setof rows (or columns)of congru-ent,equilateraltriangles.Thegiven sampleimagemustbe m mpixelswherem r2 K 1 , K is thedesirednumberof pyramidlev-els, andr is a positive integer. Then,the sampleimagewould bemappedto a triangulargrid with m rows. This triangulargrid willbethegeometryof thefinestresolutionlevel of thepyramid.There-fore, it needsto betheresultof K 1 applicationsof thebutterflysubdivision.

For example,considera 12 12 pixel sampleimagewith K 3. Thenthe geometryof G2 would have 12 rows of triangles,G1would have 12/2= 6 rows, andG0 would have 12/4= 3 rows. SeeFigure3.

Becausetheredoesnot exist a one-to-onemappingbetweentherectangulargrid of pixels andthe grid of equilateraltriangles,weusebilinear interpolationprior to resampling.In keepingwith ourassumptionof isotropy, we actuallyinterpolatetheimageontotwotriangulations–thesecondis a 90# rotationof thefirst. This effec-tively doublesour samplesize. Additionally, when evaluatinganeighborhood’s similarity, weconsideronly thebestmatchof three60# rotationsof theneighborhood.

4.7 Reduce and Project

With resamplingandconvolution definedfor the surface,suitabledefinitionsfor reduce andproject canbegiven. Reduceis straight-forward: convolve anddownsample.Project,however, mustapplya normalizationfactor to ensurethe correctrangeof valuesin itsresult. Recall that whena surfaceis upsampled,not all of the re-sultingchild faceswill benonzero-valued;only half of thefacesofthechild level inherit thevalueof their parent.In orderto preservetheoverallbrightnessof theinput,wecomputeanormalizationfac-tor for eachoutputface:

W "i ∑j ! Ω i S j wi j (7)

whereS j equalsunity if the face j inheritedits value from itsparent.Otherwise,S j equalszero.

Thevalueof facei afterupsamplingandconvolution is thus:

v "i ∑j ! Ω i v j $ wi j

W "i j % (8)

Gaussianpyramid Laplacianpyramid

l 3 l 3

l 2 l 2

l 1 l 1

l 0

Figure4: GaussianandLaplacianpyramidsfor color texture de-finedona triangulargrid.

For example Laplacianand Gaussianpyramids computedfor acolor texturedefinedona triangulargrid, seeFigure 4.

5 Synthesis

After building theanalysisandsynthesisgeometries,andcomput-ing theLaplacianpyramidtransformof theinput texture,GA 0 , thesynthesisprocesscanbegin. Thefirst stepof thesynthesisprocessis to initialize theresiduallevel of thesynthesisGaussianpyramid,GS 0 , with theRGB valuesof randomlychosenfacesof theresid-ual level of the analysisGaussianpyramid, GA 0 . After this ini-tialization,eachfaceis visited in randomorderandits valueis up-datedusingthenon-parametricsamplingprocedure.Whena valueis copiedfrom an analysisfaceto a synthesisface,the RGB val-uesare copiedinto GS k and the RGB valuesof the face’s fourLaplacianchildrenarecopiedinto LS k 1 . This completesthesynthesisof thelowestlevel of theGaussianpyramid. Eachsubse-quentGaussianpyramid level, GS k , is initialized by a singlestepof theinverseLaplacianpyramidtransform:

GS k '& LS k ( pro ject GS k 1) (9)

This operation,by itself, producesa goodinitial approximationofthene* xt Gaussianpyramidlevel. This initial approximationis thenupdatedby randomvisits to eachface,using the non-parametricsamplingprocedure.Thesynthesisprocessis repeateduntil thefi-nalGaussianlevel, GS K 1 is generated.This level representsthefinal texture. Pseudocodefor a somewhat simplified,onedimen-sionalmonochromaticversionof the above coarse-to-finetexturesynthesisprocessis shown in Algorithm 1.

SynthesizeTexture GA 0)begin+

LA GA , & LaplacianPyramidTrans f orm GA 0)K & log2 size GA 0)-GS 0' GA 0for k 1 to K 1 do

for update 1 to N 2k doi random 2k ΩS Neighborhood GS k . i +g l0 l1 , & GreedySample LA GA k ΩS

GS k / i '& gLS k 1/ 2i 0& l0LS k 1/ 2i 1'& l1

endGS k 1'& pro ject GS k )0 LS k 1

endreturn GS K 1

end

SeeFigure 5 for anexampleof how thetexturesynthesisprocessproceedsfrom coarse-to-finescales.

5.1 Non-parametric Sampling Procedure

A face’sk-distantneighborhoodis definedasthesetof faceswhoseminimum adjacency distanceis greaterthanzeroandlessthanorequalto k. If every vertex hasa valenceof six, thentheadjacencyrelationshipsbetweenfacesin the k-distantneighborhoodwill al-wayshavethree-foldsymmetry, andthecardinalityof theneighbor-hoodwill bedivisible by three.We constructtheseneighborhoodsasorderedsetsbasedon an adjacency relationwherea neighbor-hoodaboutfacei is definedto be a1 a2 a3 -121 ap b1 b2 b3 -112 bp c1 c2 c3 -121 cp wherea1, b1, andc1 areadjacentto (andin clockwiseorderaround)facei. Additionally, the orderingis suchthat the adjacency rela-tionshipbetweena j andak is thesameastheadjacency relationshipbetweenb j andbk andbetweenc j andck. SeeFigure5.1.

TheRGB valuesof eachfacein an orderedneighborhoodwithcardinality3p, canbethoughtof asapointin 9p dimensionalspace.A nearestneighborquery can updateface i with neighborhoodΩS i in the synthesispyramid by replacingits RGB value withthat of face j with neighborhoodΩA j in the analysispyramid,suchthatthe9p dimensionalEuclideandistancebetweenΩS i andΩA j is minimum.Unfortunately, apolicy of alwayschoosingthefacefrom theanalysispyramidwhichminimizestheerrordistance,will not, in general,preserve thefirst-orderstatisticsof thesampletexture. Themostobviousway to preserve thefirst-orderstatisticsis to usesamplingwithout replacement.Unfortunately, this wouldoverconstrainthe nearestneighborquery toward the end of eachpassthroughthesynthesispyramidlevel. Wehaveobservedthattheuseof samplingwithoutreplacementproducesspecklingin thesyn-thesizedtexture. A goodcompromisebetweenfaithful replicationof thefirst-orderstatisticsandover constrainingthenearestneigh-bor queryis to scaleeacherrordistanceby a factorof 2 c j cmin ,

Sample Initial Guess After Sampling

k 0 k 0 k 0

k 1 k 1 k 1

k 2 k 2 k 2

k 3 k 3 k 3

Figure 5: Coarse-to-finesynthesisof green marble texture onsphereusing incrementalinverseLaplacianpyramid transformtogenerateinitial approximations.

Figure 6: Neighborhoodstencil showing three-fold symmetry.Thereare12 one-distantneighborsand24 two-distantneighbors.

wherec j is thenumberof timesthatface j hasalreadybeencho-senduring

3theupdateprocess,andcmin is theleastnumberof times

thatany analysisfacehasbeenchosen.Sincewe assumeisotropic textures, we exploit the three-fold

symmetryof the neighborhood,by defining the distancebetweenΩS i andΩA j as the minimum of the error distancesbetweenΩA j andthree60# rotationsof ΩS i .

On theanalysisside,all verticeshave avalenceof six exceptforthoseontheboundary. Analysisneighborhoodsthatincludebound-ary verticesarenot includedin thenearestneighborsamplingpro-cess.On thesynthesisside,themeshesareconstructedso that thevastmajorityof verticeshaveavalenceof six. Forsimplicity, weas-sumethatall have a valenceof six. A vertex with a valencegreaterthansix, by ourmethodof neighborhoodconstruction,causesagapin theneighborhood.A vertex with a valencelessthansix causestheneighborhoodto overlapwith somefacesbeingincludedmorethan once. We ignore theseerrors. Neighborhoodson the syn-thesissidethatcontainboundaryverticesaretreatedspecially. All“missing” neighborsareconsideredas“wild cards”whichperfectlymatchany correspondingneighborin theanalysisneighborhood.

Pseudo code for a somewhat simplified, one dimensionalmonochromaticversionof theabove samplingprocedureis shownin Algorithm 2.

GreedySample LA GA k ΩS begin

c "min & MAXINTdmin MAXFLOATfor j 1 to 2k do

if c j 4 cmin thenc "min & c j

endΩA & Neighborhood GA k 5 j d & Distance ΩA ΩS 2 c j cmin if d 4 dmin then

jopt & jdmin & d

endendcmin & c "minc jopt 0& c jopt ( 1return

+GA k / jopt 5 LA k 1/ 2 jopt 5 LA k 1/ 2 jopt 1 ,

end

6 Experimental Results

All of theresultsshown in thissectionusefour levelsof resolution,with aneighborhoodsizeof 12in thelowestresolutionlevel and36in all otherlevels.Resultsfrom otherresearchersweredownloadedfrom eachnamedresearcher’s website.

7 Conc lusion

We have describeda fully automatic method of synthesizingisotropictexturesonsubdivisionsurfacesfrom sampleimages.Un-like previous approaches,texture synthesisis accomplishedin acoarse-to-finefashionby constructingboth Gaussianand Lapla-cian pyramid representationsof the synthetictexture. The inverseLaplacianpyramid transformis usedto generatefirst approxima-tions to the texture at eachlevel of the associatedGaussianpyra-mid. Theseapproximationsare refinedusing a modified nearestneighborsearchprocesswhichpreservesthefirst orderstatisticsofthesampletexture.

(a) (b)

(c) (d)

Figure 7: (a) Bullseye patternfrom Simoncelli and Portilla. (b)Synthesisresultdueto SimoncelliandPortilla. (c) Synthesisresultdueto Wei andLevoy. (d) Our result.

References

BAR-JOSEPH, Z., EL -YANIV, R., L ICHINSKI , D., AND WER-MAN, M. 2001. Texture mixing and texture movie synthesisusingstatisticallearning.IEEE Trans. on Visualization and Com-puter Graphics 7(2), 120–135.

BURT, P. J. 1981.Fastfilter transformsfor imageprocessing.TheVisual Computer 16, 20–51.

DEBONET, J. S. 1997. Multiresolution samplingprocedureforanalysisand synthesisof texture images. In SIGGRAPH 97,361–368.

EBERT, D. S., MUSGRAVE, F., PEACHEY, D., PERLIN, K., ANDWORLEY, S. 1994. Texturing & Modeling: A Procedural Ap-proach. AcademicPress.

EFROS, A., AND FREEMAN, W. 2001. Imagequilting for texturesynthesisandtransfer. In SIGGRAPH 01, 341–346.

EFROS, A., AND LEUNG, T. 1999. Texture synthesisby non-parametricsampling. In Intl. Conf. on Computer Vision, vol. 2,1033–1038.

HEEGER, D., AND BERGEN, J. R. 1995. Pyramid-basedtextureanalysis/synthesis.In SIGGRAPH 95, 229–238.

PRAUN, E., FINKLESTEIN, A., AND HOPPE, H. 2000. Lappedtextures.In SIGGRAPH 00, 465–470.

SIMONCELLI , E., AND PORTILLA , J. 1998. Texture characteri-zationvia joint statisticsof wavelet coefficient magnitudes.InFifth Intl. Conf. on Image Proc., vol. 1, 62–66.

TURK , G. 1992. Re-tiling polygonalsurfaces.In SIGGRAPH 92,55–64.

(a) (b)

(c) (d)

(e) (f)

Figure8: (a) Saltinecracker. (b) Swimmingpool bottom.(c) Syn-thesisresultdueto Wei andLevoy. (d) Synthesisresultdueto WeiandLevoy. (e) Our result.(f) Our result.

WEI , L ., AND LEVOY, M. 2000.Fasttexturesynthesisusingtree-structuredvectorquantization.In SIGGRAPH 00, 479–488.

WEI , L ., AND LEVOY, M. 2001. Texturesynthesisover arbitrarymanifoldsurfaces.In SIGGRAPH 01, 355–360.

ZORIN, D., SCHRODER, P., AND SWELDENS, W. 1996. Interpo-lating subdivision for mesheswith arbitrarytopology. In SIG-GRAPH 96, 189–192.