Synthesis of Progressively-Variant Textures on Arbi-

trary SurfacesSIGGRAPH 2003

Jingdan Zhang, Kun Zhou, Luiz Velho, Baining Guo, Heung-Yeung Shum

We present an approach for decorating surfaces with progres-sively variant textures◦ Can model local texture variations

Scale, orientation, color, shape variation

For 2D texture modeling, our feature-based warping tech-nique allows the user to control the shape variations of tex-ture elements

Our feature-based blending technique can create a smooth transition between two given homogeneous textures

We propose an algorithm based on texton masks◦ To prevent texture elements breaking apart as they progressively

vary

Introduction

Most of the previous work on surface texture synthesis concentrated on homogeneous tex-tures◦ However, many textures,

including the coating pat-terns of various animals such as the tiger, cannot be described by station-ary stochastic models

◦ Intuitively, their texture elements change in a progressive fashion

Introduction

Exemplar-based surface texture synthesis

◦ Gorla et al. 2001, Turk 2001, Wei and Levoy 2001, Ying et al. 2001, Dischler et al. 2002

◦ Only for homogeneous texture synthesis

Related work

Reaction-diffusion textures◦ Procedural◦ Turk [1991], Witkin

and Kass [1991]

◦ Parameter tweaking affects the result heavily

◦ Only a few kind of materials can be syn-thesized

Related work

Integrating Shape and Pat-tern in Mam-malian Models

◦ Walter et al. SIG-GRAPH 2001

◦ By biological simula-tion

Related work

Sequential Pixel-based Garber 1981, Popat & Picard 1993,

Efros & Leung 1999, Wei & Levoy 2000, Ashikhmin 2001, Hertzmann et al 2001, Tong et al 2002 …

Exemplar Synthesized

We represent a progressively-variant texture by a tuple (T,M,F,V)◦ Texture image T◦ Texton mask M

Marks which type of texture elements pixel p belongs to

◦ Transition function F Scalar function whose gradient determines how fast

the texture T is changing◦ Orientation field V

A unit vector field

Overview

A progressively-variant 2D texture can be created by our field distortion or feature-based techniques

The field distortion algorithm generates a texture by scaling and rotating the local coordinate frame at each pixel◦ Using F, V

The feature-based techniques create texton masks first first, which then guide the synthesis of the target textures◦ Feature-based warping & blending

Overview

To synthesize a progressively-variant texture on a mesh, we start with a 2D progressively-variant tex-ture sample (To,Mo,Fo,Vo)◦ User needs to specify Fs and Vs over the target mesh

The synthesis algorithm controls the scale and orien-tation variation of texture elements by matching Fs and Vs with their 2D counterparts

Our algorithm synthesizes a texton mask Ms in con-junction with the target texture Ts and uses Ms to pre-vent the breaking of texture elements

Overview

Overview

Synthesizes a progressively-variant texture To

User specifies scale and orientation vectors at a few locations◦ Interpolates these “key” scales and orientations to generate

the entire Fo and Vo by using radial basis functions

Extends [Wei and Levoy 2000] by incorporating scale and orientation variations controlled Fo and Vo

◦ Pyramid-based sequential neighborhood matching algorithm

Field Distortion Synthesis

Fo and Vo control the target texture through the construction of the neighborhood N(p)◦ N(p) is scaled using Fo(p) and rotated using Vo(p)◦ Pixels in N(p) is resampled from To using bilinear

interpolation Does not consider pixel coverage

The synthesis order has a large effect on the synthesis quality

Field Distortion Synthesis

Field Distortion Synthesis

To apply feature-based techniques, the user must specify a texton mask on a given texture

Our user interface is based on color thresholding◦ The user picks one or two pixel colors◦ Provide dilation and erosion for refining texton masks

Our experiences suggest that a texton mask indicating one or two types of the most prominent texture elements is sufficient

Work well for most textures◦ More sophisticated segmentation methods can be used to gener-

ate better texton masks

Texton Mask Specification

With input Texture Ti and texton mask Mi

◦ Produce new mask Mo

Use Fo to control the parameters in the editing opera-tions

Our system synthesizes a progressively-variant texture To using two texton masks, Mi and Mo, and known texture Ti

◦ As an application of image analogies [Hertz-mann et al. 2001]

◦ Refer to the step as ‘Texton mask filtering’

Feature-Based Warping

Feature-Based Warping

Feature-Based Warping

All masks used in this paper have fewer than four colors and usually the mask is bi-nary◦ Can easily apply morphological operations such

as dilation, erosion

Can also apply image warping techniques such as mesh warping, field warping, and warping using radial basis functions◦ Require feature points and feature lines

Feature-Based Warping

Takes two homogeneous textures T0 and T1 and gen-erates a progressively-variant texture Tb

◦ We assume T0, T1, and Tb are all of the same size and are defined on a unit square

◦ Also use simple linear transition function and texton mask M0, M1

Fb(x, y) = x

Tb can be obtained by color blending T0` and T1`◦ T0` and T1` can be obtained by synthesizing T0 and T1 using

Mb

◦ T0` and T1` have their features aligned thus does not cause ghosting when color blended

Feature-Based Blending

The key to generating T0` and T1` is the construction of Mb

We want Mb(x, y) to be like M0 when x ≈ 0 and like M1 when x ≈ 1

◦ M(x, y) = xM1(x, y) + (1−x)M0(x, y)

◦ Gaussian blur M(x, y) to reduce discontinuity◦ Convert M(x, y) to Mb using user provided thres-

hold

Feature-Based Blending

Feature-Based Blending

With (To,Mo,Fo,Vo), synthesize Ts over the mesh surface◦ User needs to specify Vs and Fs at some key loca-

tions◦ Interpolates over the entire surface

Standard L2-norm is a poor perceptual measure for neighborhood similarity◦ Synthesis without texton mask may break apart

texture elements

Surface Texture Synthesis

Our algorithm synthesizes a texton mask Ms in conjunction with the texture Ts

◦ Texton masks are resistant to damage caused by deficiencies in the L2-norm

Surface Texture Synthesis

Surface Texture Synthesis

Candidate pool C(v,ε) is constructed for each vertex v in mesh◦ A candidate pixel p from To must satisfy a condi-

tion |Fo(p)−Fs(v)| < ε, ε = 0.1

Neighborhood Nm(v) and Nc(v) is in the tan-gent plane of the surface at v, with same orientation as Vs(v)◦ Nm(p) and Nc(p) is from To, with same orientation

as Vo(p)

Surface Texture Synthesis

We use larger neighborhoods when searching for tex-ton mask value, while smaller for color value◦ Texton masks determine the layout of texture elements

whereas the synthesis of pixel colors is simply a step to fill in the details

Nc(p) should really be Nc(p, s) where s = Fo(p) is the scale at p◦ Nc(p, smin) be the smallest neighborhood and Nc(p, smax) be the

largest◦ We determine the size of Nc(p, s) by linearly interpolating be-

tween that of Nc(p, smin) and Nc(p, smax) and rounding the re-sult up to the nearest integer

◦ Applies to all type of neighborhoods

Surface Texture Synthesis

We populate C(v,ε) using k-coherence tech-nique◦ With an additional check for the transition func-

tion condition

We pre-compute k-nearest neighbors for each pixels◦ We use k = 20

Surface Texture Synthesis

Surface Texture Synthesis

An alternative approach to handle transition functions is to put the function values in the alpha channel◦ However, this may not satisfy the condition from

equ. 1

We need a orientation field for input texture as well

Surface Texture Synthesis

Texton masks are also useful for homoge-neous texture synthesis◦ Previous methods would break some texture ele-

ments due to insufficient texture measurement

Discussion

Results

Although color thresholding may not always generate a good segmentation in the tradi-tional sense, the resulting texton masks are usually good enough

We hand painted a texton mask when color thresholding fails◦ Our algorithm was still able to produce good re-

sults

Results

Our main contribution in this paper is a framework for pro-gressively variant textures on arbitrary surfaces◦ Feature-based warping and blending◦ The general framework we propose should be applicable to most tex-

tures

One area of future work is to add more user control to feature based blending◦ User may want more control over the way texture changes

Another topic is to explore the multi-way transition among more than two textures

Finally, we are interested in other ways to control the local variations of textures

Conclusion