Pattern-based Texture Metamorphosis

Pattern-based Pattern-based Texture Texture

MetamorphosisMetamorphosis

Z. Liu, C. Liu, and H. Z. Liu, C. Liu, and H. ShumShum

Microsoft Research AsiaMicrosoft Research Asia

Y. Y. YuYuUIUC

Image Morphing vs. Image Morphing vs. Texture MorphingTexture Morphing

Specify Features and Specify Features and Correspondence *Correspondence *

Warp GenerationWarp Generation Transition ControlTransition Control

Image Morphing

* Require a lot of human intervention

Textures are usually Textures are usually homogenous with homogenous with features everywhere.features everywhere. Hard to specify featuresHard to specify features Hard to build Hard to build

correspondencecorrespondence

Image Morphing vs. Image Morphing vs. Texture MorphingTexture Morphing

Texture Morphing

Direct Blending Does Not Direct Blending Does Not WorkWork

Random Semi-structured Regular

source

target

Interesting Problems Interesting Problems In Texture MorphingIn Texture Morphing

What pair of textures?What pair of textures? Similar and repeatable patterns.Similar and repeatable patterns. Pattern distributions are alike.Pattern distributions are alike.

What is the feature?What is the feature? User define pattern.User define pattern.

How to extract so many patterns?How to extract so many patterns? Semi-automatic approach.Semi-automatic approach.

How to build correspondence?How to build correspondence? Generate a smooth warp field.Generate a smooth warp field.

Our ApproachOur Approach 1. Pattern 1. Pattern

Detection and Detection and AlignmentAlignment

2. Establishing 2. Establishing Correspondence Correspondence

3. Warping and 3. Warping and BlendingBlending

Source texture Target texture

Morphing sequence

Pattern RepresentationPattern Representation

Shape DistanceShape Distance

Local Feature DistanceLocal Feature Distance

Pattern Representation & Pattern Representation & Distance MeasurementDistance Measurement

mYYCU ,, 1

C

1Y2Y

mY

3Y

4Y

5Y6Y

M

iiiUUd f

1

2),( FF

M

ii

M

ii

Ti YYY

1

2

1

M

iii YYUUds

1

2min),(

Pattern Detection & Pattern Detection & AlignmentAlignment

Step1: Initialization by Step1: Initialization by Generalized Hough Generalized Hough Transform (GHT).Transform (GHT).

Step2: Alignment by top-Step2: Alignment by top-down verification.down verification.

Step3: Refinement by human Step3: Refinement by human intervention.intervention.

Step1: InitializationStep1: Initialization

12

3

4

5 6

m

Pattern Detection & Pattern Detection & AlignmentAlignment

C

1Y2Y

mY

3Y

4Y

5Y6Y

User selected patternVoting of a pixel

Intensity image Local maximum

Original texture

(a) Independently update each (a) Independently update each landmarklandmark

(b) Update shape(b) Update shape

Iteratively do (a) and (b).Iteratively do (a) and (b).

Pattern Detection & Pattern Detection & AlignmentAlignmentStep2: AlignmentStep2: Alignment

Alignment Process

GHT initialization alignment alignment

Pattern Detection & Pattern Detection & AlignmentAlignmentStep3: RefinementStep3: Refinement

(a) False detection(a) False detection

(b) False alignment(b) False alignment

(c) More than one types of (c) More than one types of patternpattern

Correspondence by Correspondence by Minimizing Morphing PathMinimizing Morphing Path

8 56 14 21 41 36

45

51

4

0

0

21

3

45

0

0

45

23

7

0

0

24

44

47

0

0

14 7

80 85

21 7

0 0 0 0 0 0

0 0

0 0

1C

1C 2C 3C 4C nC 1nC

2C

3C

4C

1nC

nC

nCC 1 nCC 1

n

iki

n

ii iCCk

1

2

1 minarg

Warping and BlendingWarping and Blending

Forward, inverse and synthesized sequences acquiredbased on the warp field

blend

From S.Lee

More ResultsMore Results

source

target

Pattern selected

DiscussionDiscussion About Pattern SelectionAbout Pattern Selection

Can be any shapeCan be any shape User is responsibleUser is responsible

About Correspondence About Correspondence and Transition Controland Transition Control Problem of crowd patterns Problem of crowd patterns

About Warp GenerationAbout Warp Generation MFFD vs. “as rigid as possible”MFFD vs. “as rigid as possible”

Thank you !Thank you !