Fast Texture Synthesis using Tree-structured Vector Quantization

Fast Texture Synthesis using Fast Texture Synthesis using Tree-structured Vector Tree-structured Vector

QuantizationQuantization

Li-Yi Wei Marc Levoy

Computer Graphics GroupStanford University

IntroductionIntroduction

Texture Synthesis

Result

Desirable PropertiesDesirable Properties

• Result looks like the input

• Efficient

• General

• Easy to use

• Extensible

Previous WorkPrevious Work

• Procedural Synthesis– Perlin 85, Witkin 91, Worley 96

• Statistical Feature Matching– Heeger 95, De Bonet 97, Simoncelli 98

• Markov Random Fields– Popat 93, Efros 99

OutlineOutline

• Basic algorithm

• Multi-resolution algorithm

• Acceleration

• Applications

Texture ModelTexture Model

• Textures are– local– stationary

• Model textures by – local spatial neighborhoods

Basic AlgorithmBasic Algorithm

• Exhaustively search neighborhoods

NeighborhoodNeighborhood

• Use causal neighborhoods

Causal Non-causal

NeighborhoodNeighborhood

• Neighborhood size determines the quality & cost

33 55 77

99 1111 4141

423 s 528 s 739 s

1020 s 1445 s 24350 s

Multi-resolution PyramidMulti-resolution Pyramid

High resolution Low resolution

Multi-resolution Multi-resolution AlgorithmAlgorithm

BenefitBenefit

• Better image quality & faster computation

1 level55

3 levels55

1 level1111

ResultsResults

Random Oriented

Regular Semi-regular

FailuresFailures

• Non-planar structures

• Global information

ComparisonComparison

Heeger 95 De Bonet 97 Efros 99 Our method

1941 secs 503 secs

12 secs

Acceleration Acceleration

• Computation bottleneck: neighborhood search

Nearest Point SearchNearest Point Search

• Treat neighborhoods as high dimensional points

1 2 3 4 5

6 7 8 9 10

Neighborhood

1 2 3 4 5 6 7 8 9 10 11 12

High dimensional point/vector

AccelerationAcceleration

• Nearest point search in high dimensions– [Nene 97]

• Cluster-based model for textures– [Popat 93]

• Tree-structured Vector Quantization– [Gersho 92]

Tree-structured Tree-structured Vector QuantizationVector Quantization

TimingTiming

• Time complexity : O(log N) instead of O(N)– 2 orders of magnitude speedup for non-trivial images

1941 secs 503 secs 12 secs

Efros 99 Full searching TSVQ

Results: Results: Brodatz TexturesBrodatz Textures

Input Exhaustive: 360 secs TSVQ: 7.5 secs

Application 1: Application 1: Constrained SynthesisConstrained Synthesis

Possible SolutionPossible Solution

• Multi-resolution blending [Burt & Adelson 83]– produce visible boundaries

• Original raster-scan algorithm– discontinuities at right and bottom boundaries

• Adaptive neighborhoods [Efros 99]– Hard to accelerate

Modifications Modifications

• Need to use a single symmetric neighborhood

• 2 pass algorithm with extrapolation

• Spiral order synthesis

ResultResult

• Extrapolation

ResultResult

• Image editing by texture replacement

Application 2:Application 2:Temporal TextureTemporal Texture

• Indeterminate motions both in space and time– fire, smoke, ocean waves

• How to synthesize?– extend our 2D algorithm to 3D

Temporal TextureTemporal Texture

Fire Smoke Waves

Result

Future WorkFuture Work

• More general “textures”– light fields, solid textures– motion signals– displacement maps

• Real time texture synthesis

AcknowledgmentAcknowledgment

• Kris Popat• Alyosha Efros• Stanford Graphics Group• Intel, Interval, Sony

More information

http://graphics.stanford.edu/projects/texture/

Fast Texture Synthesis using Tree-structured Vector Quantization

Documents

Scalar quantization with memory...Scalar quantization with memory Speech and images have redundancy or memory that scalar quantization can not exploit. Scalar quantization for source

Subsurface Texture Mapping - COnnecting REpositories · 2016-12-27 · Subsurface Texture Mapping 5 This paper is structured as follows. Section 1.1 summarizes the related works and

07.19.Fast Texture Synthesis Using Tree-Structured Vector Quantization

Fast Texture Synthesis using Tree-structured Vector ...gopi/ICS280Win02/WeiLevoy.pdf · Fast Texture Synthesis using Tree-structured Vector Quantization Li-Yi Wei Marc Levoy Stanford

Color Quantization

Sampling & Quantization

Vector Quantization. 2 outline Introduction Two measurement : quality of image and bit rate Advantages of Vector Quantization over Scalar Quantization

Quantization Noise

An Analysis of Co-occurrence Texture Statistics as a Function of Grey Level Quantization

Contents - fmt.if.usp.brgtlandi/courses/second-quantization-4.pdf · quantization. Second quantization is extremely powerful. We will learn in these notes to think about second quantization

OpenGL Texturing. Content Texture target Texture environment Texture coordinate Texture parameter (filters, wrap) Texture object Texture transformation

Quantization and Compressive Sensing · interaction of quantization and compressive sensing and examines practical quanti- ... The interaction of quantization with sampling introduces

Vector Quantization in Speech Coding - LabROSA - …labrosa.ee.columbia.edu/~dpwe/papers/MakhRG85-vq.pdf · Vector Quantization in Speech Coding ... quantization. Sampling converts

Quantization table

QUANTIZATION DESIGN FOR STRUCTURED OVERCOMPLETE …sipi.usc.edu/~ortega/PhDTheses/BaltasarBeferull-Lozano.pdf · 2004-06-04 · and the power of the work well done and I have followed

Separating Texture and Illumination for Single-Shot ...ILIM/projects/IL/TextIllumSep/papers/CCD14.pdf · Separating Texture and Illumination for Single-Shot Structured Light Reconstruction

Quantization, after Souriau...Quantization, after Souriau Souriau Prequantization Quantization? Group algebra Classical Quantum Nilpotent Reductive E(3) J.-M. Souriau What is quantization?

Fast Texture Synthesis using Tree-structuredVector ...Fast Texture Synthesis using Tree-structuredVector Quantization Li-Yi Wei Marc Levoy Stanford University Figure 1: Our texture

Second Quantization

Digital Filter Structures and Quantization Error Analysis1.1... · Digital Filter Structures and Quantization Error Analysis By: ... Digital Filter Structures and Quantization