View
216
Download
0
Category
Preview:
Citation preview
Texture Synthesis from Multiple Sources
Texture Synthesis from Multiple Sources
Li-Yi WeiStanford University (was) NVIDIA Corporation (now)
Texture Synthesis(Single-Source) Texture Synthesis(Single-Source)
Input
Output
Synthesis
Tong et. al. 2002Soler et. al. 2002
Hertzmann et. al. 2001Efros & Freeman 2001
Yin et. al. 2001Turk 2001
Wei & Levoy 2001 Wei & Levoy 2000
Portilla & Simoncelli 1999Efros & Leung 1999
Heeger & Bergen 1995De Bonet 1997
Limitations of Single-Source SynthesisLimitations of Single-Source Synthesis
• Non-uniform, varying patterns– Junction of 2 textures– Varying scale, orientation, color, shape– Creating new textures
• Textures of different dimensions– Solid textures from 2D views
Multiple-Source Texture SynthesisMultiple-Source Texture Synthesis
• Solid textures from multiple 2D views
View 3
View 2
View 1
• Texture mixtures from multiple sources
+ = ?Source 1 Source 2
Previous WorkPrevious Work
• Solid texture from 2D views– Heeger & Bergen 1995– Ghazanfarpour & Dischler 1999
Images from [Heeger&Bergen 1995]
2D Source 3D result
Previous WorkPrevious Work
• Texture varying, morphing, and mixture– Portilla & Simoncelli 1999– Bar-Joseph, El-Yaniv, Lichinski, Werman 2001– Z. Liu, C. Liu, Shum, Yu 2003– PVT paper in SIGGRAPH 2003
Image from [Portilla&Simoncelli 1999]
Source 1 Source 2morphing
AlgorithmAlgorithm
Source
Synthesis Result
N(p)
search
N(pi)
Single-source algorithm [ICCV 99, SIGGRAPH 2000]
copy p
AlgorithmAlgorithm
Source 1
Source 2
Source 3
Synthesis Result
search
search
search
N1(p1)
N2(p2)
N3(p3)
E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2)i= 1,2,3
L2 distance
Fix p, search {pi}Fix {pi}, set p = Σ wi×pi
User weights
Average
AlgorithmAlgorithm
Source 1
Source 2
Source 3
Synthesis Result
search
search
search
E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2)i= 1,2,3
L2 distance
Fix p, search {pi}Fix {pi}, set p = Σ wi×pi
User weights
Average
Texture Mixture ExampleTexture Mixture Example
+ = Uniform
Transition
Use weights wi to control the result
Source 1 Source 2
Texture Mixture ResultsTexture Mixture Results
Source1 Source2 Mixture-uniform Mixture-transition
Texture Mixture ResultsTexture Mixture Results
Source1 Source2 Mixture-uniform Mixture-transition
Solid Texture SynthesisSolid Texture Synthesis
View 3View 2
View 1
Use {Ni} with different orientationsN2
N3
N1
Specifying ViewsSpecifying Views
Input
Views Result
×Inconsistent
О
О
Solid Texture ResultsSolid Texture Results
Solid Texture ResultsSolid Texture Results
Comparison : Surface/Volume SynthesisComparison : Surface/Volume Synthesis
Surface[SIGGRAPH 2001]
Volume[this work]
ConclusionConclusion
• Single-source synthesis– mostly mature and well done– limited capability
• Multiple-source synthesis– hard (patch-copying won’t work!)
– more useful– requires more work!
Future WorkFuture Work
• Combine the texton mask idea in SIGGRAPH 2003 PVT paper with this algorithm?
ENDEND
AlgorithmAlgorithm
1. Fix p, search {pi} to minimize |Ni(p)-Ni(pi)|2
2. Fix {pi}, set p = Σ wi×pi
Goal : Minimize E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2)Step 1Step 2
How :
iterate
ComparisonComparison
Reusable Distortion Quality Efficiency Tunable
Surface better better better
Volume better better
Recommended