Texture Synthesis from Multiple Sources Li-Yi Wei Stanford University (was) NVIDIA Corporation (now)

Texture Synthesis (Single-Source) Input Output Synthesis Tong et. al Soler et. al Hertzmann et. al Efros & Freeman 2001 Yin et. al Turk 2001 Wei & Levoy 2001 Wei & Levoy 2000 Portilla & Simoncelli 1999 Efros & Leung 1999 Heeger & Bergen 1995 De Bonet 1997

Limitations of Single-Source Synthesis Non-uniform, varying patterns –Junction of 2 textures –Varying scale, orientation, color, shape –Creating new textures Textures of different dimensions –Solid textures from 2D views

Multiple-Source Texture Synthesis Solid textures from multiple 2D views View 3 View 2 View 1 Texture mixtures from multiple sources += ? Source 1Source 2

Previous Work Solid texture from 2D views –Heeger & Bergen 1995 –Ghazanfarpour & Dischler 1999 Images from [Heeger&Bergen 1995] 2D Source3D result

Previous Work Texture varying, morphing, and mixture –Portilla & Simoncelli 1999 –Bar-Joseph, El-Yaniv, Lichinski, Werman 2001 –Z. Liu, C. Liu, Shum, Yu 2003 –PVT paper in SIGGRAPH 2003 Image from [Portilla&Simoncelli 1999] Source 1Source 2morphing

Algorithm Source Synthesis Result N(p) search N(p i ) Single-source algorithm [ICCV 99, SIGGRAPH 2000] copy p

Algorithm Source 1 Source 2 Source 3 Synthesis Result search N 1 (p 1 ) N 2 (p 2 ) N 3 (p 3 ) E(p, {p i }) = Σ w i × ( |p-p i | 2 + |N i (p)-N i (p i )| 2 ) i= 1,2,3 L2 distance Fix p, search {p i } Fix {p i }, set p = Σ w i ×p i User weights Average

Algorithm Source 1 Source 2 Source 3 Synthesis Result search E(p, {p i }) = Σ w i × ( |p-p i | 2 + |N i (p)-N i (p i )| 2 ) i= 1,2,3 L2 distance Fix p, search {p i } Fix {p i }, set p = Σ w i ×p i User weights Average

Texture Mixture Example += Uniform Transition Use weights w i to control the result Source 1Source 2

Texture Mixture Results Source1Source2Mixture-uniformMixture-transition

Texture Mixture Results Source1Source2Mixture-uniformMixture-transition

Solid Texture Synthesis View 3 View 2 View 1 Use {N i } with different orientations N2N2 N3N3 N1N1

Specifying Views Input ViewsResult × Inconsistent О О

Solid Texture Results

Comparison : Surface/Volume Synthesis Surface [SIGGRAPH 2001] Volume [this work]

Conclusion Single-source synthesis –mostly mature and well done –limited capability Multiple-source synthesis –hard (patch-copying won’t work!) –more useful –requires more work!

Future Work Combine the texton mask idea in SIGGRAPH 2003 PVT paper with this algorithm?

END

Algorithm 1.Fix p, search {p i } to minimize |N i (p)-N i (p i )| 2 2.Fix {p i }, set p = Σ w i ×p i Goal : Minimize E(p, {p i }) = Σ w i × ( |p-p i | 2 + |N i (p)-N i (p i )| 2 ) Step 1Step 2 How : iterate

Comparison ReusableDistortionQualityEfficiencyTunable Surfacebetter Volumebetter

S2003 Verdana, Bold, 37 points This subtitle is 31 points Bullets are orange; text is 26 points They have 110% line spacing, 6 points before/after Longer bullets in the form of a paragraph are harder to read if there is insufficient line spacing. This is the maximum recommended number of lines per slide (seven). –Sub-bullets look like this. This subtitle is 31 points Bullets are orange; text is 26 points They have 110% line spacing, 6 points before/after Longer bullets in the form of a paragraph are harder to read if there is insufficient line spacing. This is the maximum recommended number of lines per slide (seven). –Sub-bullets look like this.