Fast Texture Synthesis using Tree-structured Vector Quantization Li-Yi Wei Marc Levoy Computer Graphics Group Stanford University

Introduction Texture Synthesis Input Result

Desirable Properties Result looks like the input Efficient General Easy to use Extensible

Previous Work Procedural Synthesis –Perlin 85, Witkin 91, Worley 96 Statistical Feature Matching –Heeger 95, De Bonet 97, Simoncelli 98 Markov Random Fields –Popat 93, Efros 99

Outline Basic algorithm Multi-resolution algorithm Acceleration Applications

Texture Model Textures are –local –stationary Model textures by –local spatial neighborhoods

Basic Algorithm Exhaustively search neighborhoods

Neighborhood Use causal neighborhoods CausalNon-causal Input Noise

Neighborhood Neighborhood size determines the quality & cost 333355557777 9999 11  1141  s528 s 739 s 1020 s1445 s s

Multi-resolution Pyramid High resolutionLow resolution

Multi-resolution Algorithm

Benefit Better image quality & faster computation 1 level 5  5 3 levels 5  5 1 level 11  11

Results Random Oriented RegularSemi-regular

Failures Non-planar structures Global information

Comparison Heeger 95De Bonet 97Efros 99Our method Input 1941 secs 503 secs 12 secs

Acceleration Computation bottleneck: neighborhood search

Nearest Point Search Treat neighborhoods as high dimensional points Neighborhood High dimensional point/vector

Acceleration Nearest point search in high dimensions –[Nene 97] Cluster-based model for textures –[Popat 93] Tree-structured Vector Quantization –[Gersho 92]

Tree-structured Vector Quantization

Timing Time complexity : O(log N) instead of O(N) –2 orders of magnitude speedup for non-trivial images 1941 secs503 secs12 secs Efros 99Full searchingTSVQ

Results: Brodatz Textures InputExhaustive: 360 secsTSVQ: 7.5 secs D103 D20

Application 1: Constrained Synthesis ?

Possible Solution Multi-resolution blending [Burt & Adelson 83] –produce visible boundaries

Possible Solution Original raster-scan algorithm –discontinuities at right and bottom boundaries

Possible Solution Adaptive neighborhoods [Efros 99] –Hard to accelerate

Modifications Need to use a single symmetric neighborhood 2 pass algorithm with extrapolation Spiral order synthesis

Result

Result Extrapolation ? ? ? ?

Result Image editing by texture replacement

Application 2: Temporal Texture Indeterminate motions both in space and time –fire, smoke, ocean waves How to synthesize? –extend our 2D algorithm to 3D

Temporal Texture FireSmokeWaves Input Result

Future Work More general “textures” –light fields, solid textures –motion signals –displacement maps Real time texture synthesis

Acknowledgment Kris Popat Alyosha Efros Stanford Graphics Group Intel, Interval, Sony More information