Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics Workshop on Rendering (2002)

Texture Synthesis

Neighbourhood-based Compare local causal neighbourhoods Efros and Leung (ICCV ’99) Wei and Levoy (SIGGRAPH 2000) Ashikhmin (I3D 2001) InputOutput

Fast Texture Synthesis Goal: Interactivity Want synthesis that is: Fast Simple High quality

Patch-based Methods Copy patches of pixels rather than single pixels

Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997

Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al, 2002

Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al, 2002 Image Quilting, Efros and Williams, 2001

Video Textures Schodl et al, SIGGRAPH 2000 Given a sample video, generate endless video without looping Generate links between similar frames Play video, randomly following links Our Inspiration..

Our Approach Divide task into two phases: Analysis Once per input texture (need not be fast) Generates jump map Synthesis Uses jump map Fast enough for interactive applications

What is a Jump Map? Same size as input

What is a Jump Map? Same size as input Set of jumps per pixel

What is a Jump Map? Same size as input Set of jumps per pixel Jumps are weighted according to similarity Need not sum to

Jump Map Texture Synthesis Synthesis becomes a random walk Pixel-by-pixel, in scan-line order Select an already-synthesized neighbour N Select a destination D from N’s jumps Copy the pixel neighbouring D

Output Synthesis with Jump Maps Input

Output Synthesis With Jump Maps Input

Synthesis Order Synthesis order influences patch shapes Not likely to extend in directions where there aren’t already- synthesized neighbours

Synthesis Orders Serpentine Reverse direction at end of scan-line Better than scan-line, just as fast Hilbert curve Maximizes locality Much higher quality Adds some overhead

Synthesis Issues Artifacts may occur if a patch hits an input image boundary Modify probability of taking a jump Increase for jumps from input boundary Decrease for jumps to input boundary Blend patch boundaries

Texture Analysis Need best matches for each input pixel Pose as high-dimensional ANN problem Input … Neighbourhood Vectors … ANN Vectors PCA

Multi-resolution Analysis Use image pyramid and multi-resolution neighbourhood vectors Smaller neighbourhood required Improves PCA reduction

Jump Map Diversity Undesirable repitition may occur if jumps cluster spatially L 2 norm is particularly susceptible

Poisson Disc Sampling Find extra matches Iteratively accept matches satisfying Poisson disc criterion Include a Poisson disc at the source 1 4 3

Analysis Summary Need best matches for each input pixel Use multi-resolution neighbourhoods Pose as high-dimensional ANN problem Reduce dimension with PCA Filter matches with Poisson discs Normalize similarity values across the jump map

Results Current implementation: 2.1 million pixels/second scan-line 0.8 million pixels/second Hilbert Good quality on stochastic textures Not so good on structured textures TBD: demo

Future Work Analysis phase: Use perceptual metrics Clustering instead of ANN Synthesis phase Multi-resolution synthesis Output control mechanisms

Future Work Generalization to patches Reduce storage used Sample size required?

Contact Information Steve Zelinka Michael Garland