1. Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics Workshop on Rendering (2002)

2. Texture Synthesis

3. Neighbourhood-based Compare local causal neighbourhoods Efros and Leung (ICCV ’99) Wei and Levoy (SIGGRAPH 2000) Ashikhmin (I3D 2001) InputOutput

4. Fast Texture Synthesis Goal: Interactivity Want synthesis that is: Fast Simple High quality

5. Patch-based Methods Copy patches of pixels rather than single pixels

6. Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997

7. Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al, 2002

8. Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al, 2002 Image Quilting, Efros and Williams, 2001

9. Video Textures Schodl et al, SIGGRAPH 2000 Given a sample video, generate endless video without looping Generate links between similar frames Play video, randomly following links Our Inspiration..

10. Our Approach Divide task into two phases: Analysis Once per input texture (need not be fast) Generates jump map Synthesis Uses jump map Fast enough for interactive applications

11. What is a Jump Map? Same size as input

12. What is a Jump Map? Same size as input Set of jumps per pixel

13. What is a Jump Map? Same size as input Set of jumps per pixel Jumps are weighted according to similarity Need not sum to 1 0.4 0.3 0.2

14. Jump Map Texture Synthesis Synthesis becomes a random walk Pixel-by-pixel, in scan-line order Select an already-synthesized neighbour N Select a destination D from N’s jumps Copy the pixel neighbouring D

15. Output Synthesis with Jump Maps Input

16. Output Synthesis With Jump Maps Input

17. Synthesis Order Synthesis order influences patch shapes Not likely to extend in directions where there aren’t already- synthesized neighbours

18. Synthesis Orders Serpentine Reverse direction at end of scan-line Better than scan-line, just as fast Hilbert curve Maximizes locality Much higher quality Adds some overhead

19. Synthesis Issues Artifacts may occur if a patch hits an input image boundary Modify probability of taking a jump Increase for jumps from input boundary Decrease for jumps to input boundary Blend patch boundaries

20. Texture Analysis Need best matches for each input pixel Pose as high-dimensional ANN problem Input … Neighbourhood Vectors … ANN Vectors PCA

21. Multi-resolution Analysis Use image pyramid and multi-resolution neighbourhood vectors Smaller neighbourhood required Improves PCA reduction

22. Jump Map Diversity Undesirable repitition may occur if jumps cluster spatially L 2 norm is particularly susceptible

23. 1 5 6 4 3 2 Poisson Disc Sampling Find extra matches Iteratively accept matches satisfying Poisson disc criterion Include a Poisson disc at the source 1 4 3

24. Analysis Summary Need best matches for each input pixel Use multi-resolution neighbourhoods Pose as high-dimensional ANN problem Reduce dimension with PCA Filter matches with Poisson discs Normalize similarity values across the jump map

25. Results Current implementation: 2.1 million pixels/second scan-line 0.8 million pixels/second Hilbert Good quality on stochastic textures Not so good on structured textures TBD: demo

26. Future Work Analysis phase: Use perceptual metrics Clustering instead of ANN Synthesis phase Multi-resolution synthesis Output control mechanisms

27. Future Work Generalization to patches Reduce storage used Sample size required?

28. Contact Information Steve Zelinka zelinka@uiuc.edu Michael Garland garland@uiuc.edu http://graphics.cs.uiuc.edu/~zelinka/jumpmaps