1. Super-Resolution Texturing for Online Virtual Globes
Internet Vision Workshop (CVPR 2008)
Super-Resolution Texturing for Online Virtual Globes
Diego Rother, Lance Williams and Guillermo Sapiro
University of Minnesota and Google, Inc.

2. Online Virtual Globes

3. Online Virtual Globes Requests Tiles Server Client User Problems:
ClipMap pyramid.
Problems:
Huge storage space required (one of the largest organized collections of imagery on the Internet).
Expensive acquisition and high transmission bandwidth.
Interpolation beyond available resolution: unnatural.

4. Earth surface Stereotypical. Rapidly changing.
Identity important, details not.

5. Proposed Solution Proposition: Requirements:
Synthesize, on the client, details for the lower pyramid levels.
Using super-resolution techniques.
Harnessing labels and textures samples from users (wiki model).
Requirements:
Fast.
Seamless transition between layers.

6. Results

7. User input 1: Labels Original frame User provided labels Class2 (path)
(grass)
Using interactive segmentation as in: Bai, X. and Sapiro, G., "A geodesic framework for fast interactive image and video segmentation and matting." ICCV, 2007.

8. User input 2: Keyframed Texture
User provides the texture pyramid:
Keyscale1
e.g., in meters/pixel
Keyframe1
Keyscale2
Keyframe2
Texture pyramid.

9. Synthesis of a New Layer
Input:
from Server
Input:
from Users
Texture pyramid.
Output:
New Layer
ClipMap pyramid
Labels

10. System Overview Inputs Outputs New Labels Labels (from server)
Pyramid of Training Textures
(from users)
Inputs
(from server)
Last Layer
Labels
Outputs
New Layer
New Labels
Interpolation
Color Matching
Texture Transfer
Undo Color Matching
Selection of the Training Image

11. Texture transfer: 1st pass
Training Texture (from the texture pyramid)
Color matched image, without high frequencies
Y Channel
(luminance)
I and Q Channels (chrominance)
Mean and Gradient
Only Mean
Similarity between contexts considers
Source Locations
Produces 1-Pixel Patches:
Contiguous areas copied verbatim from the training texture.
Small contexts → Fast
Wei, L. and Levoy, M., "Fast Texture Synthesis using Tree-structured Vector Quantization." SIGGRAPH, 2000.
Efros, A. A. and Leung, T. K., "Texture Synthesis by Non-parametric Sampling." ICCV, 1999.

12. Texture transfer: 2nd, 3rd and 4th passes
Color
Channel
Pass
2nd
3rd and 4th
Y Channel
(luminance)
I and Q Channels (chrominance)
Similarity between contexts considers
Training Texture
Produces Bigger Patches
Few candidates → Fast
Ashikhmin, M., "Synthesizing Natural Textures." ACM Symposium on Interactive 3D Graphics

13. Texture transfer from the same texture
1st synthetic frame
2nd synthetic frame
ClipMap Pyramid
Texture Pyramid
Patch interior (lilac and violet) directly copied.
Patch boundaries (pink) synthesized in 4 passes as before.
Doubles the patch size.

14. No texture transferred
Results: Maracanã
ClipMap pyramid (result)
Texture pyramid (user input)
No texture transferred

15. Results: Maracana ClipMap pyramid (result) Source Locations (result)
No texture transferred
No texture transferred
No texture transferred
No texture transferred

16. Results: Field ClipMap pyramid (result) Texture pyramid (user input)
No texture transferred
Source Locations (result)
No texture transferred

17. Results: Beach
ClipMap pyramid (result)

18. Conclusions Proposed solution:
Reduces storage, bandwidth, and acquisition costs.
Improves appearance and information content.
Is fast (low dimensional contexts).