1. Image-Based Rendering CS 446: Real-Time Rendering & Game Technology David Luebke University of Virginia

2. 2 David Luebke Real-Time Rendering Demo Today: John Dimeo Thursday: Meng Tan

3. 3 David Luebke Real-Time Rendering Image-Based Rendering You’ve been learning how to turn geometric models into images –Specifically, images of compelling 3D objects and worlds Image-based rendering : a relatively new field of computer graphics devoted to making images from images Ex: Quicktime VR

4. 4 David Luebke Real-Time Rendering Quicktime VR

5. 5 David Luebke Real-Time Rendering Images with depth Quicktime VR is really just a 2D panoramic photograph –Spin around, zoom in and out But what if we could assign depth to parts of the image? Ex: Tour Into the Picture

6. 6 David Luebke Real-Time Rendering Tour Into the Picture

7. 7 David Luebke Real-Time Rendering Tour Into the Picture Software for: –Selecting parts of an image –Assigning a vanishing point for depth of background objects –Assigning depth to foreground objects –“Painting in” behind objects

8. 8 David Luebke Real-Time Rendering Image-Based Modeling and Editing Byong Mok Oh, Max Chen, Julie Dorsey, and Fredo Durand

9. 9 David Luebke Real-Time Rendering Depth per pixel What if we could assign an exact depth to every pixel? Ex: MIT Image-Based Editing system

10. 10 David Luebke Real-Time Rendering Depth per pixel continued What if we had a “camera” that automatically acquired depth at every pixel? Ex: deltasphere Ex: Monticello project

11. 11 David Luebke Real-Time Rendering Laser rangefinder scanner Deltasphere 3000 by 3rdTech –Time of flight laser rangefinder Infrared or red visible 20,000 samples per second 40 foot range Accuracy ~ 1 mm –Panoramic scanner with spinning mirror to scan all directions –High-resolution digital camera with same nodal point Many similar products for different niches

12. 12 David Luebke Real-Time Rendering An aside: Virtual Monticello Switch presentations…

13. The Goal: From this…

14. …to this…

16. …to this.

17. Scanning Monticello: Challenges Single scan: 5-10 million points Each room: 4-6 scans Jefferson’s private suite: 5 rooms

18. Point cloud  mesh l Simplest approach: connect the dots l (Demo) … … …  

19. Point cloud  mesh l Problems: –Shouldn’t always fill holes –Need to merge multiple scans

20. Distance field l Instead: build volumetric distance field, extract zero-valued isosurface: –Robust to small errors within and between scans –Guaranteed to produce watertight mesh 0.3 0.4 0.30.1 -0.3 0.0 -0.1 -1.2 -0.8 -1.5 -1.8... -1.6... 0.9 0.8 1.2 0.9 0.8

21. Distance fields: Efficient construction l Usually represent DF on uniform grid l Naive approach to constructing DF: –Visit every grid cell (voxel) –Find nearest polygon in mesh –Record distance to voxel center l So what’s the problem? –Would like at least 1024x1024x1024 voxels –5-10 million polygons in mesh

22. Distance fields: Efficient construction l Could instead represent DF hierarchically l Our approach: –Walk through the scan file, inserting points into grid –Calculate local normal vector to mesh incrementally –Propagate distance from mesh outward from voxel containing point, to some maximum “ramp width”: –Use low-res 1-bit occupancy grid to indicate which voxels intersect mesh –Walk through occupied voxels to produce mesh 0.3 0.4 0.30.1 -0.3 0.0 -0.1 -1.2 -0.8 -1.5 -1.8... -1.6... 0.9 0.8 1.2 0.9 0.8

23. Some Results A single scan of Monticello library room (after range noise reduction)

24. Some Results Reconstructed model: 2.86 million vertices, 5.53 million triangles. Simplified to 1 million triangles.

25. Some Results