1. Image-Based Visual Hulls Wojciech Matusik Chris Buehler Ramesh Raskar Steven Gortler Leonard McMillan Presentation by: Kenton McHenry

2. Visual Hull Project rays through silhouette of various views to carve out regions containing the object. The intersection of all such cones is the visual hull. Not the true object shape, concavities can not be captured.

3. Visual Hull

4. Previous Work CSG (Constructive Solid Geometry) –Boolean operators over polygonal silohuettes –3D intersections Volume Carving –Large memory requirements –Aliasing

5. Contribution Real time rendering –Perform intersection tests in 2D image space –No explicit hull

6. Epipolar Geometry Epipole: projection of 2 nd cameras optical center in our view Baseline: line connecting optical centers Epipolar line: Projection of ray in our view, can restrict search to this line. Epipolar plane: formed by the baseline and the epipolar lines in both images.

7. Epipolar Geometry

8. Elminiate Polyhedra Intersections … is equivelent to …

9. Elminiate Polyhedron-Line Intersections Absolute cross-section (defined by the silouhette) remains fixed (scaled). Can project our 3D ray into any plane along the cone, find intersections, and unproject.

10. Visual Hull Computation Given a desired view For each of the n 2 pixels create a ray from the optical center through the pixel and project it into each of the reference views (epipolar lines) Calculate intervals with silouhette Lift the intervals back into 3D space

11. Sampling the hull

12. Speed up Search over Edges Observation: The pixels of a scanline in the desired image scan out a pencil of line segments in the reference image whose slope varies monotonically.

13. Incremental Computation

14. Binning

15. Scanning

16. Complexity n = width/height of images k = number of reference images l = average number of intersections O(lkn 2 )

17. Partial Results

18. Shading Use reference images as textures View Dependent Texturing Assign pixel color based on reference view whose angle is most simular to desired view.

19. Visibility We need to watch out for reference views that have an occluded view of the point. All visibility interactions occur within epipolar planes.

20. 2D Visibility O(nl)

21. Discrete 3D Visibility Due to discretization the pixels of an epipolar line in one image may be on different lines in the other image. O(lkn 2 )

22. Results

24. Silhouettes Fixed cameras => Background substraction 2D version of marching cubes to recover edges of silhouette

25. The Fundamental Matrix x ref T F x des = 0 8-point algorithm Epipolar line corresponding to x des in reference image = F x des = (a b c) T such that ax + by + c = 0