1. Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( 048921 )

2. 2 Scene Reconstruction Scene Visualization: Photo Tourism: 3D photo browser Path Finder: 3D navigation interface References: Scene Reconstruction and Visualization from Internet Photo Collections. N. Snavely. 2008.

3. 3 Scene Reconstruction

4. 4 The goal Automatically reconstruct 3D geometry from large collections of photos from the Internet

5. Motivation 5  New way to organize photo collections  Experience of being at a place

6. 6 Related works  Image based modeling  Structure from motion (SfM)  Moviemaps

7. 7 Challenges  Unknown cameras position  Unknown internal camera parameters  Correspondence problem  Visualization  Algorithmic complexity Running the complete pipeline on a 3.80GHz Intel Xeon machine with 4GB of core memory. The keypoint detection and matching phases were run in parallel on 10 such machines.

8. 8 Results

9. 9 How was this result achieved?

10. 10 Finding correspondence  Feature detection - SIFT  Feature matching - RANSAC  Epipolar constraint

11. 11 Structure from Motion Algorithm

12. 12 Initial two-frame reconstruction Intermediate stageFinal reconstruction Structure from Motion Algorithm

13. 13 Failure modes  Repeating textures  Insufficient overlap or texture  Cascading errors  Bad initialization

14. 14 Connectivity graphs

15. 15 Scene Visualization: Photo Tourism: 3D photo browser

16. 16 Photo Tourism: Main components  Rendering engine  Navigation interface  Tools for scene annotation

17. 17 Transitions between photographs

18. 18 Navigation interface  Object-based navigation  Stabilized slideshow

19. 19 Scene annotation  Annotate regions  Register new photographs

20. 20 Scene Visualization: Path Finder: 3D navigation interface

21. 21 Best possible controls  The goals of 3D navigation: Exploration Search Object inspection  Principles for 3D scene navigation: Controls are scene and task dependent Constrained and automatic navigation User’s position and motion should be clear

22. 22 Pathfinder: Main components  Viewpoint scoring  Navigation controls for a scene  Rendering engine  User interface

23. 23 Viewpoint scoring: Criteria  Angular deviation  Field of view  Resolution

24. 24 Scene-specific navigation controls Goal:  Derive scene-specific controls  Find interesting views  Calculate optimized paths Solution:  Orbits and panoramas  Canonical views  Path planning

25. 25 Scene-specific controls: Panoramas and Orbits  Panoramas  Orbits

26. 26 Path planning  High-quality rendering  Shortest path  Piecewise linear path  Path smoothing Connectivity graph

27. 27 Appearance stabilization  Visual similarity  Color compensation Orbiting Trevi Fountain: Without stabilization: Similarity mode: Color compensation:

28. 28 Other applications  Viewing different appearance states  3D slideshow of a personal photo collection

29. 29 Limitations  Non circular orbits  User needs  Appearance compensation  Larger scenes (a)Pisa Doumo (b)Stonehenge

30. 30 Conclusion  Structure from motion  Photo Tourism - 3D photo browser  Path Finder - 3D navigation system  Reconstructing Rome  Capturing appearance  New photo-sharing community Future work

31. 31 Thank you