Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( )

2 Scene Reconstruction Scene Visualization: Photo Tourism: 3D photo browser Path Finder: 3D navigation interface References: Scene Reconstruction and Visualization from Internet Photo Collections. N. Snavely

3 Scene Reconstruction

4 The goal Automatically reconstruct 3D geometry from large collections of photos from the Internet

Motivation 5  New way to organize photo collections  Experience of being at a place

6 Related works  Image based modeling  Structure from motion (SfM)  Moviemaps

7 Challenges  Unknown cameras position  Unknown internal camera parameters  Correspondence problem  Visualization  Algorithmic complexity Running the complete pipeline on a 3.80GHz Intel Xeon machine with 4GB of core memory. The keypoint detection and matching phases were run in parallel on 10 such machines.

8 Results

9 How was this result achieved?

10 Finding correspondence  Feature detection - SIFT  Feature matching - RANSAC  Epipolar constraint

11 Structure from Motion Algorithm

12 Initial two-frame reconstruction Intermediate stageFinal reconstruction Structure from Motion Algorithm

13 Failure modes  Repeating textures  Insufficient overlap or texture  Cascading errors  Bad initialization

14 Connectivity graphs

15 Scene Visualization: Photo Tourism: 3D photo browser

16 Photo Tourism: Main components  Rendering engine  Navigation interface  Tools for scene annotation

17 Transitions between photographs

18 Navigation interface  Object-based navigation  Stabilized slideshow

19 Scene annotation  Annotate regions  Register new photographs

20 Scene Visualization: Path Finder: 3D navigation interface

21 Best possible controls  The goals of 3D navigation: Exploration Search Object inspection  Principles for 3D scene navigation: Controls are scene and task dependent Constrained and automatic navigation User’s position and motion should be clear

22 Pathfinder: Main components  Viewpoint scoring  Navigation controls for a scene  Rendering engine  User interface

23 Viewpoint scoring: Criteria  Angular deviation  Field of view  Resolution

24 Scene-specific navigation controls Goal:  Derive scene-specific controls  Find interesting views  Calculate optimized paths Solution:  Orbits and panoramas  Canonical views  Path planning

25 Scene-specific controls: Panoramas and Orbits  Panoramas  Orbits

26 Path planning  High-quality rendering  Shortest path  Piecewise linear path  Path smoothing Connectivity graph

27 Appearance stabilization  Visual similarity  Color compensation Orbiting Trevi Fountain: Without stabilization: Similarity mode: Color compensation:

28 Other applications  Viewing different appearance states  3D slideshow of a personal photo collection

29 Limitations  Non circular orbits  User needs  Appearance compensation  Larger scenes (a)Pisa Doumo (b)Stonehenge

30 Conclusion  Structure from motion  Photo Tourism - 3D photo browser  Path Finder - 3D navigation system  Reconstructing Rome  Capturing appearance  New photo-sharing community Future work

31 Thank you