1. Multi-Perspective Panoramas
CS194: Image Manipulation & Computational Photography
Alexei Efros, UC Berkeley, Fall 2017
Slides from Lihi Zelnik

2. Objectives Better looking panoramas Let the camera move: Any view
Natural photographing

3. Stand on the shoulders of giants
Cartographers
Artists

4. Cartographic projections

5. Common panorama projections
Perspective
Stereographic θ φ
Cylindircal

6. Global Projections
Perspective
Stereographic
Cylindircal

7. Learn from the artists Sharp discontinuity Multiple view points
perspective
Sharp discontinuity
Multiple view points
De Chirico “Mystery and Melancholy of a Street”, 1914

8. Renaissance painters solution
“School of Athens”, Raffaello Sanzio ~1510
Give a separate treatment to different parts of the scene!!

9. Personalized projections
“School of Athens”, Raffaello Sanzio ~1510
Give a separate treatment to different parts of the scene!!

10. Multiple planes of projection
Sharp discontinuities can often be well hidden

11. Single view
multi-view result

12. Single view
multi-view result

13. Single view
multi-view result

14. Single view
multi-view result

15. Objectives - revisited
Better looking panoramas
Let the camera move:
Any view
Natural photographing
Multiple views can live together

16. Multi-view compositions
3D!!
David Hockney, Place Furstenberg, (1985)

17. Why multi-view? Multiple viewpoints Single viewpoint David Hockney,
Place Furstenberg, 1985
Melissa Slemin,
Place Furstenberg, 2003

18. Long Imaging
Agarwala et al. (SIGGRAPH 2006)

19. Smooth Multi-View
Google maps

20. What’s wrong in the picture?
Google maps

21. Non-smooth
Google maps

22. The Chair
David Hockney (1985)

23. Joiners are popular Flickr statistics (Aug’07):
4,985 photos matching joiners.
4,007 photos matching Hockney.
41 groups about Hockney
Thousands of members

24. Main goals: Automate joiners Generalize panoramas to general image collections

25. Objectives For Artists: Reduce manual labor Manual: ~40min.
Fully automatic

26. Objectives For Artists: Reduce manual labor
For non-artists: Generate pleasing-to-the-eye joiners

27. Objectives For Artists: Reduce manual labor
For non-artists: Generate pleasing-to-the-eye joiners
For data exploration: Organize images spatially

28. What’s going on here?

29. A cacti garden

30. Principles

31. Principles
Convey topology
Correct
Incorrect

32. Principles Convey topology A 2D layering of images Blending: blurry
Graph-cut:
cuts hood
Desired joiner

33. Principles Convey topology A 2D layering of images
Don’t distort images
translate
rotate
scale

34. Principles Convey topology A 2D layering of images
Don’t distort images
Minimize inconsistencies
Bad
Good

35. Algorithm

36. Step 1: Feature matching
Brown & Lowe, ICCV’03

37. Large inconsistencies
Step 2: Align
Large inconsistencies
Brown & Lowe, ICCV’03

38. Reduced inconsistencies
Step 3: Order
Reduced inconsistencies

39. Ordering images
Try all orders: only for small datasets

40. Ordering images Try all orders: only for small datasets
complexity: (m+n) m = # images n = # overlaps  = # acyclic orders

41. Ordering images Observations:
Typically each image overlaps with only a few others
Many decisions can be taken locally

42. Ordering images Approximate solution:
Solve for each image independently
Iterate over all images

43. Can we do better?

44. Step 4: Improve alignment

45. Iterate Align-Order-Importance

46. Iterative refinement
Initial
Final

47. Iterative refinement
Initial
Final

48. Iterative refinement
Initial
Final

49. What is this?

50. That’s me reading

51. Anza-Borrego

52. Tractor

53. Manual by Photographer

54. Our automatic result

55. Failure?

56. GUI

57. The Impossible Bridge

58. Homage to David Hockney

59. Take home Incorrect geometries are possible and fun!
Geometry is not enough, we need scene analysis
A highly related work: "Scene Collages and Flexible Camera Arrays,” Y. Nomura, L. Zhang and S.K. Nayar, Eurographics Symposium on Rendering, Jun, 2007.

61. This Class Project from 2007