Multi-Perspective Panoramas

Slides:



Advertisements
Similar presentations
Recognising Panoramas M. Brown and D. Lowe, University of British Columbia.
Advertisements

Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France.
Image Registration  Mapping of Evolution. Registration Goals Assume the correspondences are known Find such f() and g() such that the images are best.
Data-driven methods: Texture (Sz 10.5) Cs129 Computational Photography James Hays, Brown, Spring 2011 Many slides from Alexei Efros.
Summary of Friday A homography transforms one 3d plane to another 3d plane, under perspective projections. Those planes can be camera imaging planes or.
Character Animation from 2D Pictures and 3D Motion Data ACM Transactions on Graphics 2007.
Announcements Project 2 Out today Sign up for a panorama kit ASAP! –best slots (weekend) go quickly...
Multi-perspective Panoramas Slides from a talk by Lihi Zelnik-Manor at ICCV’07 3DRR workshop.
Recognising Panoramas
Direct Methods for Visual Scene Reconstruction Paper by Richard Szeliski & Sing Bing Kang Presented by Kristin Branson November 7, 2002.
Homographies and Mosaics : Computational Photography Alexei Efros, CMU, Fall 2005 © Jeffrey Martin (jeffrey-martin.com) with a lot of slides stolen.
Lecture 7: Image Alignment and Panoramas CS6670: Computer Vision Noah Snavely What’s inside your fridge?
Automatic Panoramic Image Stitching using Local Features Matthew Brown and David Lowe, University of British Columbia.
Image-Based Lighting : Computational Photography Alexei Efros, CMU, Fall 2005 © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec.
Lecture 11: Structure from motion CS6670: Computer Vision Noah Snavely.
Image Stitching and Panoramas
Lecture 13: Projection, Part 2
1Jana Kosecka, CS 223b Cylindrical panoramas Cylindrical panoramas with some slides from R. Szeliski, S. Seitz, D. Lowe, A. Efros,
CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from:  David Lowe, UBC  Jiri Matas, CMP Prague
Panoramas and Calibration : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Rick Szeliski.
Opportunities of Scale Computer Vision James Hays, Brown Many slides from James Hays, Alyosha Efros, and Derek Hoiem Graphic from Antonio Torralba.
Wrap Up : Computational Photography Alexei Efros, CMU, Fall 2005 © Robert Brown.
Mosaics CSE 455, Winter 2010 February 8, 2010 Neel Joshi, CSE 455, Winter Announcements  The Midterm went out Friday  See to the class.
SBU Digital Media CSE 690 Internet Vision Organizational Meeting Tamara Berg Assistant Professor SUNY Stony Brook.
Single-view modeling, Part 2 CS4670/5670: Computer Vision Noah Snavely.
CSC 589 Lecture 22 Image Alignment and least square methods Bei Xiao American University April 13.
Photocollage System. What is Photocollage? Photographic approach in combining multiple images to create a new whole Result is an expressive composition.
Internet-scale Imagery for Graphics and Vision James Hays cs195g Computational Photography Brown University, Spring 2010.
Image Stitching Tamara Berg CSE 590 Computational Photography Many slides from Alyosha Efros & Derek Hoiem.
Automatic Registration of Color Images to 3D Geometry Computer Graphics International 2009 Yunzhen Li and Kok-Lim Low School of Computing National University.
Computational photography CS4670: Computer Vision Noah Snavely.
An Interesting Geography Tool A new way to create Photosynths using Microsoft ICE This Photosynth was created from 9 photographs taken with a FUJI FINEPIX.
Image-Based Lighting © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec CS194: Image Manipulation & Computational Photography Alexei Efros,
Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( )
ELE 488 Fall 2006 Image Processing and Transmission ( )
Improvements on Automated Registration CSc83020 Project Presentation Cecilia Chao Chen.
Hebrew University Image Processing Exercise Class 8 Panoramas – Stitching and Blending Min-Cut Stitching Many slides from Alexei Efros.
Homographies and Mosaics : Computational Photography Alexei Efros, CMU, Fall 2006 © Jeffrey Martin (jeffrey-martin.com) with a lot of slides stolen.
Today’s Lecture Neural networks Training
More Mosaic Madness © Jeffrey Martin (jeffrey-martin.com)
Image-based Lighting Computational Photography
CS 4501: Introduction to Computer Vision Sparse Feature Detectors: Harris Corner, Difference of Gaussian Connelly Barnes Slides from Jason Lawrence, Fei.
donated by Paul Debevec
Cubism Photomontage.
COSC579: Image Align, Mosaic, Stitch
Markus Lipp1, Daniel Scherzer1, Peter Wonka2 and Michael Wimmer1
Scene Modeling for a Single View
Image-Based Rendering
Multi-perspective Panoramas
RANSAC and mosaic wrap-up
© 2005 University of Wisconsin
More Mosaic Madness : Computational Photography
A special case of calibration
Modeling the world with photos
Real-Time Image Mosaicing
What have we learned so far?
More Mosaic Madness © Jeffrey Martin (jeffrey-martin.com)
Image Based Modeling and Rendering (PI: Malik)
More Mosaic Madness : Computational Photography
High Dynamic Range Images
Image Stitching Computer Vision CS 678
Image Quilting for Texture Synthesis & Transfer
Noah Snavely.
More Mosaic Madness © Jeffrey Martin (jeffrey-martin.com)
Announcements Project 1 artifact voting Project 2 out today
Computational Photography
Camera Calibration from Planar Patterns
Calibration and homographies
Lecture 15: Structure from motion
Image Registration  Mapping of Evolution
Presentation transcript:

Multi-Perspective Panoramas CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2017 Slides from Lihi Zelnik

Objectives Better looking panoramas Let the camera move: Any view Natural photographing

Stand on the shoulders of giants Cartographers Artists

Cartographic projections

Common panorama projections Perspective Stereographic θ φ Cylindircal

Global Projections Perspective Stereographic Cylindircal

Learn from the artists Sharp discontinuity Multiple view points perspective Sharp discontinuity Multiple view points De Chirico “Mystery and Melancholy of a Street”, 1914

Renaissance painters solution “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

Personalized projections “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

Multiple planes of projection Sharp discontinuities can often be well hidden

Single view multi-view result

Single view multi-view result

Single view multi-view result

Single view multi-view result

Objectives - revisited Better looking panoramas Let the camera move: Any view Natural photographing Multiple views can live together

Multi-view compositions 3D!! David Hockney, Place Furstenberg, (1985)

Why multi-view? Multiple viewpoints Single viewpoint David Hockney, Place Furstenberg, 1985 Melissa Slemin, Place Furstenberg, 2003

Long Imaging Agarwala et al. (SIGGRAPH 2006)

Smooth Multi-View Google maps

What’s wrong in the picture? Google maps

Non-smooth Google maps

The Chair David Hockney (1985)

Joiners are popular Flickr statistics (Aug’07): 4,985 photos matching joiners. 4,007 photos matching Hockney. 41 groups about Hockney Thousands of members

Main goals: Automate joiners Generalize panoramas to general image collections

Objectives For Artists: Reduce manual labor Manual: ~40min. Fully automatic

Objectives For Artists: Reduce manual labor For non-artists: Generate pleasing-to-the-eye joiners

Objectives For Artists: Reduce manual labor For non-artists: Generate pleasing-to-the-eye joiners For data exploration: Organize images spatially

What’s going on here?

A cacti garden

Principles

Principles Convey topology Correct Incorrect

Principles Convey topology A 2D layering of images Blending: blurry Graph-cut: cuts hood Desired joiner

Principles Convey topology A 2D layering of images Don’t distort images translate rotate scale

Principles Convey topology A 2D layering of images Don’t distort images Minimize inconsistencies Bad Good

Algorithm

Step 1: Feature matching Brown & Lowe, ICCV’03

Large inconsistencies Step 2: Align Large inconsistencies Brown & Lowe, ICCV’03

Reduced inconsistencies Step 3: Order Reduced inconsistencies

Ordering images Try all orders: only for small datasets

Ordering images Try all orders: only for small datasets complexity: (m+n) m = # images n = # overlaps  = # acyclic orders

Ordering images Observations: Typically each image overlaps with only a few others Many decisions can be taken locally

Ordering images Approximate solution: Solve for each image independently Iterate over all images

Can we do better?

Step 4: Improve alignment

Iterate Align-Order-Importance

Iterative refinement Initial Final

Iterative refinement Initial Final

Iterative refinement Initial Final

What is this?

That’s me reading

Anza-Borrego

Tractor

Manual by Photographer

Our automatic result

Failure?

GUI

The Impossible Bridge

Homage to David Hockney

Take home Incorrect geometries are possible and fun! Geometry is not enough, we need scene analysis A highly related work: "Scene Collages and Flexible Camera Arrays,” Y. Nomura, L. Zhang and S.K. Nayar, Eurographics Symposium on Rendering, Jun, 2007.

This Class Project from 2007 http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj_final/www/echuangs/