1 What do color changes reveal about an outdoor scene? KalyanFabianoWojciechToddHanspeter SunkavalliRomeiroMatusikZicklerPfister Harvard University Adobe.

Slides:

Advertisements

Similar presentations

A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Color and Reflectance in Imaging and Computer Vision Workshop 2009 October.

Advertisements

What do color changes reveal about an outdoor scene? Kalyan Sunkavalli Fabiano Romeiro Wojciech Matusik Todd Zickler Hanspeter Pfister Harvard University.

Change Detection C. Stauffer and W.E.L. Grimson, “Learning patterns of activity using real time tracking,” IEEE Trans. On PAMI, 22(8): , Aug 2000.

www-video.eecs.berkeley.edu/research

Machine Vision Laboratory The University of the West of England Bristol, UK 2 nd September 2010 ICEBT Photometric Stereo in 3D Biometrics Gary A. Atkinson.

Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference:

Color Image Understanding Sharon Alpert & Denis Simakov.

Computational Photography and Capture: Time-Lapse Video Analysis & Editing Gabriel Brostow & Tim Weyrich TAs: Clément Godard & Fabrizio Pece.

Color-Invariant Motion Detection under Fast Illumination Changes Paper by:Ming Xu and Tim Ellis CIS 750 Presented by: Xiangdong Wen Advisor: Prof. Latecki.

Evolving Color Constancy Marc Ebner Universit ä t W ü rzburg, Germany Pattern Recognition Letters 27 （ 2006 ） Elsevier.

Scalability with many lights II (row-column sampling, visibity clustering) Miloš Hašan.

Nathan Johnson1 Background Subtraction Various Methods for Different Inputs.

Internet Vision - Lecture 3 Tamara Berg Sept 10. New Lecture Time Mondays 10:00am-12:30pm in 2311 Monday (9/15) we will have a general Computer Vision.

Forward-Backward Correlation for Template-Based Tracking Xiao Wang ECE Dept. Clemson University.

Physics-based Illuminant Color Estimation as an Image Semantics Clue Christian Riess Elli Angelopoulou Pattern Recognition Lab (Computer Science 5) University.

School of Computer Science Simon Fraser University November 2009 Sharpening from Shadows: Sensor Transforms for Removing Shadows using a Single Image Mark.

A Signal-Processing Framework for Forward and Inverse Rendering COMS , Lecture 8.

Shadow removal algorithms Shadow removal seminar Pavel Knur.

International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July Video Compression and Retrieval of Moving Object.

Iterative Image Registration: Lucas & Kanade Revisited Kentaro Toyama Vision Technology Group Microsoft Research.

Color Image Understanding Sharon Alpert & Denis Simakov.

Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows Kalyan Sunkavalli, Harvard University Joint work with Todd Zickler and Hanspeter Pfister.

Segmentation by Clustering Reading: Chapter 14 (skip 14.5) Data reduction - obtain a compact representation for interesting image data in terms of a set.

Introduction to Computer Vision CS / ECE 181B Tues, May 18, 2004 Ack: Matthew Turk (slides)

6/23/2015CIC 10, Color constancy at a pixel [Finlayson et al. CIC8, 2000] Idea: plot log(R/G) vs. log(B/G): 14 daylights 24 patches.

Color, lightness & brightness Lavanya Sharan February 7, 2011.

Colour Constancy T.W. Hung. Colour Constancy – Human A mechanism enables human to perceive constant colour of a surface over a wide range of lighting.

Structured Light in Scattering Media Srinivasa Narasimhan Sanjeev Koppal Robotics Institute Carnegie Mellon University Sponsor: ONR Shree Nayar Bo Sun.

Chromatic Framework for Vision in Bad Weather Srinivasa G. Narasimhan and Shree K. Nayar Computer Science Department Columbia University IEEE CVPR Conference.

Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance Yasuyuki Matsushita, Member, IEEE, Ko Nishino, Member, IEEE, Katsushi.

Object recognition under varying illumination. Lighting changes objects appearance.

Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley （ 2001 ） Two assumptions 1. the camera ’ s sensors are sufficiently.

Stereo Guest Lecture by Li Zhang

Project 1 artifact winners Project 2 questions Project 2 extra signup slots –Can take a second slot if you’d like Announcements.

Shadow Detection In Video Submitted by: Hisham Abu saleh.

Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2006 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.

Optical flow (motion vector) computation Course: Computer Graphics and Image Processing Semester:Fall 2002 Presenter:Nilesh Ghubade

Structure from images. Calibration Review: Pinhole Camera.

Online Tracking of Outdoor Lighting Variations for Augmented Reality with Moving Cameras Yanli Liu 1,2 and Xavier Granier 2,3,4 1: College of Computer.

1 Preview At least two views are required to access the depth of a scene point and in turn to reconstruct scene structure Multiple views can be obtained.

Joint Depth Map and Color Consistency Estimation for Stereo Images with Different Illuminations and Cameras Yong Seok Heo, Kyoung Mu Lee and Sang Uk Lee.

November 2012 The Role of Bright Pixels in Illumination Estimation Hamid Reza Vaezi Joze Mark S. Drew Graham D. Finlayson Petra Aurora Troncoso Rey School.

Accidental pinhole and pinspeck cameras: revealing the scene outside the picture A. Torralba and W. T. Freeman Proceedings of 25 IEEE Conference on Computer.

Colour changes in a natural scene due to the interaction between the light and the atmosphere Raúl Luzón González Colour Imaging Laboratory.

The Quotient Image: Class-based Recognition and Synthesis Under Varying Illumination T. Riklin-Raviv and A. Shashua Institute of Computer Science Hebrew.

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #16.

Expectation-Maximization (EM) Case Studies

Announcements Office hours today 2:30-3:30 Graded midterms will be returned at the end of the class.

Determining the location and orientation of webcams using natural scene variations Nathan Jacobs.

Announcements Final Exam Friday, May 16 th 8am Review Session here, Thursday 11am.

Using computer vision for analysis of plant growth condition: what to consider? Hans Jørgen Andersen Computer Vision and Media Technology laboratory Aalborg.

3D Reconstruction Using Image Sequence

Tal Amir Advanced Topics in Computer Vision May 29 th, 2015 COUPLED MOTION- LIGHTING ANALYSIS.

Learning and Removing Cast Shadows through a Multidistribution Approach Nicolas Martel-Brisson, Andre Zaccarin IEEE TRANSACTIONS ON PATTERN ANALYSIS AND.

Bo Sun Kalyan Sunkavalli Ravi Ramamoorthi Peter Belhumeur Shree Nayar Columbia University Time-Varying BRDFs.

1 Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization Neil G. Alldrin Satya P. Mallick David J. Kriegman University of California, San.

From local motion estimates to global ones - physiology:

Using Clouds Shadows to Infer Scene Structure and Camera Calibration

Intrinsic images and shape refinement

Image Restoration using Model-based Tracking

Rogerio Feris 1, Ramesh Raskar 2, Matthew Turk 1

A Unified Algebraic Approach to 2D and 3D Motion Segmentation

Optical Flow Estimation and Segmentation of Moving Dynamic Textures

Computer Vision Lecture 4: Color

Shadow Detection and Removal

Color-Invariant Motion Detection under Fast Illumination Changes

SURFACE COLOR AND SHADOWS

COLOR CONSTANCY IN THE COMPRESSED DOMAIN

Shape from Shading and Texture

Presentation transcript:

1 What do color changes reveal about an outdoor scene? KalyanFabianoWojciechToddHanspeter SunkavalliRomeiroMatusikZicklerPfister Harvard University Adobe Systems * * IEEE Conference on Computer Vision and Pattern Recognition 2008

2 Outdoor time-lapse

3 Time-lapse in Computer Vision Geo-location [ Jacobs et al ] Intrinsic Images [ Weiss 2001, Matsushita et al. 2003,… ] £ = Time-lapse sequence Texture Time-varying Shading [ Sunkavalli et al ] Editing

4 Coherence in time-lapse Most previous work makes use of only the temporal structure in time-lapse. overcast skyclear skysunset Outdoor time-lapse also has a colorimetric structure.

5 Outline 1.A physics-based model. 2.A method to fit the model to data. 3.Applications of the model.

6 Outline 1.A physics-based model. 2.A method to fit the model to data. 3.Applications of the model.

7 A model for outdoor time-lapse Assumption 1 : Static scene

8 A model for outdoor time-lapse ¸ ¸ ¸ observed pixel values scene color under canonical lighting Assumption 2 : Linear Transforms for Re-illumination color transform

9 A model for outdoor time-lapse Assumption 3 : Ambient sky and direct sun illumination sky illumination sun illumination mixing weights

10 A model for outdoor time-lapse Assumption 4 : 2-d subspace for daylight spectra [ Judd et al. 1964, Sastri and Das 1968, …, Hernández-Andrés et al ]

11 A model for outdoor time-lapse Assumption 4 : 2-d subspace for daylight spectra

12 A model for outdoor time-lapse illumination coefficients

13 A model for outdoor time-lapse scene color under canonical lighting (normalized)

14 A model for outdoor time-lapse

15 A model for outdoor time-lapse shadows

16 A model for outdoor time-lapse shading

17 A model for outdoor time-lapse 1-d projection of normals onto solar plane

18 Outline 1.A physics-based model. 2.A method to fit the model to data. 3.Applications of the model.

19 Fitting the model albedoshadowsnormals illum. bases angular velocity illum. coeffs. brightness over-constrained system 320 X 240 images, 100 frames 23,040,000 measurements 8,141,220 parameters

20 Fitting the model 1. Estimate albedo and illumination bases 2. Estimate shadows Reconstruct 3. Estimate remaining parameters

21 Fitting results originalreconstruction 240 x 360 images, 120 frames, ~5 mins. interval, 7.36% RMS error

22 Fitting results originalerror x x 360 images, 120 frames, ~5 mins. interval, 7.36% RMS error

23 Outline 1.A physics-based model. 2.A method to fit the model to data. 3.Applications of the model.

24 Application I : Color Constancy [ D’Zmura and Lennie 1986, Forsyth 1990, Funt et al. 1991, Finlayson and Funt 1994, Brainard and Freeman 1997, Barnard et al. 1997, Finlayson et al. 2001, Ebner 2004,… ]

25 Application I : Color Constancy originalcolor corrected

26 Application II : Scene Reconstruction

27 Application II : Scene Reconstruction

28 Application II : Scene Reconstruction

29 Application III : Geo-location

30 Application III : Geo-location

31 Summary 1.Outdoor time-lapse sequences are ubiquitous. 2.They are a rich source of scene information. 3.By using both temporal and colorimetric coherence we can access that scene information.

32 Future Work 1.Robust Fitting (mutual illumination, non-lambertian surfaces, foreground clutter). 2.Multiple viewpoints. 3.Estimating weather and atmospheric conditions.

33 Thank you!