Physics-based Illuminant Color Estimation as an Image Semantics Clue Christian Riess Elli Angelopoulou Pattern Recognition Lab (Computer Science 5) University.

Slides:

Advertisements

Similar presentations

A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Color and Reflectance in Imaging and Computer Vision Workshop 2009 October.

Advertisements

Road-Sign Detection and Recognition Based on Support Vector Machines Saturnino, Sergio et al. Yunjia Man ECG 782 Dr. Brendan.

Evaluating Color Descriptors for Object and Scene Recognition Koen E.A. van de Sande, Student Member, IEEE, Theo Gevers, Member, IEEE, and Cees G.M. Snoek,

Announcements Project 2 due today Project 3 out today –demo session at the end of class.

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #12.

3D Modeling from a photograph

Color Image Understanding Sharon Alpert & Denis Simakov.

Image Segmentation Image segmentation (segmentace obrazu) –division or separation of the image into segments (connected regions) of similar properties.

Color spaces CIE - RGB space. HSV - space. CIE - XYZ space.

Multiple People Detection and Tracking with Occlusion Presenter: Feifei Huo Supervisor: Dr. Emile A. Hendriks Dr. A. H. J. Stijn Oomes Information and.

Rob Fergus Courant Institute of Mathematical Sciences New York University A Variational Approach to Blind Image Deconvolution.

School of Computer Science Simon Fraser University November 2009 Sharpening from Shadows: Sensor Transforms for Removing Shadows using a Single Image Mark.

Machinen Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen CT50A6100 Lectures 8&9: Image Segmentation Professor Heikki Kälviäinen Machine Vision.

A Study of Approaches for Object Recognition

Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2005 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.

Color Image Understanding Sharon Alpert & Denis Simakov.

Feature matching and tracking Class 5 Read Section 4.1 of course notes Read Shi and Tomasi’s paper on.

WORD-PREDICTION AS A TOOL TO EVALUATE LOW-LEVEL VISION PROCESSES Prasad Gabbur, Kobus Barnard University of Arizona.

Color, lightness & brightness Lavanya Sharan February 7, 2011.

Colour Constancy T.W. Hung. Colour Constancy – Human A mechanism enables human to perceive constant colour of a surface over a wide range of lighting.

Region Filling and Object Removal by Exemplar-Based Image Inpainting

Image Primitives and Correspondence

Object recognition under varying illumination. Lighting changes objects appearance.

Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley （ 2001 ） Two assumptions 1. the camera ’ s sensors are sufficiently.

Announcements PS3 Due Thursday PS4 Available today, due 4/17. Quiz 2 4/24.

A New Correspondence Algorithm Jitendra Malik Computer Science Division University of California, Berkeley Joint work with Serge Belongie, Jan Puzicha,

Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2006 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.

Statistical Color Models (SCM) Kyungnam Kim. Contents Introduction Trivariate Gaussian model Chromaticity models –Fixed planar chromaticity models –Zhu.

Computer Vision Spring ,-685 Instructor: S. Narasimhan WH 5409 T-R 10:30am – 11:50am Lecture #15.

Computer Vision Spring ,-685 Instructor: S. Narasimhan PH A18B T-R 10:30am – 11:50am Lecture #13.

Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.

Computer vision.

Gwangju Institute of Science and Technology Intelligent Design and Graphics Laboratory Multi-scale tensor voting for feature extraction from unstructured.

Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.

November 2012 The Role of Bright Pixels in Illumination Estimation Hamid Reza Vaezi Joze Mark S. Drew Graham D. Finlayson Petra Aurora Troncoso Rey School.

Why is computer vision difficult?

Computer Vision, Robert Pless

1 University of Texas at Austin Machine Learning Group 图像与视频处理计算机学院 Motion Detection and Estimation.

Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.

CS654: Digital Image Analysis

1 Research Question  Can a vision-based mobile robot  with limited computation and memory,  and rapidly varying camera positions,  operate autonomously.

Expectation-Maximization (EM) Case Studies

Histograms of Oriented Gradients for Human Detection(HOG)

Color and Brightness Constancy Jim Rehg CS 4495/7495 Computer Vision Lecture 25 & 26 Wed Oct 18, 2002.

Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science.

1Ellen L. Walker 3D Vision Why? The world is 3D Not all useful information is readily available in 2D Why so hard? “Inverse problem”: one image = many.

Thresholding Foundation:. Thresholding In A: light objects in dark background To extract the objects: –Select a T that separates the objects from the.

Instructor: Mircea Nicolescu Lecture 5 CS 485 / 685 Computer Vision.

Chapter 24: Perception April 20, Introduction Emphasis on vision Feature extraction approach Model-based approach –S stimulus –W world –f,

Announcements Final is Thursday, March 18, 10:30-12:20 –MGH 287 Sample final out today.

Learning and Removing Cast Shadows through a Multidistribution Approach Nicolas Martel-Brisson, Andre Zaccarin IEEE TRANSACTIONS ON PATTERN ANALYSIS AND.

Shadow Detection in Remotely Sensed Images Based on Self-Adaptive Feature Selection Jiahang Liu, Tao Fang, and Deren Li IEEE TRANSACTIONS ON GEOSCIENCE.

1 What do color changes reveal about an outdoor scene? KalyanFabianoWojciechToddHanspeter SunkavalliRomeiroMatusikZicklerPfister Harvard University Adobe.

MAN-522 Computer Vision Spring

- photometric aspects of image formation gray level images

Color Image Processing

CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.

Color Image Processing

Computer Vision, Robotics, Machine Learning and Control Lab

Rogerio Feris 1, Ramesh Raskar 2, Matthew Turk 1

Image Primitives and Correspondence

Image formation and the shape from shading problem

Image formation and the shape from shading problem

Image Segmentation Techniques

Color Image Processing

RGB-D Image for Scene Recognition by Jiaqi Guo

Filtering Things to take away from this lecture An image as a function

Image and Video Processing

COLOR CONSTANCY IN THE COMPRESSED DOMAIN

Filtering An image as a function Digital vs. continuous images

Presentation transcript:

Physics-based Illuminant Color Estimation as an Image Semantics Clue Christian Riess Elli Angelopoulou Pattern Recognition Lab (Computer Science 5) University of Erlangen-Nuremberg November 8, 2009

Page 2 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Illumination as an Image Semantics Clue Semantic analysis: Cues on objects, location, time Can we add illumination color as a clue?

Page 3 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Existing Methods in Illuminant Estimation Machine Learning-based, e.g.  Gamut Mapping (e.g. Forsyth)  Recent Gray Edge Variants (e.g. Gijsenij, Gevers)  Color by Correlation (e.g. Finlayson) Physics-based, e.g.  Dichromatic Reflectance-based (e.g. Klinker et al.)  Intersection of diffuse color planes (e.g. Finlayson)  Inverse-Intensity Chromaticity (Tan et al.) Require proper training Require “clean” input [1] Tan, Nishino, Ikeuchi. “Color Constancy through Inverse-Intensity Chromaticity Space,” Journal of the Optical Society of America A, 21(3): , [1]

Page 4 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Goal Show that physics-based methods can be used  On arbitrary images,  Maybe sacrificing the exact estimate,  But giving higher level information (semantics).

Page 5 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue The Inverse Intensity-Chromaticity Approach Neutral Interface Assumption (NIA): specular color = illuminant Image formation Let  Chromaticity  Diffuse chromaticity  Specular chromaticity  Rewritten w/ chromaticities Diffuse geometry Diffuse reflectance Specular geometry Specular reflectance

Page 6 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue The Inverse Intensity-Chromaticity Approach Tan et al. showed, where NIA: specular color = light color =. The illuminant chromaticity relates linearly to the sum of intensities and the pixel chromaticities. Specular regions form a triangle in inverse-intensity space, pointing to the illuminant color.

Page 7 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Towards Automated Scene Assessment Tan et al. propose to estimate the illuminant chromaticity using the Hough transform along the y-axis. Hypothesis: The histogram shape gives clues on the illuminant estimate quality, and the illumination environment in general. Hough Space along y-axis Hough Space, as a histogram in its own right

Page 8 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Influence of the Specularity Segmentation Experiments with different specularity segmentations R, G, B estimates surprisingly stable Noise component shows as a broader peak.

Page 9 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Experiments with similar scenes Illuminant color estimates not sufficiently reliable: Left image illuminant color estimate (0.378, 0.303, 0.318), Right image illuminant color estimate (0.339, 0.341, 0.319). Influence of the Specularity Segmentation

Page 10 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Towards automated Self-assessment Is it possible to detect unmet constraints (e.g. poor specularity segmentation)? Shape of the histogram peak  Fit Gaussian to peak  Fit triangle to peak  Optimization criteria  Intersection area  Sum of squared differences

Page 11 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Multiple illuminants mix in the histogram  Observed peak might be a mixture of noise components from multiple light sources.  Decomposition into ROIs shows the mixing property. Further Challenges on the Histogram Shape

Page 12 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Barely specularities, high amount of noise  Indoor illumination, reddish interreflections  Red/blue channels: Peak can be “hidden” Further Challenges on the Histogram Shape

Page 13 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Summary This is preliminary work. Shape of the histogram can be used as a semantic source of information. Method is insensitive to variations in specularity segmentation. Further analysis of histogram shape:  Impact of binning  Noise vs. multiple illuminants  Learn histogram shapes?  Iterative simultaneous segmentation and illuminant estimation

Page 14 Christian Riess Nov. 8, 2009Physics-based Illuminant Color Estimation as an Image Semantics Clue Image Sources Barnard et al.: A Comparison of Computational Color-Constancy Algorithms, IEEE TIP 11(9), , Thank you for your attention!