Manuel Gomez-Rodriguez* Jens Kober† Bernhard Schölkopf†

Slides:



Advertisements
Similar presentations
By Kevin Saunders.  tml tml.
Advertisements

Using a DSLR to Study Variable Stars. Canon 450D - linear over a magnitude using RAW images with a specific setting.
Topic 6 - Image Filtering - I DIGITAL IMAGE PROCESSING Course 3624 Department of Physics and Astronomy Professor Bob Warwick.
SVM—Support Vector Machines
Digital Photography with Flash and No-Flash Image Pairs By: Georg PetschniggManeesh Agrawala Hugues HoppeRichard Szeliski Michael CohenKentaro Toyama,
CASTLEFORD CAMERA CLUB
Multi-Task Compressive Sensing with Dirichlet Process Priors Yuting Qi 1, Dehong Liu 1, David Dunson 2, and Lawrence Carin 1 1 Department of Electrical.
CMPUT 466/551 Principal Source: CMU
Low Complexity Keypoint Recognition and Pose Estimation Vincent Lepetit.
An Introduction to Sparse Coding, Sparse Sensing, and Optimization Speaker: Wei-Lun Chao Date: Nov. 23, 2011 DISP Lab, Graduate Institute of Communication.
Data mining and statistical learning - lecture 6
Bayesian Robust Principal Component Analysis Presenter: Raghu Ranganathan ECE / CMR Tennessee Technological University January 21, 2011 Reading Group (Xinghao.
Presenter: Yufan Liu November 17th,
Detecting Digital Image Forgeries Using Sensor Pattern Noise presented by: Lior Paz Jan Lukas, jessica Fridrich and Miroslav Goljan.
Advanced Computer Graphics (Spring 2005) COMS 4162, Lectures 18, 19: Monte Carlo Integration Ravi Ramamoorthi Acknowledgements.
Mutual Information Mathematical Biology Seminar
Image Denoising via Learned Dictionaries and Sparse Representations
Image Forgery Detection by Gamma Correction Differences.
Learning Low-Level Vision William T. Freeman Egon C. Pasztor Owen T. Carmichael.
Lecture 4 Measurement Accuracy and Statistical Variation.
Monitoring and Pollutant Load Estimation. Load = the mass or weight of pollutant that passes a cross-section of the river in a specific amount of time.
Experimental Evaluation
CS Bayesian Learning1 Bayesian Learning. CS Bayesian Learning2 States, causes, hypotheses. Observations, effect, data. We need to reconcile.
Methods Eye (visible window) Camera – film Camera – CCD (Digital) Collecting Electromagnetic Information.
DIGITAL PHOTOGRAPHY: BACK TO BASICS Source:
Signal vs Noise: Image Calibration First… some terminology:  Light Frame: The individual pictures you take of your target.  Dark Frame: An image taken.
Photography Is the capture of reflective light on light sensitive material. Film-Base Photography used “silver” as the light sensitive material. Digital.
Digital Photography Basics. Pixels A pixel is a contraction if the term PIcture ELement. Digital images are made up of small squares, just like a tile.
Noise Estimation from a Single Image Ce Liu William T. FreemanRichard Szeliski Sing Bing Kang.
Digital Photography DeCal EECS98/198 Nathan Yan About this course -Technology of Camera Systems -Photographic Technique -Digital Lightroom About Me ^-doesn’t.
In Three Parts. » Know ˃What exposure is ˃What affects exposure » Show ˃Define Exposure ˃Identify an over, under, and perfectly exposed photo ˃Use exposure.
Computer vision: models, learning and inference Chapter 6 Learning and Inference in Vision.
Binary Variables (1) Coin flipping: heads=1, tails=0 Bernoulli Distribution.
Modeling And Visualization Of Aboriginal Rock Art in The Baiame Cave
A critical review of the Slanted Edge method for MTF measurement of color cameras and suggested enhancements Prasanna Rangarajan Indranil Sinharoy Dr.
1 Patch Complexity, Finite Pixel Correlations and Optimal Denoising Anat Levin, Boaz Nadler, Fredo Durand and Bill Freeman Weizmann Institute, MIT CSAIL.
AST3 detector properties
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 5 of Data Mining by I. H. Witten, E. Frank and M. A. Hall 報告人:黃子齊
Digital Photography A tool for Graphic Design Graphic Design: Digital Photography.
How A Camera Works Image Sensor Shutter Mirror Lens.
Astrophotography The Basics. Image Capture Devices Digital Compact cameras Webcams Digital SLR cameras Astronomical CCD cameras.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 11: Bayesian learning continued Geoffrey Hinton.
Chapter 3: Image Restoration Introduction. Image restoration methods are used to improve the appearance of an image by applying a restoration process.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
Digital Image Processing Lecture 10: Image Restoration March 28, 2005 Prof. Charlene Tsai.
PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Principles of Parameter Estimation.
Digital Image Processing Lecture 10: Image Restoration
Digital Image Processing CSC331 Image restoration 1.
Digital Photography A tool for Graphic Design Graphic Design: Digital Photography.
By: David Gelbendorf, Hila Ben-Moshe Supervisor : Alon Zvirin
Large-Scale Matrix Factorization with Missing Data under Additional Constraints Kaushik Mitra University of Maryland, College Park, MD Sameer Sheoreyy.
REVIEW OF PHOTO TERMS & INTRODUCTION TO DSLR CAMERA MODES PHOTO 2 - 8/20/13 [A DAY]
Regularization of energy-based representations Minimize total energy E p (u) + (1- )E d (u,d) E p (u) : Stabilizing function - a smoothness constraint.
Astrophotography The hunt for darker skies. Celestron advanced series GT C100ED-R.
Using Neumann Series to Solve Inverse Problems in Imaging Christopher Kumar Anand.
Evaluating Hypotheses. Outline Empirically evaluating the accuracy of hypotheses is fundamental to machine learning – How well does this estimate its.
Jianchao Yang, John Wright, Thomas Huang, Yi Ma CVPR 2008 Image Super-Resolution as Sparse Representation of Raw Image Patches.
Crime Scene Photography CJ327 Jonathan Dorriety Adjunct Professor School of Public Safety Kaplan University.
Digital Image -M.V.Ramachandranwww.youtube.com/postmanchandru
Equivalent Exposures and available light photography.
Observation of RR Lyrae Variable RS Boo Results and Future Work
Announcements After a short lecture we will adjourn to the Farm. Tonight is a Dark Sky make-up night. Class will meet Thursday night. If clear: meet at.
Unit 1 The History of Photography & The Camera
Photography Lab.
A tool for Graphic Design
A tool for Graphic Design
On Optimal Distributed Kalman Filtering in Non-ideal Situations
CSCI 1290: Comp Photo Fall Brown University James Tompkin
32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society Denoising of LSFCM images with compensation for the Photoblinking/Photobleaching.
A tool for Graphic Design
Presentation transcript:

Denoising photographs using dark frames optimized by quadratic programming Manuel Gomez-Rodriguez* Jens Kober† Bernhard Schölkopf† †Max Planck Institute for Biological Cybernetics Tübingen *Electrical Engineering Department Stanford University

Long exposure photographs Long exposure photographs (e.g., astronomical photographs) contain substantial amounts of noise. Dark current noise is the dominant source of noise in long exposure photographs. We have access to samples of the joint distribution of the noise of our camera using bias frames and dark frames

Noise profile A bias frame – raw image taken with closed shutter and exposure time ~ 0 seconds. The bias value is cased by the readout noise. A dark frame – raw image taken with closed shutter and nonzero exposure time. It contains a bias frame plus a component that increases with exposure time, in a way that depends on several other factors (i.e. temperature, ISO setting, …) A light frame – raw image to denoise.

The problem Given the observed image I + D and a few points sampled from the multidimensional noise distribution, we want to estimate I. X1 X2 XN D I Noise distribution

The problem We want to include the joint statistics of the sensor noise in our denoising method How should we combine the dark frames? Does it generalize to different conditions? Is the problem computationally tractable? Given a noisy long exposure setting A library of dark frames Denoised image

Naïve approach Single dark frame: record a dark frame of matching exposure time after each long exposure. This dark frame is subtracted from the light frame It is implemented on commercial cameras It doubles the amount of time The temperature tends to change One-point sample from the joint distribution of the noise

Average of dark frames approach Average of dark frames: a set of dark frames under conditions matching the ones of the light frame. The mean of the set is substracted from the light frame. Used, for example, in astrophotography Works well for professional cooled CCDs with precise temperature control Better estimate of the expected noise (multi-point sample)

Our approach The distribution of noise for a given camera depends on various conditions, including temperature, ISO settings and exposure time. If we knew the conditions for the image to be denoised, we should ideally use a library that matches the conditions of the image. But, The exact temperature is usually unknown We cannot store dark frames for every possible condition

Our approach Our method generates a synthetic dark frame from the convex hull of the dark frames D(1)…D(N), taken under different conditions, such that subtracting it from a noisy image optimizes a quality measure or prior for the class of images to denoise X1 X2 XN D I Noise distribution Image prior

Optimization problem If the quality measure is the smoothness of the image (i.e. discrete derivative), the convex optimization problem can be formulated as, where is the variable, is a real convex cost function, is a set of evaluation points and is the 8-neighbor set of the location in the raw image

Quadratic programming problem If a quadratic penalty function, , is chosen, the optimization problem is equivalent to the following quadratic program (QP) where

Solution of the QP A solution that generalizes well to the full image should be sparse because only the dark frames that were taking under similar conditions as the noisy image should be used for denoising; this is enforced by the constraints and , implying = 1 Our method also allows to estimate in an indirect way the exposure time, temperature and ISO of a photograph

Evaluation points As evaluation points, we use points that have high variance between dark frames and, The selection of evaluation points is done only once for a specific camera and a relatively low number of evaluation points (~1000) is enough The complexity does not depend on the size of the images but the # of dark frames As the solution is usually sparse, we only need to load a few full dark frames to denoise

Evaluation The same evaluation metric in the training set S and the test set T to numerically evaluate the performance; however S and T are disjoint → True generalization performance Dark frames taken with a Canon EOS 1Ds with, ISO of 800, 1000, 1250 Exposure times 1, 2, 4, 8,… 128 seconds, and 21 seconds Various temperature conditions have been used for the analysis

Evaluation Three problem instances in increasing order of difficulty are proposed to validate our method, Temperature Exposure time 1st problem Constant and matching the noisy image Variable, including the same exposure time as the noisy image 2nd problem Variable 3rd problem Variable, not including the same exposure time as the noisy image

Evaluation: 1st case Light frame with ISO 800, 16 seconds of exposure time 18 dark frames: constant temperature, variable exposure time Correct exposure time! Not used!

Evaluation: 2nd case Light frame: ISO 1000, 16 seconds of exposure time 175 dark frames: variable temperature, variable exposure time Not used! Correct exposure time!

Evaluation: 3rd case Light frame: ISO 1000, 21 seconds of exposure time 175 dark frames: variable temperature, variable exposure time (not inc. 21 sec) 200 evaluation points!

Noisy image Our method Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens

Bilateral filter Our method Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens

Wavelet denoising Our method Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens

Our method + wavelet denoising Horsehead nebula Barnard 33 in nebula IC 434, flame nebula NGC 2024, Canon EOS 5D with 300mm f/2.8 lens

Part of Orion constellation. Combination of ca Part of Orion constellation. Combination of ca. 10 R, G, and   B images, denoised using the proposed method. Canon 200mm lens, SBIG CCD camera using Kodak KAF 11002 CCD chip

Magnified detailed (Running Man Nebula)

Conclusions A relatively simple method with low complexity can help denoise long exposure images in raw format Our method can beneficially be combined with image-based noise reduction methods If available, our method could use evaluation points from the "optical black” (an area around the main image portion of the sensor which does not get light). We believe that the proposed method can become a practical tool for digital photography

Thank You!