Radiometric Self Calibration Tomoo Mitsunaga Shree K. Nayar Hashimoto Signal Processing Lab. Dept. of Computer Science Sony Corporation Columbia University CVPR Conference Ft. Collins, Colorado June 1999
Usual imaging systems have : Problem Statement How well does the image represent the real world? Image M1 (High exposure) Image M2 (Low exposure) Limited dynamic range Usual imaging systems have : Non-linear response June/1999 CVPR99
Scene Radiance and Image Irradiance L E Radiance Irradiance Image irradiance : Ideal camera response : Aperture area Exposure : June/1999 CVPR99
Scene Radiance and Measured Brightness Video Image Formation Image Exposure Camera Electronics Digitization CCD Scaled radiance I Measured brightness M Scene radiance L linear Photo Image Formation Image Exposure Film Development Film Scanning f (M) : The radiometric response function June/1999 CVPR99
Calibration with Reference Objects The scene must be controlled The reflectance of the objects must be known The illumination must be controlled June/1999 CVPR99
Calibration without Reference Objects Differently exposed images from an arbitrary scene Recover the response function from the images Calibrate the images with the response function Response function Input Images High dynamic range radiance image June/1999 CVPR99
Previous Works Mann and Picard (95) : Debevec and Malik (97) : Take two images with known exposure ratio R Restrictive model for f : Find parameters a, b, g by regression Debevec and Malik (97) : General model for f : only smoothness constraint Take several (say, 10) high quality images At precisely measured exposures (shutter speed) June/1999 CVPR99
Obtaining Exposure Information We have only rough estimates Mechanical error Reading error (ex. F-stop number) June/1999 CVPR99
Radiometric Self-Calibration Works with roughly estimated exposures Inputs : Differently exposed images Rough estimates of exposure values ex. F-stop reading Outputs : Estimated response function Corrected exposure values June/1999 CVPR99
A Flexible Parametric Model video posi nega High order polynomial model : f (M) Parameters to be recovered : Coefficients cn Order N M f(M) of some popular imaging products June/1999 CVPR99
Response Function and Exposure Ratio Images: q = 1,2,….Q , Pixels: p = 1, 2, …..P Exposure ratio: Using polynomial model : Objective function : Thus, we obtain ... June/1999 CVPR99
An Iterative Scheme for Optimization Rough estimates Rq,q+1(0) Rq,q+1(i) Optimize for f Optimize for Rq,q+1 f (i) Optimized f and Rq,q+1 June/1999 CVPR99
Evaluation : Noisy Synthetic Images f (M) M Solid : Computed response function Dots : Actual response function June/1999 CVPR99
Evaluation : Noisy Synthetic Images (cont’d) Percentage Error in Computed Response Function Trial Number Maximum Error : 2.7 % June/1999 CVPR99
Computing a High Dynamic Range Image Calibrating by the response function Normalizing by corrected exposure values Averaging with SNR-based weighting June/1999 CVPR99
Results : Low Library (video) Captured images I Calibration chart M Computed response function June/1999 CVPR99
Results : Low Library (video) Captured images Computed radiance image June/1999 CVPR99
Results : Adobe Room (photograph) Captured images I M Computed response function Computed radiance image June/1999 CVPR99
Results : Taos Clay Oven (photograph) Captured images I M Computed response function Computed radiance image June/1999 CVPR99
A Practical Radiometric Self-calibration Method Conclusions A Practical Radiometric Self-calibration Method Works with Arbitrary still scene Rough estimates of exposure Recovers Response function of the imaging system High dynamic range image of the scene Software and Demo http://www.cs.columbia.edu/CAVE/ June/1999 CVPR99
Obtaining Quality Measurements Automatic noise reducing pre-processing For random noise within a pixel Temporal averaging For object movement and risky object edges* Selecting pixels from spatially static area For vignetting Preferring the center part of the image * Object edges are sensitive to noise June/1999 CVPR99