Automatic Compensation for Camera Settings for Images Taken under Different Illuminants School of Electrical Engineering and Computer Science Kyungpook National Univ. FourteenthColor Imaging Conference Fourteenth Color Imaging Conference2006 Cheng Lu and Mark S. Drew

2 / 16  Combination of two images for the same scene under different illumination –Estimating scene illumination –Enhancing photographs shot for shadow removal  Proposed method –Adjusting parametrically two images to compensate for the difference in camera setting Exposure speed, ISO, aperture size, and white balance –Existing effect of illumination Solving by using shadow effect Abstract

3 / 16  Work on image pairs taken for the same scene under different illumination –Analysis of the difference image (with flash image and no-flash image) Estimating surface reflectance and illumination in the no-flash image Enhancing photographs in dark environments and removing red-eye effects Removing the ambient shadow from the no-flash image – Inference of the contribution of the flash Must be in same camera settings Different lighting conditions Causing changes of camera settings Not investigated in this problem Introduction

4 / 16  Proposed method –Masking model to describe camera settings Assuming additivity and proportionality of the different factors involved Using 2nd-order masking model with 9 parameters –Training phase Image A ; only one light source, illuminant 1 Image B ; two light sources, illuminant 1 and illuminant 2

5 / 16 –An example of images Fig. 1. (a,b): Ambient-light image A, unscaled, and scaled for display. (c,d): “Both” image B (ambient + flash), unscaled, and scaled. (a) (c) (b) (d)

6 / 16  Additivity property –Subtracting the image A from image B Taken under illuminant 2 only –Situation with flash image and no-flash image Difference image –Pure flash image –Disappearance of shadow caused by sunlight

7 / 16  Consumer-grade cameras settings –Focal length –Exposure time (shutter speed) –Aperture (f-number) –ISO (film speed) –White balance  Other settings –Fixing the focal length –Using a tripod –Using RAW format Camera Settings and Image Acquisition

8 / 16  Exposure value (EV) –Sum of Aperture Value (AV) and Time Value (TV) AV TV  ISO (Film speed) –Measure of a photographic film’s sensitivity to light –Using 100 and 200  White balance settings –Auto, Daylight, Fluorescent, and Tungsten Where N ; the f-number Where t ; the shutter time in seconds

9 / 16  Goal ; Finding a model –Describing the relation between the difference of the two images and the camera settings –Transforming A to A’ Shadows caused by illuminant 1 will be removed in the difference image B−A’ – ; shadow region, ; out-of-shadow region A Masking Model for Compensating for Camera Settings Fig. 2. Ambient-light image A with in-shadow and out-of- shadow regions.

10 / 16 –Transforming A to A’ via a 3-coefficient vector M M ; a coefficient for each color channel –Using compensation of the camera settings of the two images –Function of the ratios of exposure value, ISO, and white balance between the two images no shadow effect M is a 3-vector.

11 / 16 –2nd-order masking model Describing the difference by camera settings Proposed originally for characterizing color printers –Using logarithms –Assuming additivity and proportionality The form of the 2nd-order model

12 / 16 –Difference image Fig. 3. (a): Pure-flash image (B−A) without compensation (b): With compensation (B-A’) (a) (b)

13 / 16  Experimental imaging environment –Using five lighting sources Direct sunlight Cloudy daylight Tungsten light lamp Incandescent lamp Xenon flash light –Using five objects with different colors on five different tablecloths Experiments and Results Fig. 4. Experimental imaging environment

14 / 16 –Capture of images under five situations Using direct sunlight as illuminant 1 to create shadows and adding the flash as illuminant 2 Using tungsten light as illuminant 1 and adding the flash as illuminant 2 Using tungsten light as illuminant 1 and adding cloudy daylight as illuminant 2 Using incandescent light as illuminant 1 and adding the flash as illuminant 2 Using incandescent light as illuminant 1 and adding cloudy daylight as illuminant 2

15 / 16  Examples of the results Fig. 5,6. Results: (a) images A, (b) image B, (c) image B−A’ (a)(b) (c)

16 / 16  Addressing the problem –Compensation for camera settings for image pairs for the same scene under different lighting conditions –Difficulty Difference of the two images –Composite of camera settings and scene illumination  Proposed method –Simple masking model to describe camera settings Summary