1. Temperature Dependence of FPN in Logarithmic CMOS Image Sensors Dileepan Joseph¹ and Steve Collins² ¹University of Alberta, Canada ²University of Oxford, UK

2. May 1–3IMTC 20072 Outline Image Sensors Image Sensors –CCD versus CMOS –Linear versus logarithmic Logarithmic Imagers Logarithmic Imagers –Fixed pattern noise (FPN) –Colour rendition –Temperature dependence Conclusion Conclusion

3. May 1–3IMTC 20073 Image Sensors A digital camera consists of many components, of which the image sensor is considered principal A digital camera consists of many components, of which the image sensor is considered principal Typical imagers may be charge coupled device (CCD) sensors or complementary metal-oxide- semiconductor (CMOS) sensors Typical imagers may be charge coupled device (CCD) sensors or complementary metal-oxide- semiconductor (CMOS) sensors

4. May 1–3IMTC 20074 CCD Image Sensors March photo-generated charge systematically from an array of pixels to an output amplifier March photo-generated charge systematically from an array of pixels to an output amplifier Established technology Established technology High resolution, high sensitivity, low noise High resolution, high sensitivity, low noise Fabrication process is optimised for imaging Fabrication process is optimised for imaging

5. May 1–3IMTC 20075 CMOS Image Sensors Work like memory with photosensitive pixels inside each cell Work like memory with photosensitive pixels inside each cell Signal processing may be incorporated on the same die as pixels Signal processing may be incorporated on the same die as pixels High yield and good video performance High yield and good video performance May be fabricated by microchip makers May be fabricated by microchip makers

6. May 1–3IMTC 20076 Linear Pixels Linear pixels (either CCD or CMOS type) “count” photons over a discrete period of time Linear pixels (either CCD or CMOS type) “count” photons over a discrete period of time They produce a voltage directly proportional to the light intensity They produce a voltage directly proportional to the light intensity Unfortunately, the response may saturate white or black easily Unfortunately, the response may saturate white or black easily © IMS Chips http://www.ims-chips.de/

7. May 1–3IMTC 20077 Logarithmic Pixels Logarithmic pixels (CMOS only) measure the “rate” of photon incidence continuously Logarithmic pixels (CMOS only) measure the “rate” of photon incidence continuously They produce a voltage directly proportional to the logarithm of the light intensity They produce a voltage directly proportional to the logarithm of the light intensity The response is similar to that of human vision The response is similar to that of human vision © IMS Chips http://www.ims-chips.de/

8. May 1–3IMTC 20078 The Problem Logarithmic pixels are great for high dynamic range video but… Logarithmic pixels are great for high dynamic range video but… FPN is worse compared to typical linear pixels FPN is worse compared to typical linear pixels Colours are worse than for typical linear pixels Colours are worse than for typical linear pixels Impact of temperature on the image quality is poorly understood Impact of temperature on the image quality is poorly understood

9. May 1–3IMTC 20079 The Solution… IMTC 2001 IMTC 2001 –We fixed the fixed pattern noise IMTC 2002 IMTC 2002 –We improved the colour rendition IMTC 2007 IMTC 2007 –We considered temperature

10. May 1–3IMTC 200710 Fixed Pattern Noise Two photodetectors in the human eye or in a digital camera are not going to be identical Two photodetectors in the human eye or in a digital camera are not going to be identical A varying response to light stimulus causes “fixed pattern noise” A varying response to light stimulus causes “fixed pattern noise” The eye uses motion to factor out the FPN; not practical for cameras The eye uses motion to factor out the FPN; not practical for cameras

11. May 1–3IMTC 200711 Fixed Pattern Noise Modelling the FPN of logarithmic pixels, we improved calibration Modelling the FPN of logarithmic pixels, we improved calibration Responses to uniform stimuli were used to define corrections Responses to uniform stimuli were used to define corrections Our correction reduced the FPN to the same order as the random temporal noise Our correction reduced the FPN to the same order as the random temporal noise

12. May 1–3IMTC 200712 Colour Rendition We have shown how to render accurate colours with logarithmic pixels We have shown how to render accurate colours with logarithmic pixels A colour mapping was defined using images of a reference chart A colour mapping was defined using images of a reference chart Perceptual error of the rendered colours was comparable to that of consumer cameras Perceptual error of the rendered colours was comparable to that of consumer cameras Rendered Reference

13. May 1–3IMTC 200713 Temperature Dependence Unlike with humans, digital cameras do not regulate temperature Unlike with humans, digital cameras do not regulate temperature Temperature affects the response of a pixel to a light stimulus Temperature affects the response of a pixel to a light stimulus A “new” FPN appears when the temperature dependence varies from pixel to pixel A “new” FPN appears when the temperature dependence varies from pixel to pixel

14. May 1–3IMTC 200714 Temperature Dependence The dark response of a pixel depends only on temperature The dark response of a pixel depends only on temperature Thus, it may be used to correct FPN due to temperature in the light response Thus, it may be used to correct FPN due to temperature in the light response We validated this idea by simulation with real mismatch data We validated this idea by simulation with real mismatch data

15. May 1–3IMTC 200715 Conclusion Logarithmic CMOS image sensors are ideal for capturing high dynamic range video Logarithmic CMOS image sensors are ideal for capturing high dynamic range video Our research aims to improve the image quality of these cameras from machine grade to consumer grade and better Our research aims to improve the image quality of these cameras from machine grade to consumer grade and better The dark response of the image sensor may be used to correct temperature-dependent fixed pattern noise in the light response The dark response of the image sensor may be used to correct temperature-dependent fixed pattern noise in the light response Future work will simplify our methods and implement them in a complete prototype Future work will simplify our methods and implement them in a complete prototype

16. May 1–3IMTC 200716 Acknowledgements The authors gratefully acknowledge the support of the Natural Sciences and Engineering Research Council of Canada and the Engineering and Physical Sciences Research Council of the United Kingdom The authors gratefully acknowledge the support of the Natural Sciences and Engineering Research Council of Canada and the Engineering and Physical Sciences Research Council of the United Kingdom