Modelling, calibration and rendition of colour logarithmic CMOS image sensors Dileepan Joseph and Steve Collins Department of Engineering Science University.

Slides:



Advertisements
Similar presentations
Computer Vision Radiometry. Bahadir K. Gunturk2 Radiometry Radiometry is the part of image formation concerned with the relation among the amounts of.
Advertisements

A Standardized Workflow for Illumination-Invariant Image Extraction Mark S. Drew Muntaseer Salahuddin Alireza Fathi Simon Fraser University, Vancouver,
Quality Assurance and Digital Radiography
Use of Kalman filters in time and frequency analysis John Davis 1st May 2011.
Simon Fraser University Computational Vision Lab Lilong Shi, Brian Funt and Tim Lee.
O-24 A Reexamination of SRM as a Means of Beer Color Specification A.J. deLange ASBC 2007 Annual Meeting June 19, 2007.
Pixel-level delta-sigma ADC with optimized area and power for vertically-integrated image sensors 1 Alireza Mahmoodi and Dileepan Joseph University of.
Temperature Dependence of FPN in Logarithmic CMOS Image Sensors Dileepan Joseph¹ and Steve Collins² ¹University of Alberta, Canada ²University of Oxford,
Modelling, calibration and correction of nonlinear illumination-dependent fixed pattern noise in logarithmic CMOS image sensors Dileepan Joseph and Steve.
BMVC 2009 Specularity and Shadow Interpolation via Robust Polynomial Texture Maps Mark S. Drew 1, Nasim Hajari 1, Yacov Hel-Or 2 & Tom Malzbender 3 1 School.
Logarithmic CMOS image sensors Dr. Dileepan Joseph Dept. of Engineering Science University of Oxford, UK.
School of Computing Science Simon Fraser University
Active Calibration of Cameras: Theory and Implementation Anup Basu Sung Huh CPSC 643 Individual Presentation II March 4 th,
Motion Tracking. Image Processing and Computer Vision: 82 Introduction Finding how objects have moved in an image sequence Movement in space Movement.
Detecting Digital Image Forgeries Using Sensor Pattern Noise presented by: Lior Paz Jan Lukas, jessica Fridrich and Miroslav Goljan.
Video Coding with Linear Compensation (VCLC) Arif Mahmood, Zartash Afzal Uzmi, Sohaib A Khan Department of Computer.
Image processing. Image operations Operations on an image –Linear filtering –Non-linear filtering –Transformations –Noise removal –Segmentation.
Digital Cameras CCD (Monochrome) RGB Color Filter Array.
Motion based Correspondence for Distributed 3D tracking of multiple dim objects Ashok Veeraraghavan.
Mark S. Drew and Amin Yazdani Salekdeh School of Computing Science,
Modeling the imaging system Why? If a customer gives you specification of what they wish to see, in what environment the system should perform, you as.
High Dynamic Range Imaging: Spatially Varying Pixel Exposures Shree K. Nayar, Tomoo Mitsunaga CPSC 643 Presentation # 2 Brien Flewelling March 4 th, 2009.
Color Fidelity in Multimedia H. J. Trussell Dept. of Electrical and Computer Engineering North Carolina State University Raleigh, NC
Achieving True Color Fidelity
1D or 2D array of photosensors can record optical images projected onto it by lens system. Individual photosensor in an imaging array is called pixel.
Statistical Color Models (SCM) Kyungnam Kim. Contents Introduction Trivariate Gaussian model Chromaticity models –Fixed planar chromaticity models –Zhu.
Light and shading Source: A. Efros.
Noise Estimation from a Single Image Ce Liu William T. FreemanRichard Szeliski Sing Bing Kang.
Orit Skorka and Dileepan Joseph University of Alberta, Canada Reducing Crosstalk in Vertically- Integrated CMOS Image Sensors.
Digital Imaging and Remote Sensing Laboratory Mixed Pixels and Spectral Unmixing 1 Spectral Mixtures areal aggregate intimate Linear mixing models (areal.
Digital Imaging Systems –I/O. Workflow of digital imaging Two Competing imaging format for motion pictures Film vs Digital Video( TV) Presentation of.
A 200x100 Array Of Electronically Calibratable Logarithmic CMOS Pixels
Photon Transfer Method 1. Using two identical flat field exposures it is possible to measure the read noise of a CCD with the Photon Transfer method. Two.
Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka , Japan 1August.
Colour an algorithmic approach Thomas Bangert PhD Research Topic.
Computer Science 631 Lecture 7: Colorspace, local operations
Purdue University Optimization of Sensor Response Functions for Colorimetry of Reflective and Emissive Objects Mark Wolski*, Charles A. Bouman, Jan P.
Speech Enhancement Using Spectral Subtraction
A Projection Method to Generate Anaglyph Stereoscopic Images Eric Dubois VIVA Lab (Video, Image, Vision & Audio Research Lab) School of Information Technology.
CMOS Image Sensor Design. M. Wäny Nov EMVA Standard 1288 Standard for Measurement and Presentation of Specifications for Machine Vision Sensors and.
An Electronic Calibration Scheme for Logarithmic CMOS Pixels Bhaskar Choubey, Satoshi Ayoma*, Stephen Otim, Dileepan Joseph**, Steve Collins, University.
Bio-inspired design: nonlinear digital pixels for multiple-tier processes (invited paper) SPIE Nano/Bio/Info-Tech Sensors and Systems March 2013 O. Skorka,
16421: Vision Sensors Lecture 7: High Dynamic Range Imaging Instructor: S. Narasimhan Wean 5312, T-R 1:30pm – 3:00pm.
Module 2 : Linearity AGENDA TVI Vision, Kari Siren Linearity in general Theory, what does non-linearity mean when measuring “true” colour How to measure,
Karman filter and attitude estimation Lin Zhong ELEC424, Fall 2010.
1 An Observatory for Ocean, Climate and Environment SAC-D/Aquarius HSC - Radiometric Calibration H Raimondo M Marenchino 7th SAC-D Aquarius Science Meeting.
Sensory Information Processing
Computer and Robot Vision II Chapter 20 Accuracy Presented by: 傅楸善 & 王林農 指導教授 : 傅楸善 博士.
Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science.
Sampling Pixel is an area!! – Square, Rectangular, or Circular? How do we approximate the area? – Why bother? Color of one pixel Image Plane Areas represented.
ECE 638: Principles of Digital Color Imaging Systems
ECE 638: Principles of Digital Color Imaging Systems Lecture 3: Trichromatic theory of color.
EE 638: Principles of Digital Color Imaging Systems Lecture 17: Digital Camera Characterization and Calibration.
Greg Ward Exponent - Failure Analysis Assoc. Elena Eydelberg-Vileshin
ECE 638: Principles of Digital Color Imaging Systems Lecture 12: Characterization of Illuminants and Nonlinear Response of Human Visual System.
In conclusion the intensity level of the CCD is linear up to the saturation limit, but there is a spilling of charges well before the saturation if.
Date of download: 6/1/2016 Copyright © 2016 SPIE. All rights reserved. (a) Optical image of fore and hind wings from a male S. charonda butterfly at different.
Date of download: 6/22/2016 Copyright © 2016 SPIE. All rights reserved. Schematic representation of the near-infrared (NIR) structured illumination instrument,
ECE 638: Principles of Digital Color Imaging Systems
Digital Light Sources First introduced in 2001.
Color Image Processing
ECE 638: Principles of Digital Color Imaging Systems
DR. QUAZI DELWAR HOSSAIN
Linear Filters and Edges Chapters 7 and 8
Color Image Processing
Perception and Measurement of Light, Color, and Appearance
Sampling Theorem told us …
Statistical Methods For Engineers
Exposing Digital Forgeries by Detecting Traces of Resampling Alin C
Human Perception 9: Color.
Presentation transcript:

Modelling, calibration and rendition of colour logarithmic CMOS image sensors Dileepan Joseph and Steve Collins Department of Engineering Science University of Oxford

IMTC May Outline Logarithmic CMOS image sensors Modelling sensor response Image sensor calibration  Fixed pattern noise  Sensation of colour Rendition of images  CIE Lab (perceptual error)  IEC sRGB (standard display) Summary and future work

IMTC May Logarithmic CMOS image sensors CMOS displacing CCD because of integration of signal processing and economies of scale Logarithmic sensors offer high dynamic range and high frame rate Linear sensors offer low fixed pattern noise and good colour rendition Example images taken from IMS Chips website Linear CCD sensor Logarithmic sensor Logarithmic sensor

IMTC May Modelling sensor response Since I k = ∫ f k (λ) s(λ) dλ  For photocurrent I k, spectral response f k (λ) and light stimulus s(λ) at a pixel, where k = R, G or B And f k (λ) = g L (λ) g k (λ) g P (λ)  For spectral responses of lens g L (λ), colour filter g k (λ) and photodiode g P (λ)  Approximating a linear combination of three CIE XYZ basis functions Then I k = d k x  For mask coefficients d k and tricolour vector x, i.e. s(λ) in CIE XYZ space Ideally, y = a + b ln (c + I k ) + ε  For digital response y of pixel with offset a, gain b, bias c and error ε  Pixel-to-pixel variation of a, b or c causes fixed pattern noise (FPN)

IMTC May FPN calibration Three types of FPN of interest:  Offset variation  Offset and gain variation  Offset, gain and bias variation Partition pixels by colour filter to permit FPN calibration of three monochromatic sensors Take images of uniform stimuli under different illuminances Calibrate each pixel’s response to average response of all pixels by least squares estimation of varying model parameters Fuga 15RGB sensor exhibits offset, gain and bias variation

IMTC May Colour calibration Take and segment images of a standard chart, having patches of known CIE XYZ colour, under different illuminances Calibrate pixel responses to colour by estimating non-varying model parameters (e.g. mask d k ), using estimates of varying parameters Ideal model fails for Fuga 15RGB because absolute relationship between y and I k invalid (strong inversion component?) Empirical model y = a + b ln (c + (α + d k x) β ) worked well, with no change to relative responses of pixels or FPN calibration

IMTC May Image rendition (CIE Lab) Images of a Macbeth Colour Chart, taken by the Fuga 15RGB, were rendered into CIE Lab space with the calibrated empirical model The perceptual error increases in dim lighting as the bias term c dominates the photocurrent I k Excluding the dimmest image (i.e. 5 lux), the error equals 12 over a 60 dB dynamic range for offset, gain and bias variation Images in Digital Photographer show that conventional (linear) digital cameras have an error of 15 over a 30 dB dynamic range

IMTC May Image rendition (IEC sRGB) A Fuga 15RGB image of the Macbeth Chart, taken in 11 lux of illuminance, was rendered into IEC sRGB space with the calibrated empirical model Results for offset variation (top- left), offset and gain variation (top- right), offset, gain and bias variation (bottom-left) and true colours (bottom-right) are shown Two types of residual deviation for the rendered patches are visible:  Fixed pattern noise  Colour desaturation

IMTC May Summary and future work Logarithmic image sensors offer high dynamic range and frame rate Combine theories of colour linear sensors and monochromatic logarithmic sensors to model colour logarithmic sensors Calibrate FPN, using images of uniform stimuli, by relative estimation of model parameters that vary from pixel to pixel Calibrate colour, using images of a colour chart, by absolute estimation of model parameters that do not vary Fuga 15RGB results expose limitations of ideal model in absolute estimation but reveal empirical model that works well Macbeth Chart results show colour rendition with calibrated Fuga 15RGB competes with conventional digital cameras Seek to minimise bias variation, so simple FPN models suffice, and bias magnitude, to improve colour rendition in dim lighting

IMTC May Acknowledgements The authors gratefully acknowledge the support of the Natural Sciences and Engineering Research Council (Canada) and the Engineering and Physical Sciences Research Council (UK)