Overview of Modern Imaging Sensors: Proper Uses for CCD, EMCCD, and CMOS Cameras

Charge-Coupled Device (CCD) Charge-Coupled Device (CCD) • Introduced in 1969 (Scientific Imaging Standard for 40 years) Electron Multiplying Charge-Coupled Device (EMCCD) • Introduced in 2001 (Scientific Low-Light Imaging Standard for 10 years) Scientific Grade Complementary Metal Oxide Semiconductor (CMOS) • Introduced in 2010 (Beginning to be Adopted)

Same Basic Pixel Architecture Peak Efficiency when Front-Illuminated: ~65% CCD or ~57-65% scientific CMOS Incident Light Electrical Connection Polysilicon Gate Silicon Dioxide e- e- e- e- Silicon Potential Well Light when Back-Illuminated Efficiency when Back-Illuminated: ~95%

CCD Collecting Photons: Analogous to Catching Raindrops

CCD Collecting Photons: Analogous to Catching Raindrops

CCD Collecting Photons: Analogous to Catching Raindrops

Clocking and Other Control Electronics CCD Camera Components Sensor Custom Electronics Clocking and Other Control Electronics Photons  Electrons Parallel Register Gain A/D Converter Serial Register Electrons  Voltage

CCD Image Acquisition Types Full Frame Frame Transfer CCD Interline CCD

EMCCD Cameras Standard Readout Amplifier & ADC Back-illuminated, frame-transfer CCD (for >90% QE) or standard front-illuminated (~65% QE) Standard Readout Amplifier & ADC High Voltage EM register multiplies signal electrons by up to ~1000x BEFORE readout Normal Voltage Serial Register High Voltage Serial Register EM Readout Amplifier & ADC

EMCCD Cameras Back-illuminated, frame-transfer CCD (for >90% QE) or standard front-illuminated (~65% QE) Normal Voltage Serial Register High Voltage Serial Register

CMOS Sensor Components Photosensitive Diode Amplifier Row Select Column Select Photons  Electrons  Voltage Electronics Built into CMOS Chip

CMOS Sensor Readout Modes Global Shutter Rolling Shutter

Arrows Indicate Movement Direction Rolling Shutter If rolling shutter captures in this direction, Arrows Indicate Movement Direction Observed object In rolling shutter mode, each row starts and ends an exposure at a different point in time, not simultaneously as in global shutter mode. If an imaged object is moving, different parts of it will be captured at different times by the rolling shutter, causing possible distortion if object moves quickly enough. The distortion will depend on which direction the sampled object is moving relative to the rolling shutter direction.

CMOS Sensor Readout Modes Rolling Shutter (More Common) Global Shutter

How to quantitatively compare performance: Signal-to-Noise Ratio (SNR) 𝑆𝑁𝑅= 𝐼𝑚𝑎𝑔𝑒 𝑆𝑖𝑔𝑛𝑎𝑙 𝑇𝑜𝑡𝑎𝑙 𝑁𝑜𝑖𝑠𝑒 Increasing SNR Noise: Fluctuations in signal that produces uncertainty in the signal Larger SNR = easier to distinguish image from noise = higher confidence in measurements

CCD Primary Noise Sources SN = Shot (Photon) noise = signal (Physical property of light due to individual photons) DN = Dark noise = dark current (Temperature dependent) RN = Read noise (Gaussian distribution of values imparted by single output node/readout amplifier) -----use common terminology photon noise and signal

EMCCD Primary Noise Sources SN = Shot (Photon) noise = signal (Physical property of light) DN = Dark noise = dark current (Temperature dependent) RN = Read noise (Gaussian distribution of values imparted by single output node/readout amplifier) ENF = Excess noise factor = 1.4 (Caused by probability of creating new electrons due to electron multiplication) -----use common terminology photon noise and signal

CMOS Primary Noise Sources SN = Shot (Photon) noise = signal (Physical property of light, regardless of sensor) DN = Dark noise = dark current (Temperature dependent and higher for global shutter) RN = Read noise (This includes Random Telegraph Noise (RTN), which is non-Gaussian, and depends on multiply column and pixel amplifiers) -----use common terminology photon noise and signal

SNR Equation: CCDs S = Signal = Photon flux * time * QE DN = Dark noise = dark current RN = Read noise SN = Shot (Photon) noise = signal

SNR Equation: EMCCDs S = Signal = Photon flux * time * QE DN = Dark noise = dark current RN = Read noise SN = Shot (Photon) noise = signal ENF = Excess noise factor = 1.4 G = EM Gain factor (Typically up to ~1000x, effective read noise is reduced by this factor)

SNR Equation: CMOS S = Signal = Photon flux * time * QE DN = Dark noise = dark current RN = Read noise (This includes Random Telegraph Noise (RTN), a significant component of CMOS noise) SN = Shot (Photon) noise = signal -----use common terminology photon noise and signal

CMOS Random Telegraph Noise (Salt-and-Pepper Noise)

CMOS Random Telegraph Noise: Why it Happens SHR = Reference signal SHS = Sampled signal Normal signal Low signal due to e- trapped in pixel defect Correlated double sampling involves a reference being subtracted from the sample to remove drift. This works great if the reference and sample are at the same signal level but will cause a positive (bright pixel) or negative (dark pixel) whenever the reference is at a different level.

Skewed CMOS Read Noise Compared to Gaussian CCD Read Noise 40% outside of Gaussian fit 3% outside of Gaussian fit The CCD has a read noise distribution close to Gaussian The CMOS read noise distribution is skewed to much larger values due to the noisy pixels (RTN). It is skewed from Gaussian.

Skewed Distribution of Read Noise: How does it behave compared to Gaussian noise? Averaging frames is less effective in reducing noise for CMOS. What it means: A researcher will need to acquire more data with a CMOS (compared to CCD) to decrease the error bars by the same amount for low light images. Scientific CMOS Chip Bias Stack Subtracted Center Quadrant   # Frames Averaged Stdev Noise Reduction Expected Noise Reduction %Difference 1 3.347645 = SQRT(Frame #) 4 1.694679 0.51 0.50 9 1.153824 0.34 0.33 3 16 0.887285 0.27 0.25 6 25 0.732452 0.22 0.20 36 0.631351 0.19 0.17 13 49 0.562065 0.14 18 Traditional CCD 25.051838 12.496225 0.499 0.500 0.2 8.315734 0.332 0.333 0.4 6.235033 0.249 0.250 4.985057 0.199 0.200 0.5 4.159281 0.166 0.167 3.566815 0.142 0.143 0.3

Binning (in CCDs and EMCCDs) Binning provides the ability to combine pixels into a larger “super-pixel” before digitization of data. Boosts signal detection capability by increasing effective pixel size Traditional Analog Binning Signal is combined before digitization – only 1x RN is applied For 2x2 binning: 4x pixels of signal, 1x RN gives a 4:1 boost in signal to noise ratio

Binning in Scientific CMOS Binning with the scientific CMOS sensor is slightly different when compared to CCDs. In CMOS cameras, binning is applied after readout. So, read noise has already been introduced to each individual pixel before combining in software. Signal is combined after digitization and after RN has been applied to each pixel. Adding 4 pixels together reduces 4x RN to 2x RN. 4x pixels of signal, 2x RN gives only a 2:1 boost in signal to noise ratio

Advantages of conventional CCDs: - Flexibility Conventional CCDs (e.g. ICX 285) allow for greater flexibility and general purpose imaging than sCMOS or EMCCDs. Available in a variety of pixel sizes and formats Available in a wide variety of price points Accessibility of flexible binning Availability of deep-cooled, very low dark current variants

Advantages of EMCCDs: - Low-light fast EMCCDs offer the greatest possible sensitivity, with accessibility of very high speeds Available in back-illuminated frame transfer formats EM multiplication offers the lowest effective read noise Availability of binning Availability of deep-cooled, very low dark current variants Generally at higher price points

Advantages of sCMOS cameras: - high speed, high resolution sCMOS sensors allow for higher speed operation with similar noise performance to conventional CCDs sCMOS readouts allow for high speed operation (>30 fps) with large(er) pixel arrays than EM or conventional CCDs Potentially lower (different ?) noise than comparable ICX285 CCDs Moderate price points Novel technology with room to grow

Summary Conventional CCDs – flexible, workhorse Well established. Good combination of speed, sensitivity and resolution EMCCDs – low light (fast) Offers the ultimate sensitivity coupled with high speed when needed sCMOS Emerging technology, which offers high speed large format images