ART 147 DIGITAL MATERIALS & PROCESSES Powerpoint No ART 147 DIGITAL MATERIALS & PROCESSES Powerpoint No. 2 Digital Image Formation

Digital Image Formation Capturing of Light from Sensor to Memory Card

Digital Image Formation Color Filter Array Photodiodes are monochrome devices, they are unable to tell the difference between different wavelengths of light (color). A mosaic pattern of color filters Color Filter Array is positioned on the top of the sensor to filter out the red, green, and blue components of light falling onto it.

Digital Image Formation Color Filter Array Each pixel measures only one primary color. Each sensor captures only 25% of the red and blue light and 50% of the green. The highest percentage was chosen for green because our eyes are most sensitive to this part of the visual spectrum.

Digital Image Formation Color Interpolation Missing information in each color layer (beige squares) has to be estimated by the software in the camera. The software estimates the missing information for each color channel and creates the missing information.

Digital Image Formation Color Interpolation

Digital Image Formation Light Sensitive Pixels Sensors consist of pixels and the main component of a pixel is the Photodiode which converts the energy of the incoming photons (light energy) into an electrical charge. Photodiode (Photon to Electron Converter) Photodiodes are monochrome devices. They are unable to tell the difference between different wavelengths of light (color).

Digital Image Formation Light Sensitive Pixels The electrical charge is converted to a voltage which is then amplified to a level at which it can be processed further by the Analog to Digital Converter. Photodiode (Photon to Electron Converter) Other Circuitry Electron to Voltage Converter Amplifier

Digital Image Formation Analog to Digital Converter The Analog to Digital Converter (ADC) classifies the analog voltage of each pixel into one of 256 levels (8 bit ADC) and assigns a digital value to it.

Digital Image Formation Review of How Sensors Work The Photodiode converts the energy of the incoming photons (light) into an electrical charge. The Electron to Voltage Converter converts the electrical charge into voltage. The Amplifier strengthens the voltage signal and transfers the signal to the ADC. The Analog to Digital Converter classifies the analog voltage into digital data and transfers the data to a buffer.

Digital Image Formation Buffer A buffer consists of RAM memory which temporarily holds the image information before it is written to the memory card. The buffer will determine: The time between shots How many shots that can be taken in sequence when shot using continuous shooting mode. (a burst)

Digital Image Formation In Camera Processing: RAW Images Sensors capture 10-16 bits of RAW data per pixel This data goes through Color Interpolation The RAW data is transferred to the camera’s buffer and then is written directly to the memory card. This RAW data does include the camera’s Meta Data information including camera settings, date, time, ect.

Digital Image Formation In Camera Processing: JPEG Images Sensor captures 10-16 bits of RAW data per pixel This data goes through: Color Interpolation Color temperature & White Balance adjustment Noise Reduction Tonal Curve (contrast) File Size Interpolation (compact cameras) Sharpening Compression reduces file size and image quality

Digital Image Formation In Camera Processing: Color Interpolation Color Interpolation estimates the missing information for each color channel and creates the missing information.

Digital Image Formation In Camera Processing: White Balance White Balance compensates for the color temperature that different light sources have that affect the color quality of the image. The camera uses the color white as a reference point and calculates all the other colors based on the white point. In Camera White Balance Settings Auto: Calculates a white balance based on the scene. However this system is fooled if the scene is dominated by one color. Light Source Specific: Daylight Cloudy Tungsten Fluorescent Flash

Digital Image Formation In Camera Processing: Image Noise Fixed Pattern Noise CMOS sensors have one amplifier per pixel, therefore the pixels cause “Fixed Pattern Noise”. But because it has a fixed pattern it can be reduced during in-camera image processing. Dark Current Noise Is created by the activity of the sensor itself when the camera is turned on. Increases with the temperature of the sensor Decreases as more light exposes the pixel Readout Noise Amplification is the main cause. Increases with speed (ISO)

Digital Image Formation Noise Summary Increases as you increase the ISO (sensitivity) of the camera Red and Blue channels will have more noise because these channels are amplified more than the green channel. Why? Noise is most visible in uniform surfaces such as blue sky and shadows. Increase with temperature. Increases as pixel size decreases. Professional grade cameras use more advance noise removal algorithms to reduce noise.

Digital Image Formation In Camera Processing: Tonal Curves A Tonal Curve is applied to the linear RAW data so that the data is similar to the perception of human vision. Two types of tonal curve adjustment: Gamma Correction to adjust for contrast. Dynamic Range Compression to compensate for the characteristics of human vision. (Dynamic Range = Total image tonality from shadows to highlights) Sensor: Linear Human Vision: Non Linear

Digital Image Formation Tonal Curve Summary Sensors respond to light in a linear way. Human perception of light is nonlinear. Human vision “amplifies” the shadows and “compresses” the highlights Gamma corrects tonality by changing the overall contrast. Dynamic Range compresses the data to better match human vision. Tonal Curves can be applied to ‘RAW” images using conversion software during image processing. This allows you the control of the full dynamic range of the sensor.

Digital Image Formation File Size Interpolation Interpolation = Upsampling = Resampling Some digital cameras use interpolation to produce a larger image than the senor can capture. Interpolation increases the number of pixels in an image Example: the camera interpolates 6 million pixels of information based on the measurement of 3 million pixels on the sensor. This in-camera enlargement is of better quality than those performed on your computer because it is done before JPEG compression. Also used in “digital zoom”. Most common use of in-camera interpolation today in compact digital cameras.

Digital Image Formation File Size Interpolation Interpolation = Up-Sampling = Re-Sampling Determines the value of a new pixel based on a weighted average of the 4 pixels in the nearest 2X2 neighborhood of the pixel in the original capture.

Digital Image Formation Sharpening Optical Sharpness is defined by the quality of the lens and the sensor. Software Sharpness will create an ‘optical illusion” of sharpness by increasing the contrast along the edges of shapes within the image. (In-camera or Software processing) Unable to create detail beyond the camera’s resolution. It only helps to bring out captured detail. Very hard to undo over sharpening. It is recommended when shooting in JPEG to apply a low or normal in-camera sharpening. RAW images can be sharpened or unsharpened during image processing.

Digital Image Formation Sharpening

Digital Image Formation Image Conversion & Compression TIFF Lossless Conversion: No information is lost in the process. Separates the image information into categories and assigns tags for each category. JPEG Lossy Compression: Reduces the file size by discarding information. Separates the image into color and detail information. Compresses color information more than detail information. Sorts the detail information into fine and coarse detail Discards the fine detail first because our eyes are more sensitive to coarse detail

Digital Image Formation Image Conversion & Compression Uncompressed “RAW” Image File On Monday the weather was sunny On Tuesday the weather was sunny On Wednesday the weather was sunny On Thursday the weather was sunny On Friday the weather was sunny On Saturday the weather was sunny On Sunday the weather was sunny

Digital Image Formation Image Conversion & Compression Example of Image Compression Lossless “TIFF” Image File 1 Mon 2 3 4 Lookup Table 1 Tues 2 3 4 1 Wednes 2 3 4 whereby 1 = On 1 Thurs 2 3 5 2 - day 1 Fri 2 3 4 3 = the weather was 1 Satur 2 3 4 4 = sunny 1 Sun 2 3 4 5 = rainy

Digital Image Formation Image Conversion & Compression Example of Image Compression Lossy “JPEG” Image File Last week the weather was mostly sunny

Digital Image Formation Image Conversion & Compression NUMERICAL EXAMPLES OF FILE SIZES Uncompressed TIFF = 14.1 MB Uncompressed RAW = 7.7 MB 100% Quality JPEG = 2.3 MB 80% Quality JPEG = 1.3 MB 60% Quality JPEG = 0.7 MB 20% Quality JPEG = 0.2 MB

Digital Image Formation In Camera Processing Special Note: Camera’s Display No matter what file format type all image data is sent through Color Interpolation, Color Temperature & White Balance settings for the image preview on the camera’s LCD but this information is temporary and not written to the image file.

Digital Image Formation Sensors & Pixels

Digital Image Formation Sensor Types

Digital Image Formation Sensor Types CMOS Sensors: Complementary Metal Oxide Semiconductor Active Pixel Sensor: the captured light is converted into voltage and amplified within the pixel itself. RGB is recorded using a Color Filter Array. Based on a grid of square pixels Uses color interpolation Plus the analog to digital conversion is completed within the sensor itself.

Digital Image Formation CMOS Sensors

Digital Image Formation Sensor Types CCD Sensors: Charged Couple Device Interline Transfer CCD: Compact Cameras & DSLR: Canon-APS-C, Nikon-DX, Panasonic-4/3rds. Frame Transfer CCD: Full frame or larger size sensors. The captured light is converted into voltage and amplified for the entire sensor at one time. RGB is recorded using a Color Filter Array. Based on a grid of square pixels Uses color interpolation

Digital Image Formation Interline Transfer CCD Transfers the information one line at a time. Resulting in slower operation.

Digital Image Formation Foveon X3 Sensor Three sensor layers stacked on top of each other RGB is recorded by using frequencies for each layer. No color filter array No color interpolation Based on a grid of square pixels Currently used in Sigma SLR’s

Digital Image Formation

Digital Image Formation Sensor Size Imaging sensors come in different sizes, ranging from smaller than the nail on a baby’s pinky finger to almost the size of 645 format film.

Digital Image Formation Sensor Size Compact Point & Shoot Canon APS-C Nikon DX Panasonic Micro 4/3’s Canon: Full Frame Nikon: FX