EE 7700 Color

Bahadir K. Gunturk2 References On Color: Wikipedia, Gonzalez, Poynton, many others… On HDR: Slides and papers by Debevec, Ward, Pattaniak, Nayar, Durand, et al…

Bahadir K. Gunturk3 Color Color is a perceptual property. It comes from the spectrum of light (energy distribution of light versus wavelength) interacting with the spectral sensitivities of the light receptors (photoreceptors) in the eye.

Bahadir K. Gunturk4 Human Visual System Human visual system is sensitive to a narrow range of the electromagnetic spectrum. (Approximately from 380nm to 740nm.)

Bahadir K. Gunturk5 Human Visual System The diameter of the eyeball is around 22mm. Retina is a thin layer of neural cells that lines the back of the eyeball. Retina contains photoreceptors (rods and cons) that respond to light. Fovea is the most sensitive part of the retina; it is responsible for our sharp central vision. Some birds (such as hawks) have more than one fovea. (two). The axons (coming from receptors) exit the eye at the optic disc (blind spot), forming the optic nerve. There are 1.2million axons in the optic nerve. There are 130million photoreceptors.  A large amount of pre-processing is done within the retina. 10% of the axons are devoted to the fovea area.

Bahadir K. Gunturk6 0.5mm

Bahadir K. Gunturk7 Human Visual System There are two classes of receptors: cones and rods. Cones:  Sensitive to color (there are three cone types in humans)  Produces high-resolution vision  6-7 million cone receptors, located primarily in the central portion of the retina Rods:  Not involved in color vision  million rod receptors, distributed over retina  Sensitive to low levels of illumination. Not effective in bright light.  Produces lower-resolution vision

Bahadir K. Gunturk8 Human Visual System There are three types of cones in humans A side note: Humans and some monkeys have three types of cones (trichromatic vision); most other mammals have two types of cones (dichromatic vision). Marine mammals have one type of cone. Most birds and fish have four types. Lacking one or more type of cones result in color blindness. Human lens and cornea are increasingly absorvative to smaller wavelengths, which sets wavelength sensitivity limit to around 380nm. Humans lacking lens reported to see ultraviolet. 65% sensitive to Long-wavelength 33% sensitive to Medium 2% sensitive to Small

Bahadir K. Gunturk9 Human Visual System Light is reduced to three color components by the eye. These values are called tristimulus values. The set of all possible tristimulus values determines the human color space. It is estimated that humans can distinguish around 10million colors. The mechanisms of color vision within the retina are explained well in terms of tristimulus values. The way the values sent out of eye is little different: A dominant theory says that color is sent out of the eye in three opponent channels: a red-green channel, a blue-yellow channel and a black-white "luminance" channel. These channels are constructed from the tristimulus values.

Bahadir K. Gunturk10 Human Visual System Color constancy (Chromatic adaptation): The perceived color of objects remains relatively constant under varying illumination conditions. This helps us identify objects. A red apple appears red in sunlight, at sunset, in florescent illumination, etc. Of course, this works only if the illumination contains a range of wavelengths. The HVS determines the approximate composition of the illuminating light, and then discounted to obtain the objects “true color” or reflectance.

Bahadir K. Gunturk11 Human Visual System Which square is darker? A or B?

Bahadir K. Gunturk12 Human Visual System

Bahadir K. Gunturk13 Human Visual System

Bahadir K. Gunturk14 A Color Blindness Test

Bahadir K. Gunturk15 Human Visual System Colors consisting of a single wavelength are called pure spectral or monochromatic colors. Most light sources are mixtures of various wavelengths of light. If they produce a similar stimulus in the eye, a non-monochromatic light source can be perceived as a monochromatic light. For a non-monochromatic light source, we may talk about the dominant wavelength (or color), which identifies the single wavelength of light that produces the most similar sensation. Of course, there are many color perceptions that cannot be identified by pure spectral colors, such as pink, tan, magenta, achromatic colors (black, gray, white).

Bahadir K. Gunturk16 Human Visual System Two different light spectra that have the same effect on the three color receptors will be perceived as the same color. Most human color perceptions can be generated by a mixture of three colors, called primaries. This is used to reproduce color in photography, printing, TV, etc.

Bahadir K. Gunturk17 CIE In 1931, the Commission Internationale de l’Eclairage (CIE) established standards for color representation. Subjects were shown color patches and asked to match the color by adjusting three monochromatic colors. Based on the experiments, they defined the color-matching-functions:

Bahadir K. Gunturk18 Tristimulus Let X, Y, and Z be the tristimulus values. A color can be specified by its trichromatic coefficients, defined as X ratio Y ratio Z ratio Two trichromatic coefficients are enough to specify a color. (x + y + z = 1)

Bahadir K. Gunturk19 CIE Chromaticity Diagram Input light spectrum x y

Bahadir K. Gunturk20 CIE Chromaticity Diagram Input light spectrum x y

Bahadir K. Gunturk21 CIE Chromaticity Diagram Input light spectrum Boundary x y 380nm 700nm

Bahadir K. Gunturk22 CIE Chromaticity Diagram Input light spectrum Boundary

Bahadir K. Gunturk23 CIE Chromaticity Diagram Light composition

Bahadir K. Gunturk24 CIE Chromaticity Diagram Light composition

Bahadir K. Gunturk25 CIE Chromaticity Diagram The CIE chromaticity diagram shows the human color space as a function of x and y. Boundary indicates the pure spectrum colors. (Full saturation.) Inside the boundary shows mixture of spectrum colors. Boundary

Bahadir K. Gunturk26 CIE Chromaticity Diagram The CIE chromaticity diagram is helpful to determine the range of colors that can be obtained from any given colors in the diagram. Source: Gamut: The range of colors that can be produced by the given primaries.

Bahadir K. Gunturk27 CIE Chromaticity Diagram Green: ColorMatch primaries, D50 Orange: sRGB primaries, D65 R’G’B’: Gamma corrected values Green: Corresponding RGB with gamma 1.8 Orange: … with gamma 2.2

Bahadir K. Gunturk28 Mixtures of Light The primary colors (primaries) can be added to produce the secondary colors of light. Example: Color TV displays use this additive nature of colors. An electron gun hits red, green, blue phosphors (with different energies) in a small region to produce different shades of color.

Bahadir K. Gunturk29 Mixtures of Light In printing, subtractive primaries are used: Cyan absorbs only Red. Magenta absorbs only Green. Yellow absorbs only Blue. In printing, dark colors may be obtained by addition of black ink. Such color systems are known as CMYK systems. Y M C

Bahadir K. Gunturk30 Color Space A color space relates numbers to actual colors; it contains all realizable color combinations. A color space could be device-dependent or device-independent. An RGB color space has three components: Red, Green, and Blue. But, it does not specify the exact color unless Red, Green, and Blue are defined. The sRGB is a device-independent color space. It was created in 1996 by HP and Microsoft for use on monitors and printers. It is the most commonly used color space. R G B

Bahadir K. Gunturk31 Color Space The Adobe RGB is developed by Adobe in It was designed for printers; it has a wider gamut than sRGB.

Bahadir K. Gunturk32 Color Space HSV color space defines color in terms of Hue, Saturation, and Value. Hue is the color type (such as, red, blue, yellow). (0-360 degrees) Saturation is the purity of the color. (0-100%) Value is the brightness of the color. (0-100%) HSV is not device-independent. It is defined in terms of RGB intensities. It is commonly used in computer graphics applications.

Bahadir K. Gunturk33 Color Space YUV color space defines color in terms of one luminance (brightness) and two chrominance (color) components. YUV is created from RGB components. YUV YCbCr

Bahadir K. Gunturk34 Color Space Input device Output device Color space conversion International Color Consortium (ICC) was established in 1993 to create an open color management system. The system involves three things: color profiles, color spaces, and color space conversion. The color profile keeps track of what colors are produces for a particular device’s RGB or CMYK numbers, and maps these colors as a subset of the “profile connection space”. Profile Connection Space

Bahadir K. Gunturk35 Color Space Input device Output device Color space conversion When there is gamut mismatch, There should be color rendering. Profile Connection Space

Bahadir K. Gunturk36 CIELAB (CIE L*a*b*) It was found that CIExyz is not a perceptually uniform color space: The minimum distance between two discernable colors differs in different parts of the CIExyz diagram. Perceptually linear means that a change of the same amount in a color value should produce a change of about the same visual importance. When storing colors in limited precision values, this can improve the reproduction of tones. L*a*b* color space was defined in Conversion from XYZ to L*a*b* is for otherwise Xn, Yn and Zn are the CIE XYZ values of the reference white point.white point

Bahadir K. Gunturk37 White Point A white point is the reference point to define the color “white”. Primaries plus the white point (indicating power ratio of primaries) should be given. Depending on the application, different definitions of white are needed to get acceptable results. For example, photographs taken indoors may be lit by incandescent light, which are relatively orange compared to daylight. Defining “white” as daylight will give unacceptable results when attempting to color-correct a photograph. A list of common white points: NamexyNotes E1/3 Equal energy D TV, sRGB color space A Incandescent tungsten

Bahadir K. Gunturk38 High Dynamic Range (HDR) Imaging star light moon light office light day light search light The range of radiances is more than 10^12 candela/m2 Range of human eye at an instant is around 10^4:1 (4log units) Human eye can adapt to see much wider range. Candela is the unit of luminous intensity (power emitted by a light source in a particular direction, with wavelengths weighted by the sensitivity of the human eye.

Bahadir K. Gunturk39 HDR star light moon light office light day light search light The range of radiances is more than 10^12 candela/m Range of Typical Displays: from ~1 to ~100 cd/m 2

Bahadir K. Gunturk cd/m^2 Cone dominated log L a Gain rod cone log Gain Sensitivity of Eye

Bahadir K. Gunturk cd/m^2 Rod dominated log L a Gain rod cone log Gain Sensitivity of Eye

Bahadir K. Gunturk42 Sensitivity of Eye

Bahadir K. Gunturk43 HDR The range of image capture devices is also low

Bahadir K. Gunturk44 HDR The range of image capture devices is also low

Bahadir K. Gunturk45 HDR HDR image rendered to be displayed on a LDR display.

Bahadir K. Gunturk46 HDR Problems: How to capture an HDR image with LDR cameras? How to display an HDR image on LDR displays?

Bahadir K. Gunturk47 Capture multiple images with varying exposure. Combine them to produce an HDR image.

Bahadir K. Gunturk48 Creating HDR from Multiple Pictures Measured intensity, z t1 t2 t1 t2 Irradiance, E

Bahadir K. Gunturk49 Creating HDR from Multiple Pictures Measured intensity, z t1 t2 t1 t2 Irradiance, E z1 z2 E z1 = t1 * E z2 = t2 * E E1=z1/t1 E2=z2/t2 Estimates: Take a weighted sum of E1 and E2: w1 w2 E=( w1*E1 + w2*E2 ) / (w1+w2) E

Bahadir K. Gunturk50 Creating HDR from Multiple Pictures Measured intensity, z t1 t2 t1 t2 Irradiance, E z1 z2 E z1 = t1 * E z2 = t2 * E E1=z1/t1 E2=z2/t2 Estimates: Take a weighted sum of E1 and E2: w E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2)) z

Bahadir K. Gunturk51 Creating HDR from Multiple Pictures In general, the camera response is not linear. t1 t2 z1 = f ( t1 * E ) z2 = f ( t2 * E ) E1= g (z1) / t1 E2= g (z2) / t2 E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2)) f g w z z Questions: How to estimate g and t? w is sometimes chosen as the derivative of f. (Mann)

Bahadir K. Gunturk52 Radiometric Self Calibration Polynomial model Exposure ratios: Cost function Solve using If exposure ratios are not known, solve iteratively (Nayar)

Bahadir K. Gunturk53 Tone Mapping Given an HDR image, how are we going to display it in an LDR display?

Bahadir K. Gunturk54 Tone Mapping Given an HDR image, how are we going to display it in an LDR display? Linear Nonlinear

Bahadir K. Gunturk55 Durand & Dorsey

Bahadir K. Gunturk56 Durand & Dorsey

Bahadir K. Gunturk57 Durand & Dorsey

Bahadir K. Gunturk58 Durand & Dorsey

Bahadir K. Gunturk59 Durand & Dorsey

Bahadir K. Gunturk60 Durand & Dorsey  Bilateral filter

Bahadir K. Gunturk61 Durand & Dorsey

Bahadir K. Gunturk62 Durand & Dorsey

Bahadir K. Gunturk63 Durand & Dorsey

Bahadir K. Gunturk64 Spatially Varying Exposures Instead of capturing multiple pictures, allow different amounts of light pass for different pixel positions. Estimate the missing pixels. Combine to obtain an HDR image. 100%75% 50%25% Nayar

Bahadir K. Gunturk65 Image Reconstruction: Interpolation

Bahadir K. Gunturk66 Image Reconstruction: Aggregation

Bahadir K. Gunturk67 HDR image examples

Bahadir K. Gunturk68 HDR image examples

Bahadir K. Gunturk69 HDR image examples

Bahadir K. Gunturk70 Retinex Image Processing Received intensity is a product of illuminance and reflectance: I = L*R Illumination components changes slowly. Reflectance component changes fast. Take the logarithm of I: log(I) = log(L) + log(R) Apply a high-pass filter to obtain the reflectance. Homomorphic filter Multi-scale retinex

Bahadir K. Gunturk71 Retinex Image Processing

Bahadir K. Gunturk72 Retinex Image Processing

Bahadir K. Gunturk73 Retinex Image Processing

Bahadir K. Gunturk74 Retinex Image Processing Vivek Agarwal

Bahadir K. Gunturk75 Retinex Image Processing