CS 376b Introduction to Computer Vision 02 / 05 / 2008 Instructor: Michael Eckmann

Michael Eckmann - Skidmore College - CS 376b - Spring 2008 Today’s Topics finish up introduction to openCV mostly material from chapter 2 (please read ch. 2)‏ –Electromagnetic radiation, color, visible light, human eye capabilities –Image acquisition/formation and representation CCDs etc. video problems with digital images some definitions –RGB color model –quantization problems

Michael Eckmann - Skidmore College - CS Spring 2008 Electromagnetic radiation Physically, color is electromagnetic radiation in the visible light frequency range. Psychologically, how humans perceive color is more than just the frequency. Hue (dominant wavelength), brightness (amount of light energy) and purity (how little white light is added) are what characterize what color is seen.

Michael Eckmann - Skidmore College - CS Spring 2008 Electromagnetic radiation Frequency range of visible light is approx. from: 4.3*10^14 Hz up to 7.9*10^14 Hz

Michael Eckmann - Skidmore College - CS Spring 2008 Electromagnetic radiation Frequency is measured in Hertz (Hz) which is 1 / seconds Wavelength is often used to describe colors where the wavelength = speed of light / frequency Speed of light in a vaccum is approx. 3*10^8 m / s So, visible light’s wavelength range is approximately: –wavelength of the low frequency end 3.8 * 10^14 Hz is: 3*10^8 m/s / 4.3 * 10^14 Hz = 697 nm (Red end)‏ –wavelength of the high frequency end of 7.9 * 10^14 Hz is: 3*10^8 m/s / 7.9 * 10^14 Hz = 379 nm (Violet end)‏

Electromagnetic radiation Human eye senses radiation wavelengths between approx nanometers radiation with wavelengths less than visible light –Ultraviolet, X-rays radiation with wavelengths greater than visible light –Infrared, Microwaves, TV, Radio There are sensors to detect these various non-visible wavelengths –just today I read about a terahertz radiation sensor – –see paragraph starting with “The rays permeate” Michael Eckmann - Skidmore College - CS 376b - Spring 2008

Michael Eckmann - Skidmore College - CS Spring 2008 Color

Michael Eckmann - Skidmore College - CS Spring 2008 Color The last slide shows a color that is reddish and fairly pure due to the fact that E D is >> E W. If the graph contained approximately the same amount of energy of each frequency then it would represent white. how bright it is correlates to amount of energy –brighter implies more light energy how pure it is correlates to how much white light is added –more pure implies less white light

Michael Eckmann - Skidmore College - CS Spring 2008 Color The tristimulus theory of color perception states that the retina in the human eye has three kinds of color sensors (cones) with peak sensitivies to red, blue and green light. –Cones (6-8 million in retina, concentrated in fovea) for color / bright light vision –Rods (over 100 million in retina, none in fovea) for achromatic brightness levels / night vision –fovea is in center of retina --- that's where most of the cones are –Let's look here for the rod & cone distribution throughout the retina:

Michael Eckmann - Skidmore College - CS Spring 2008 Color The tristimulus theory of color perception –Experiments have shown that based on this theory that “blue”'s peak is 440 nm, “green”'s is 545 nm, and “red”'s is 580 nm. –Also, experiments have shown that the eye's response to blue light is much less strong than to red and green. According to the website on the last slide, “throughout the whole retina the ratio of L- and M- cones to S-cones is about 100:1.” L-cones are long wavelength which I referred to as red. M-cones are medium wavelength which I referred to as green. S-cones are short wavelength which I referred to as blue. so the “red” and “green” cones together, outnumber the “blue” cones 100 to 1

Michael Eckmann - Skidmore College - CS Spring 2008 Imaging devices x-ray images –can penetrate the human body/bones infrared (IR) images (higher temperatures emit higher infrared radiation)‏ –often used for night photography film camera digital camera –usually a CCD with cells in a rectangular grid but not necessarily If the device does not generate a digital image, then it needs to be converted into one Also, if the device does not detect visible light, typically the different values captured by the devices are converted to visible colors.

Michael Eckmann - Skidmore College - CS Spring 2008 CCD cameras image figure 2.2 in “Computer Vision” by Shapiro and Stockman discrete cells convert light energy into electrical charges and they integrate the light energy falling on them during the time the shutter is open

Michael Eckmann - Skidmore College - CS Spring 2008 CCD arrays image figure 2.4 in “Computer Vision” by Shapiro and Stockman various array geometries (b) is linear (e.g. a moving scanner) (c) is for integrating the light in various sectors or bands --- has use in quantizing power spectrum images (see chap. 5)‏

Michael Eckmann - Skidmore College - CS Spring 2008 Video 30 frames (images) per second for NTSC video –can be a different rate of images e.g. 15, 60 almost always compressed due to the large size –example: a 640 x 480 image with 3 bytes per pixel uncompressed is how big? a 90 minute video w/ 30 frames per second uncompressed with each frame as above is how big? COTS (commercial off the shelf) CCD video cameras are made for other purposes than for computer vision –e.g. pixels may not be square (4:3 ratio)‏ computer vision most likely prefers/assumes square pixels

Michael Eckmann - Skidmore College - CS Spring 2008 Problems in digital images geometric distortion due to an imperfect lens the light beams don't travel on the correct path e.g. barrel distortion for small focal lengths scattering medium that the light travels through is rarely a vacuum, hence the light bends (e.g. air, water, other liquids...)‏ blooming each CCD cell is close to their neighbors so, light can leak between neighboring cells CCD variations some cells might not be all equally sensitive due to variations in the manufacturing process

Michael Eckmann - Skidmore College - CS Spring 2008 Problems in digital images clipping or wrap-around when an analog value is converted to digital, high values may be “clipped” to the max digital value, wrap around is like a modulus operator quantization effects converting a continuous range to a discrete range chromatic distortion lens has the index of refraction (bending) different for different wavelengths (colors) let's look back at the lightbulb picture, what's the effect?

Michael Eckmann - Skidmore College - CS Spring 2008 Problems in digital images images from “Computer Vision” by Shapiro and Stockman Blooming(left) & Barrel distortion (right)‏

Michael Eckmann - Skidmore College - CS Spring 2008 some definitions analog image –“infinite” precision in space and intensity value digital image –discrete 2D array of limited precision intensity values grey-scale image –one intensity per pixel multispectral image –a vector of values at each pixel (for color, usually 3 values representing values for red, green and blue)‏ binary image –each pixel has value 0 or 1

Michael Eckmann - Skidmore College - CS Spring 2008 some definitions labelled image –all pixel values are from some finite alphabet –usually generated from a digital image based on some way to decide which label a pixel gets –example on board picture function –f(x,y) where x and y are spatial variables and f(x,y) is the intensity at x,y

Michael Eckmann - Skidmore College - CS Spring 2008 RGB Color Model Color = rR + gG + bB r, g and b range from 0.0 to 1.0 notice: white's (r,g,b) = (1,1,1)‏ black's (r,g,b) = (0,0,0)‏ complementary colors are those that add up to (1,1,1)‏ can anyone tell me 2? image from “Computer Graphics” by Hearn and Baker

from Shapiro & Stockman figure 2.9 in “Computer Vision” quantization problems example