1. Instructor: Mircea Nicolescu Lecture 4 CS 485 / 685 Computer Vision

2. Alternative Color Spaces Various other color representations can be computed from RGB This can be done for: −Decorrelating the color channels: −principal components −Bringing color information to the fore: −Hue, saturation and brightness −Perceptual uniformity: −CIELuv, CIELab, … 2

3. RGB (CIE), RnGnBn (TV - N ational T elevision S tandard C ommittee ) XYZ (CIE) UVW (UCS de la CIE), U*V*W* (UCS modified by the CIE) YUV, YIQ, YCbCr YDbDr DSH, HSV, HLS, IHS Munsel color space (cylindrical representation) CIELuv CIELab SMPTE-C RGB YES (Xerox) Kodak Photo CD, YCC, YPbPr,... Alternative Color Spaces 3

4. Red Green Blue T -1 Processing Processing Strategy T Red Green Blue 4

5. Color Transformation - Examples 5

6. Skin Color RGBrg r g 6

7. Skin Detection M. Jones and J. Rehg, Statistical Color Models with Application to Skin Detection, International Journal of Computer Vision, 2002.Statistical Color Models with Application to Skin Detection 7

8. 8 Image File Formats Many image formats adhere to the simple model shown below (line by line, no breaks between lines) The header contains at least the width and height of the image Most headers begin with a signature or “magic number” – short sequence of bytes identifying the file format

9. 9 Common Image File Formats GIF (Graphic Interchange Format) PNG (Portable Network Graphics) JPEG (Joint Photographic Experts Group) TIFF (Tagged Image File Format) PGM (Portable Gray Map) FITS (Flexible Image Transport System) …

10. 10 PGM Format A popular format for grayscale images (8 bits/pixel) Closely-related formats are: −PBM (Portable Bitmap) – for binary images (1 bit/pixel) −PPM (Portable Pixelmap) – for color images (24 bits/pixel) ASCII or binary (raw) storage: ASCII: Binary:

11. Image Filtering 11 f(x,y)g(x,y) filtering

12. 12 Early Vision – One Image Classification of image operations −Spatial domain methods −Point Processing Transformations −Area/Mask Processing Transformations −Frame Processing Transformations −Geometric Transformations −Frequency domain methods

13. Image Filtering Methods Spatial Domain Frequency Domain (i.e., uses Fourier Transform) 13

14. Spatial Domain Methods f(x,y) g(x,y) f(x,y) 14

15. 15 Point Processing Methods The most primitive, yet essential, image processing operations Intensity transformations that convert an old pixel into a new pixel based on some predefined function Operate on a pixel based solely on that pixel’s value Used primarily for image enhancement Transformation function

16. 16 Point Processing Methods Identity transformationNegative transformation

17. 17 Point Processing Methods Contrast stretching / compression −Stretch gray-level ranges where we desire more information

18. 18 Point Processing Methods Thresholding

19. 19 Point Processing Methods Intensity-level slicing −Highlight a specific range of gray-levels only −Similar to double thresholding

20. 20 Point Processing Methods Non-linear transformations −We may use any function, provided that is gives a one-to-one or many-to-one (i.e., single-valued) mapping. Logarithmic −Useful for enhancing details in the darker regions of the image at the expense of detail in the brighter regions.

21. 21 Point Processing Methods Exponential −The effect is the reverse of that obtained with logarithmic mapping.

22. 22 Point Processing Methods Histogram equalization −Low contrast images are usually mostly dark, mostly bright, or mostly gray. −Good contrast images exhibit a wide range of pixel values (i.e., no single gray level dominates the image).

23. 23 Point Processing Methods Histogram equalization −The histogram of an image (i.e., a plot of the gray-level frequencies) provides important information regarding the contrast of an image −Histogram clustered at the low end: dark image −Histogram clustered at the high end: bright image −Histogram with a small spread: low contrast image −Histogram with wide spread: high contrast image

24. 24 Point Processing Methods Histogram equalization −is a transformation that stretches the contrast by redistributing the gray-level values uniformly −is fully automatic compared to other contrast stretching techniques

25. 25 Point Processing Methods Histogram equalization

26. Area Processing Methods Need to define: (1)Area shape and size (2) Operation output image 26

27. Area Shape and Size Area shape is typically defined using a rectangular mask Area size is determined by mask size e.g., 3x3, 5x5, 7x7, … Mask size is an important parameter 27

28. Operation Typically, linear combinations of pixel values −e.g., weigh pixel values and add them together Different results can be obtained using different weights −e.g., smoothing, sharpening, edge detection mask 28

29. Example MaskModified image data Local image neighborhood 29

30. Common Linear Operations Correlation Convolution 30

31. Correlation A filtered image is generated as the center of the mask visits every pixel in the input image g(i,j) h(i,j) f(i,j) filtered image nxn mask 31

32. Handling Pixels Close to Boundaries pad with zeroes or 0 0 0 ……………………….0 wrap around 32

33. Correlation – Example 33

34. Geometric Interpretation of Correlation Suppose x and y are two n-dimensional vectors: The dot product of x with y is defined as: cos(θ) measures the similarity between x and y using vector notation: x y θ 34

35. Correlation generalizes the notion of dot product: Normalized correlation (divide by lengths) Geometric Interpretation of Correlation 35

36. Normalized Correlation = mask Measure the similarity between images or parts of images 36

37. Application: TV Remote Control 37

38. Application: TV Remote Control 38

39. Application: TV Remote Control 39

40. Application: TV Remote Control 40

41. Application: TV Remote Control 41

42. Application: TV Remote Control 42

43. Normalized Correlation Traditional correlation cannot handle changes due to: size orientation shape (e.g., deformable objects) ? 43

44. Convolution Same as correlation, except that the mask is flipped, both horizontally and vertically h * f = f * h Notation: For symmetric masks (h(i,j)=h(-i,-j)), convolution is equivalent to correlation 44

45. Correlation/Convolution Examples Correlation: Convolution: 45

46. How to Choose the Mask Weights? Depends on the application Usually by sampling certain functions and their derivatives Gaussian 1 st derivative of Gaussian 2 nd derivative of Gaussian Good for image smoothing Good for image sharpening 46

47. Normalization of Mask Weights Sum of weights affects overall intensity of output image Positive weights −Normalize them so that they sum to one Both positive and negative weights −Should sum to zero (but not always) 1/91/16 47