1. Introduction to Image Processing CS474/674 – Prof. Bebis

2. What is Image Processing? Manipulation of digital images by computer. Image processing focuses on two major tasks: –Improvement of pictorial information for human interpretation and high level processing. –Processing of image data for storage and transmission.

3. Related Areas Image Processing Computer Vision Computer Graphics

4. Image Processing

5. Image Enhancement

6. Image Processing (cont’d) Image Restoration

7. Image Processing (cont’d) Image Compression

8. Computer Graphics

9. Image Output: Geometric Models SyntheticCamera Projection, shading, lighting models

10. Computer Vision

11. Model Output: Real Scene CamerasImages

12. Applications: Image Enhancement One of the most common uses of IP techniques: improve quality, remove noise etc

13. Applications: Space Launched in 1990 the Hubble telescope can take images of very distant objects An incorrect mirror made many of Hubble’s images useless Image processing techniques were used to fix this!

14. Applications: Medicine Take slice from MRI scan of a dog’s heart, and find boundaries between different types of tissue –Image with gray levels representing tissue density –Use a suitable filter to highlight edges Original MRI image of a dog’s heart Edge detection image

15. Applications: GIS Geographic Information Systems –Digital image processing techniques are used extensively to manipulate satellite imagery. meteorology terrain classification

16. Applications: Industrial Inspection Human operators are expensive, slow and unreliable Make machines do the job instead! Industrial vision systems are used in all kinds of industries

17. Applications: Law Enforcement Image processing techniques are used extensively by law enforcers Fingerprint recognition Number plate recognition for speed cameras or automated toll systems

18. Examples: HCI Make Human Computer Interaction (HCI) more natural –Face recognition –Gesture recognition

19. Key Stages in Digital Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

20. Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

21. Image Enhancement Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

22. Image Restoration Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

23. Morphological Processing Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

24. Segmentation Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

25. Representation & Description Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

26. Object Recognition Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

27. Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression

28. Color Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Color Image Processing Image Compression

29. How are images represented in the computer?

30. Color images

31. A Simple model of image formation

32. What is (visible) light? The visible portion of the electromagnetic (EM) spectrum. –Approximately between 400 and 700 nanometers.

33. Examples: Gama-Ray Imaging Gamma-ray imaging: nuclear medicine and astronomical observations

34. Examples: X-Ray Imaging X-rays: medical diagnostics, industry, and astronomy, etc.

35. Examples: Ultraviolet Imaging Ultraviolet: industrial inspection, microscopy, lasers, biological imaging, and astronomical observations

36. Examples: Infrared Imaging Infrared bands: light microscopy, astronomy, remote sensing, industry, and law enforcement.

37. Sonic images Produced by the reflection of sound waves off an object. High sound frequencies are used to improve resolution.

38. Range images Can be produced by using laser range-finders. An array of distances to the objects in the scene.

39. Image formation There are two parts to the image formation process: –The geometry of image formation, which determines where in the image plane the projection of a point in the scene will be located. –The physics of light, which determines the brightness of a point in the image plane as a function of illumination and surface properties.

40. Pinhole camera This is the simplest device to form an image of a 3D scene on a 2D surface. Straight rays of light pass through a “pinhole” and form an inverted image of the object on the image plane.

41. Camera optics In practice, the aperture must be larger to admit more light. Lenses are placed in the aperture to focus the bundle of rays from each scene point onto the corresponding point in the image plane

42. Physics of Light f(x,y)=i(x,y)r(x,y) where 1)i(x,y) the amount of illumination incident to the scene 2)r(x,y) the reflectance from the object

43. CCD (Charged-Coupled Device) cameras Tiny solid state cells convert light energy into electrical charge. The image plane acts as a digital memory that can be read row by row by a computer.

44. Frame grabber Usually, a CCD camera plugs into a computer board (frame grabber). The frame grabber digitizes the signal and stores it in its memory (frame buffer).

45. Image digitization Sampling means measuring the value of an image at a finite number of points. Quantization is the representation of the measured value at the sampled point by an integer.

46. Image digitization (cont’d) 0 255

47. Image digitization (cont’d) 2D example

48. Effect of Image Sampling original image sampled by a factor of 2 sampled by a factor of 4 sampled by a factor of 8

49. Effect of Image Quantization 256 gray levels (8bits/pixel) 32 gray levels (5 bits/pixel) 16 gray levels (4 bits/pixel) 8 gray levels (3 bits/pixel) 4 gray levels (2 bits/pixel) 2 gray levels (1 bit/pixel)

50. Representing Digital Images The result of sampling and quantization is a matrix of integer numbers. Here we have an image f(x,y) that was sampled to produce M rows and N columns.

51. Representing Digital Images (cont’d) There is no requirements about M and N Usually L= 2 k Dynamic Range : [0, L-1] The number of bits b required to store an image: b = M x N x k where k is the number of bits/pixel

52. Image file formats Many image formats adhere to the following simple model: –Header –Data (line by line, no breaks between lines).

53. Image file formats (cont.) Header contains at least: –A signature or “magic number” (i.e., a short sequence of bytes for identifying the file format). –The width and height of the image.

54. Common image file formats PGM (Portable Gray Map) PNG (Portable Network Graphics) GIF (Graphic Interchange Format) – JPEG (Joint Photographic Experts Group) TIFF (Tagged Image File Format) FITS (Flexible Image Transport System)

55. PGM format A popular format for grayscale images (8 bits/pixel) Closely-related formats are: –PBM (Portable Bitmap), for binary images (1 bit/pixel) –PPM (Portable Pixelmap), for color images (24 bits/pixel) ASCII or binary (raw) storage ASCII Raw

56. Image Class class ImageType { public: ImageType(); // constructor ~ImageType(); // destructor void getImageInfo(int&, int&, int&); void setImageInfo(int, int, int); void setVal(int, int, int); void getVal(int, int, int&); // more functions... private: int N, M, Q; //N: # rows, M: # columns int **pixelValue; };

57. Input / Output Functions C++ routine to read the header of a PGM image: ReadImageHeader.cpp C++ routine to read a PGM image: ReadImage.cpp C++ routine to write a PGM image: WriteImage.cpp

58. An example - Threshold.cpp void readImageHeader(char[], int&, int&, int&, bool&); void readImage(char[], ImageType&); void writeImage(char[], ImageType&); void main(int argc, char *argv[]) { int i, j, M, N, Q; bool type; int val, thresh; // read image header readImageHeader(argv[1], N, M, Q, type); // allocate memory for the image array ImageType image(N, M, Q);

59. Threshold.cpp (cont’d) // read image readImage(argv[1], image); cout << "Enter threshold: "; cin >> thresh; // threshold image for(i=0; i<N; i++) for(j=0; j<M; j++) { image.getVal(i, j, val); if(val < thresh) image.setVal(i, j, 0; else image.setVal(i, j, 255); } // write image writeImage(argv[2], image); }

60. Reading/Writing PGM images (1D array of unsigned char) (2D array of int) Use “write” Use “read”

61. Writing a PGM image to a file void writeImage(char fname[], ImageType& image) int N, M, Q; unsigned char *charImage; ofstream ofp; image.getImageInfo(N, M, Q); charImage = (unsigned char *) new unsigned char [M*N]; // convert integer values to unsigned char int val; for(i=0; i<N; i++) for(j=0; j<M; j++) image.getVal(i, j, val); charImage[i*M+j]=(unsigned char)val; }

62. Writing a PGM image... (cont’d) ofp.open(fname, ios::out | ios::binary); if (!ofp) { cout << "Can't open file: " << fname << endl; exit(1); } ofp << "P5" << endl; ofp << M << " " << N << endl; ofp << Q << endl; ofp.write( reinterpret_cast (charImage), (M*N)*sizeof(unsigned char)); if (ofp.fail()) { cout << "Can't write image " << fname << endl; exit(0); } ofp.close(); }

63. Reading a PGM image from a file void readImage(char fname[], ImageType& image) { int i, j; int N, M, Q; unsigned char *charImage; char header [100], *ptr; ifstream ifp; ifp.open(fname, ios::in | ios::binary); if (!ifp) { cout << "Can't read image: " << fname << endl; exit(1); }

64. Reading a PGM image from a file // read header ifp.getline(header,100,'\n'); if ( (header[0]!=80) || // 'P' (header[1]!=53) ) { // '5' cout << "Image " << fname << " is not PGM" << endl; exit(1); } ifp.getline(header,100,'\n'); // skip comments while(header[0]=='#') ifp.getline(header,100,'\n'); M=strtol(header,&ptr,0); // read M, N N=atoi(ptr);

65. Reading a PGM image …. (cont’d) ifp.getline(header,100,'\n'); Q=strtol(header,&ptr,0); charImage = (unsigned char *) new unsigned char [M*N]; ifp.read( reinterpret_cast (charImage), (M*N)*sizeof(unsigned char)); if (ifp.fail()) { cout << "Image " << fname << " has wrong size" << endl; exit(1); } ifp.close();

66. Reading a PGM image…(cont’d) // Convert unsigned characters to integers int val; for(i=0; i<N; i++) for(j=0; j<M; j++) { val = (int)charImage[i*M+j]; image.setVal(i, j, val); }

67. How do I “see” images on my computer? Unix/Linux: xv, gimp Windows: Photoshop Irfanview

68. How do I convert an image from one format to another? Use “Save As” option

69. More Information on Image Processing and Computer Vision Computer Vision Home Page http://www.cs.cmu.edu/afs/cs/project/cil/ftp/html/vision.html UNR Computer Vision Laboratory http://www.cs.unr.edu/CVL