Introduction to Image Processing CS474/674 – Prof. Bebis

What is Image Processing? Manipulation of digital images by computer. Image processing focuses on two major tasks: –Improvement of pictorial information for human interpretation and high level processing. –Processing of image data for storage and transmission.

Related Areas Image Processing Computer Vision Computer Graphics

Image Processing

Image Enhancement

Image Processing (cont’d) Image Restoration

Image Processing (cont’d) Image Compression

Computer Graphics

Image Output: Geometric Models SyntheticCamera Projection, shading, lighting models

Computer Vision

Model Output: Real Scene CamerasImages

Applications: Image Enhancement One of the most common uses of IP techniques: improve quality, remove noise etc

Applications: Space Launched in 1990 the Hubble telescope can take images of very distant objects An incorrect mirror made many of Hubble’s images useless Image processing techniques were used to fix this!

Applications: Medicine Take slice from MRI scan of a dog’s heart, and find boundaries between different types of tissue –Image with gray levels representing tissue density –Use a suitable filter to highlight edges Original MRI image of a dog’s heart Edge detection image

Applications: GIS Geographic Information Systems –Digital image processing techniques are used extensively to manipulate satellite imagery. meteorology terrain classification

Applications: Industrial Inspection Human operators are expensive, slow and unreliable Make machines do the job instead! Industrial vision systems are used in all kinds of industries

Applications: Law Enforcement Image processing techniques are used extensively by law enforcers Fingerprint recognition Number plate recognition for speed cameras or automated toll systems

Examples: HCI Make Human Computer Interaction (HCI) more natural –Face recognition –Gesture recognition

Key Stages in Digital Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Image Enhancement Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Image Restoration Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Morphological Processing Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Segmentation Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Representation & Description Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Object Recognition Image Acquisition Image Restoration Morphological Processing Segmentation Object Recognition Image Enhancement Representation & Description Problem Domain Colour Image Processing Image Compression

Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Colour Image Processing Image Compression

Color Image Processing Image Acquisition Image Restoration Morphological Processing Segmentation Representation & Description Image Enhancement Object Recognition Problem Domain Color Image Processing Image Compression

How are images represented in the computer?

Color images

A Simple model of image formation

What is (visible) light? The visible portion of the electromagnetic (EM) spectrum. –Approximately between 400 and 700 nanometers.

Examples: Gama-Ray Imaging Gamma-ray imaging: nuclear medicine and astronomical observations

Examples: X-Ray Imaging X-rays: medical diagnostics, industry, and astronomy, etc.

Examples: Ultraviolet Imaging Ultraviolet: industrial inspection, microscopy, lasers, biological imaging, and astronomical observations

Examples: Infrared Imaging Infrared bands: light microscopy, astronomy, remote sensing, industry, and law enforcement.

Sonic images Produced by the reflection of sound waves off an object. High sound frequencies are used to improve resolution.

Range images Can be produced by using laser range-finders. An array of distances to the objects in the scene.

Image formation There are two parts to the image formation process: –The geometry of image formation, which determines where in the image plane the projection of a point in the scene will be located. –The physics of light, which determines the brightness of a point in the image plane as a function of illumination and surface properties.

Pinhole camera This is the simplest device to form an image of a 3D scene on a 2D surface. Straight rays of light pass through a “pinhole” and form an inverted image of the object on the image plane.

Camera optics In practice, the aperture must be larger to admit more light. Lenses are placed in the aperture to focus the bundle of rays from each scene point onto the corresponding point in the image plane

Physics of Light f(x,y)=i(x,y)r(x,y) where 1)i(x,y) the amount of illumination incident to the scene 2)r(x,y) the reflectance from the object

CCD (Charged-Coupled Device) cameras Tiny solid state cells convert light energy into electrical charge. The image plane acts as a digital memory that can be read row by row by a computer.

Frame grabber Usually, a CCD camera plugs into a computer board (frame grabber). The frame grabber digitizes the signal and stores it in its memory (frame buffer).

Image digitization Sampling means measuring the value of an image at a finite number of points. Quantization is the representation of the measured value at the sampled point by an integer.

Image digitization (cont’d) 0 255

Image digitization (cont’d) 2D example

Effect of Image Sampling original image sampled by a factor of 2 sampled by a factor of 4 sampled by a factor of 8

Effect of Image Quantization 256 gray levels (8bits/pixel) 32 gray levels (5 bits/pixel) 16 gray levels (4 bits/pixel) 8 gray levels (3 bits/pixel) 4 gray levels (2 bits/pixel) 2 gray levels (1 bit/pixel)

Representing Digital Images The result of sampling and quantization is a matrix of integer numbers. Here we have an image f(x,y) that was sampled to produce M rows and N columns.

Representing Digital Images (cont’d) There is no requirements about M and N Usually L= 2 k Dynamic Range : [0, L-1] The number of bits b required to store an image: b = M x N x k where k is the number of bits/pixel

Image file formats Many image formats adhere to the following simple model: –Header –Data (line by line, no breaks between lines).

Image file formats (cont.) Header contains at least: –A signature or “magic number” (i.e., a short sequence of bytes for identifying the file format). –The width and height of the image.

Common image file formats PGM (Portable Gray Map) PNG (Portable Network Graphics) GIF (Graphic Interchange Format) – JPEG (Joint Photographic Experts Group) TIFF (Tagged Image File Format) FITS (Flexible Image Transport System)

PGM format A popular format for grayscale images (8 bits/pixel) Closely-related formats are: –PBM (Portable Bitmap), for binary images (1 bit/pixel) –PPM (Portable Pixelmap), for color images (24 bits/pixel) ASCII or binary (raw) storage ASCII Raw

Image Class class ImageType { public: ImageType(); // constructor ~ImageType(); // destructor void getImageInfo(int&, int&, int&); void setImageInfo(int, int, int); void setVal(int, int, int); void getVal(int, int, int&); // more functions... private: int N, M, Q; //N: # rows, M: # columns int **pixelValue; };

Input / Output Functions C++ routine to read the header of a PGM image: ReadImageHeader.cpp C++ routine to read a PGM image: ReadImage.cpp C++ routine to write a PGM image: WriteImage.cpp

An example - Threshold.cpp void readImageHeader(char[], int&, int&, int&, bool&); void readImage(char[], ImageType&); void writeImage(char[], ImageType&); void main(int argc, char *argv[]) { int i, j, M, N, Q; bool type; int val, thresh; // read image header readImageHeader(argv[1], N, M, Q, type); // allocate memory for the image array ImageType image(N, M, Q);

Threshold.cpp (cont’d) // read image readImage(argv[1], image); cout << "Enter threshold: "; cin >> thresh; // threshold image for(i=0; i<N; i++) for(j=0; j<M; j++) { image.getVal(i, j, val); if(val < thresh) image.setVal(i, j, 0; else image.setVal(i, j, 255); } // write image writeImage(argv[2], image); }

Reading/Writing PGM images (1D array of unsigned char) (2D array of int) Use “write” Use “read”

Writing a PGM image to a file void writeImage(char fname[], ImageType& image) int N, M, Q; unsigned char *charImage; ofstream ofp; image.getImageInfo(N, M, Q); charImage = (unsigned char *) new unsigned char [M*N]; // convert integer values to unsigned char int val; for(i=0; i<N; i++) for(j=0; j<M; j++) image.getVal(i, j, val); charImage[i*M+j]=(unsigned char)val; }

Writing a PGM image... (cont’d) ofp.open(fname, ios::out | ios::binary); if (!ofp) { cout << "Can't open file: " << fname << endl; exit(1); } ofp << "P5" << endl; ofp << M << " " << N << endl; ofp << Q << endl; ofp.write( reinterpret_cast (charImage), (M*N)*sizeof(unsigned char)); if (ofp.fail()) { cout << "Can't write image " << fname << endl; exit(0); } ofp.close(); }

Reading a PGM image from a file void readImage(char fname[], ImageType& image) { int i, j; int N, M, Q; unsigned char *charImage; char header [100], *ptr; ifstream ifp; ifp.open(fname, ios::in | ios::binary); if (!ifp) { cout << "Can't read image: " << fname << endl; exit(1); }

Reading a PGM image from a file // read header ifp.getline(header,100,'\n'); if ( (header[0]!=80) || // 'P' (header[1]!=53) ) { // '5' cout << "Image " << fname << " is not PGM" << endl; exit(1); } ifp.getline(header,100,'\n'); // skip comments while(header[0]=='#') ifp.getline(header,100,'\n'); M=strtol(header,&ptr,0); // read M, N N=atoi(ptr);

Reading a PGM image …. (cont’d) ifp.getline(header,100,'\n'); Q=strtol(header,&ptr,0); charImage = (unsigned char *) new unsigned char [M*N]; ifp.read( reinterpret_cast (charImage), (M*N)*sizeof(unsigned char)); if (ifp.fail()) { cout << "Image " << fname << " has wrong size" << endl; exit(1); } ifp.close();

Reading a PGM image…(cont’d) // Convert unsigned characters to integers int val; for(i=0; i<N; i++) for(j=0; j<M; j++) { val = (int)charImage[i*M+j]; image.setVal(i, j, val); }

How do I “see” images on my computer? Unix/Linux: xv, gimp Windows: Photoshop Irfanview

How do I convert an image from one format to another? Use “Save As” option

More Information on Image Processing and Computer Vision Computer Vision Home Page UNR Computer Vision Laboratory