COMP 9517 Computer Vision Digital Images 1/28/2018 COMP 9517 S2, 2009

Overview of Digital Images Humans derive a great deal of information about the world through their visual sense – eyes. Three components for construction of images: A scene of objects Illumination of the objects Sensing the illumination 1/28/2018 COMP 9517 S2, 2009

Overview of Digital Image 2D digital images is an array of intensity samples reflected from or transmitted through objects Digital images contain fixed number of rows and columns of Pixels Pixels (picture elements) are little tiles holding quantised values (0-255) represent the brightness at the points of the image Colour images have three values for each pixel (for example, RGB) 1/28/2018 COMP 9517 S2, 2009

Overview of Digital Image 1 2 3 4 5 6 7 130 146 133 95 71 62 78 92 139 120 55 117 112 110 142 143 125 156 159 168 159 3 band for colour image 159 159 1/28/2018 COMP 9517 S2, 2009

Digital Images - 2D Projection of 3D 3D world has color, texture, surfaces, volumes, light sources, objects, motion, connections, etc. 2D image is a projection of a scene from a specific viewpoint; many 3D features are captured, but some missed. 1/28/2018 COMP 9517 S2, 2009

Image Receives Reflections Light reaches surfaces of objects Surfaces reflect Camera receives light energy 1/28/2018 COMP 9517 S2, 2009

Radiation Different types of electromagnetic radiation, such as X-ray, infra-red Different wavelengths of radiation have different properties Different devices to detect different radiation 1/28/2018 COMP 9517 S2, 2009

Image Devices CCD (charge-coupled device) cameras Lens collects light rays Cells (array of small fixed elements) convert light energy into electrical charge Through frame grabber or IEEE 1394 to PC 1/28/2018 COMP 9517 S2, 2009

Computer Vision System Camera inputs to frame buffer Program can interpret data Program can add graphics Program can add imagery 1/28/2018 COMP 9517 S2, 2009

Image Formation The geometry of image formation: the projection of each point of the 3D scene through the centre of projection (or lens centre) onto the image plane Pinhole Camera Perspective projection Affine projection 1/28/2018 COMP 9517 S2, 2009

Perspective Projection The apparent size of object depends on their distance: far object appear smaller By similar triangles Ignore the third coordinate, and get 1/28/2018 COMP 9517 S2, 2009

Affine Project Scene depth is small relative to the average distance from the camera Let magnification to be positive constant, since is negative, i.e. treat all points in scene being at constant distance from camera Leads to weak perspective projection 1/28/2018 COMP 9517 S2, 2009

Affine Project The camera always remains at a roughly constant distance from the scene Orthographic projection when normalise m to be -1 1/28/2018 COMP 9517 S2, 2009

Picture function A picture function is a mathematical representation f(x,y) of a picture as a function of two spatial variables x and y. x and y: real values defining points of the picture f(x,y): real value defining the intensity of point (x,y) 1/28/2018 COMP 9517 S2, 2009

Picture Function and Digital Images Analog image: a 2D image F(x,y) has infinite precision in both spatial parameters x, y and intensity at each spatial point (x,y) Digital image: a 2D image I[r,c] by a discrete 2D array of intensity samples with limited precision Can be stored in a 2D computer memory structure 2D array of discrete values. In C, char I[512][512] Intensity as an 8-bit number allows values of 0-255 3 such values for colour image. 1/28/2018 COMP 9517 S2, 2009

Sampling and Quantisation Digitisation: convert analog image to digital image Sampling: digitising the coordinate spatial discretisation of a picture function f (x,y) use a grid of sampling points, normally rectangular: image sampled at points x = j Δx, y = k Δy, j = 1...M, k = 1...N. Δx, Δy called the sampling interval. 1/28/2018 COMP 9517 S2, 2009

Spatial Resolution Spatial Resolution: pixels per unit of length Resolution decreases by one half Human faces can be recognized at 64 x 64 pixels per face Appropriate resolution is essential: too little resolution, poor recognition too much resolution,slow and wastes memory 1/28/2018 COMP 9517 S2, 2009

Sampling and Quantisation Quantisation: digitising the amplitude values called intensity or gray level quantisation Gray-level resolution: usually has 16, 32, 64, ...., 128, 256 levels number of levels should be high enough for human perception of shading details - human visual system requires about 100 levels for a realistic image. 1/28/2018 COMP 9517 S2, 2009

Image Coordinate System Raster oriented: down-leftward (a) Cartesian coordinate: up-leftward (b, c) Relationship btn pixel centre point to I[i,j] 1/28/2018 COMP 9517 S2, 2009

Type of images Gray-scale image: a monochrome digital image I[r,c] with one intensity value per pixel Multispectral image: a 2D image M[x,y] has a vector of values at each pixel, colour image (r,g,b) Binary image: a digital image with all pixel values 0 or 1 Labelled image: a digital image L[r,c] with pixel values as symbols denoting the decisions made for that pixel 1/28/2018 COMP 9517 S2, 2009

Digital Image Format Image file header: non image information for labelling and decoding data Image data Data Compression Lossless: can be recovered exactly Lossy: may lose quality 1/28/2018 COMP 9517 S2, 2009

Common Image Format Run-Coded Binary Image: an efficient coding scheme for binary or labelled images 1/28/2018 COMP 9517 S2, 2009

Common Image Format PGM(PBM/PGM, PPM): Portable gray map One of the simplest file formats 1/28/2018 COMP 9517 S2, 2009

Common Image Format Gif(GIF): Graphics Interchange Format, WWW, 8 bits – 256 colour levels, may be lossless Tiff(TIFF/TIF): Tag Image File Format, 1-24 bits, lossy or lossless Jpeg(JFIF/JFI/JPG): Joint Photographic Experts Group, up to 24 bits, recent standard, independent of colour system, lossy or lossless PostScript(PDF/PDL/EPS): encoded by ASCII Mpeg(MPG/MPEG/MPEG-2): Motion Picture Expert Group, stream-oriented encoding of video 1/28/2018 COMP 9517 S2, 2009

References Driscoll, W. and Vaughan, W., eds (1978), Handbook of Optics, McGrtaw-Hill. Boyle, W. and Smith, G. (1970), Charge coupled semiconductor devices, Bell Syst. Tech. J. 49, 587-593. Huang, T.S.(1965), PCM Picture Transmission. IEEE Spectrum, vol. 2, no.12, pp.57-63. 1/28/2018 COMP 9517 S2, 2009

Acknowledgement Some material, including images and tables, were drawn from the textbook and Stockman’s online resources. 1/28/2018 COMP 9517 S2, 2008