Visual Information Systems Basic images processing

Image Resolution How many pixels –spatial resolution How many shades of grey/colours –amplitude resolution How many frames per second –temporal resolution

Spatial Resolution n, n/2, n/4, n/8, n/16 and n/32 pixels per unit length

amplitude resolution -Shades of Grey 8, 4, 2 and 1 bit images.

Temporal Resolution –how much does an object move between frames? –Can motion be understood unambiguously? Nyquist’s Theorem –A periodic signal can be reconstructed if the sampling interval is half the period –An object can be detected if two samples span its smallest dimension

Colour Representation three primaries could approximate many colours red, green, blue C= rR+gG+bB

Other Colour Models YMCK HSI YCrCb etc

Colour image and video sequence colour can be conveyed by combining different colours of light, using three components (red, green and blue): R = r(x,y); G = g(x,y); B = b(x,y), where R, G, B are defined in a similar way to F. The vector (r(x,y), g(x,y), b(x,y)) defines the intensity and colour at the point (x,y) in the colour image. A video sequence is, in effect, a time-sampled representation of the original moving scene. Each frame in the sequence is a standard colour, or monochrome image and can be coded as such. a monochrome video sequence may be represented digitally as a sequence o 2-D arrays [F1, F2, F3..F N ].

Image processing and transform Objectives: To enhance features that are useful Obtain key representations (silent features) of image content

Classification of Image Transforms Point transforms –modify individual pixels –modify pixels’ locations Local transforms –output derived from neighbourhood Global transforms –whole image contributes to each output value

Point Transforms Manipulating individual pixel values –Brightness adjustment –Contrast adjustment Histogram manipulation –equalisation Image magnification

Monadic, Point-by-point Operators Monadic point-to-point operator

Brightness Adjustment Add a constant to all values g(x,y) = f(x,y) + k (k = 50)

Intensity Shift g(x,y) <= Where k is a user defined variable 0 a(x,y)+k < 0 f(x,y) +k 0<=f(x,y) +k <=W W W<f(x,y) +k

Contrast Adjustment Scale all values by a constant g(x,y) = a* f(x,y) (a = 1.5)

General expression for brightness and contrast modification g(x,y) = a * f(x,y) +b If we do not want to specify a gain and a bias, but would rather map a particular range of grey levels, [f1,f2],, onto a new range, [g1,g2]. This form of mapping is accomplished using (g(x,y)-g1)/(f(x,y)-f1)=(g2-g1)/(f2-f1) i.e. g(x,y) = g1 +((g2-g1)/(f2-f1))[f(x,y)-f1]

linear mapping f g 255 0

Linear and non-linear mapping If ‘a’ is a constant, then it is a linear mapping; When ‘a’ is a function, then it is a non-linear mapping Non-linear mapping functions have a useful properties: the gain, ‘a’, applied to input grey level, can vary. Thus the way in which contrast is modified depends on the input grey level. If a range of grey level is mapped to a wider ranger of grey level, the contrast is enhanced If a range of grey level is mapped to a narrower range of grey level, the contrast is reduced.

Non-linear mapping Generally, logarithmic mapping is for to enhance details in the darker regions of the image, at the expense of detail in the brighter regions. Exponential mapping has a reverse effect, contrast in the brighter parts of an image is increased at the expense of contrast in the darker parts.

Image Histogram Measure frequency of occurrence of each grey/colour value

Calculation of an image histogram Create an array histogram with 2 b elements for all grey levels, i, do histogram[i] = 0 end for for all pixel coordinates, x and y, do Increment histogram [f(x,y)] by 1 end for

Histogram Manipulation Modify distribution of grey values to achieve some effect

Cumulative Histogram Records the cumulative frequency distribution of grey levels in an image. A cumulative histogram is a mapping that counts the cumulative number of pixels in all of the bins up to the specified bin. That is, the cumulative histogram H k of a histogram h k is defined as: (k’ can start from 0, if the range is from 0-255)

Histogram Equalisation Histogram equalisation is based on the argument that the image’s appearance will be improved if the distribution of pixels over the available grey level is even. A non-linear mapping of grey level, specific to that image, that will yield an optimal improvement in contrast. Redistributes grey levels in an attempt to flatten the frequency distribution. More grey levels are allocated where there are most pixels, fewer grey levels where there are fewer pixels. This tends to increase contrast in the most heavily populated regions of the histogram, and often reveals previously hidden detail.

Histogram Equalisation If we are to increase contrast for the most frequently occurring grey levels and reduce contrast in the less popular part of the grey level range, then we need a mapping function which has a steep slope (a>1) at grey levels that occur frequently, and a gentle slope (a<1) at unpopular grey levels. The cumulative histogram of the image has these properties. The mapping function we need is obtained simply by rescaling the cumulative histogram so that its values lie in the range

Calculating histogram equalisation Compute a scaling factor, a=255/number of pixels Calculate histogram (see previous algorithm) c[0] = a*histogram[0] for all remaining grey levels, i, do c[i] = c[i-1]+a*histogram[i] end for for all pixel coordinates, x and y, do g(x,y) = c[f(x,y)] end for

Equalisation/Adaptive Equalisation Specifically to make histogram uniform

Threshold This is an important function, which converts a grey scale image to a binary format. Unfortunately, it is often difficult, or even impossible to find satisfactory values for the user defined integer threshold value

Threshold C(x,y) <= for (int y=0; y<height; y++) for(int x=0; x<width; x++) if (input.getxy(x,y) < threshold) output.setxy(x,y, BLACK); else output.setxy(x,y,WHITE); W a(x,y)>=threshold 0 otherwise

Thresholding Transform grey/colour image to binary if f(x, y) > T output = W (or 1) else 0 How to find T?

Threshold Value Manual –User defines a threshold P-Tile – if we know the proportion of the image is occupied by the object, define the threshold as that grey level which has the correct proportion Mode - Threshold at the minimum between the histogram’s peaks Other automatic methods

Image Magnification Reducing –new value is weighted sum of nearest neighbours –new value equals nearest neighbour Enlarging –new value is weighted sum of nearest neighbours –add noise to obscure pixelation (such operations will be practised in the labs)

Local Transforms Consider neighbourhood information Convolution Applications –smoothing –sharpening –Matching Very useful, but computationally costly

Local Operators The concept behind local operators is that the intensities of several pixels are combined together in order to calculate the intensity of just one pixel. Amongst the simplest of the local operators are those which use a set of nine pixels arranged in a 3 x 3 square region. It computes the value for one pixel on the basis of the intensities within a region containing 3 x 3 pixels. Other local operators employ larger windows.

A sub-image (x-1, y-1) (x, y-1) (x+1, y-1) (x-1, y) (x,y) (x+1,y) (x-1,y+1) (x, y+1) (x+1, y+1)

Local Operators

Convolution Definition Place template on image Multiply overlapping values in image and template Sum products and normalise (Templates usually small)

Example ImageTemplateResult … … … … … … … … … … … … … … … … … Divide by template sum

Separable Templates Convolve with n x n template –n 2 multiplications and additions Convolve with two n x 1 templates –2n multiplications and additions

Example Laplacian template Separated kernels 0 – –1 0 –

Applications Usefulness of convolution is the effects generated by changing templates –Smoothing Noise reduction –Sharpening Edge enhancement

Smoothing Aim is to reduce noise What is “noise”? –Noise is deviation of a value from its expected value How is it reduced –Addition –Adaptively –Weighted

OriginalSmoothed Median Smoothing

Sharpening What is it? –Enhancing discontinuities –Edge detection Why do it? –Perceptually important –Computationally important

Edge Definition An edge is a significant local change in image intensity. Significant: in relation to the proportional change in image intensity across the edge Local: Neighbourhood relationship matters

Algorithms for Edge Detection Algorithms for detecting edges - edge detectors –differentiation based estimated the derivatives of the image intensity function, the idea being that large image derivatives reflect abrupt intensity changes. –Model based determine whether the intensities in a small area conform to some model for the edges that we have assumed.

First Derivative, Gradient Edge Detection If an edge is a discontinuity Can detect it by differencing

Roberts Cross Edge Detector Simplest edge detector

Prewitt/Sobel Edge Detector

Edge Detection Combine horizontal and vertical edge estimates

Example Results

Global Transforms Computing a new value for a pixel using the whole image as input Cosine and Sine transforms Fourier transform –Frequency domain processing Hough transform Karhunen-Loeve transform Wavelet transform

Geometric Transformations Definitions –Affine and non-affine transforms Applications –Manipulating image shapes

Affine Transforms Scale, Shear, Rotate, Translate Change values of transform matrix elements according to desired effect. a, e  scaling b, d  shearing a, b, d, e  rotation c, f  translation = x’ y’ 1 [] a b c d e f g h i [] xy1xy1 [] Length and areas preserved.

Affine Transform Examples

Warping Example Ansell Adams’ Aspens

Summary Point transforms –scaling, histogram manipulation,thresholding Local transforms –edge detection, smoothing Global transforms –Fourier, Hough, Principal Component, Wavelet Geometrical transforms