CSE 6367 Computer Vision Image Operations and Filtering “You cannot teach a man anything, you can only help him find it within himself.” ― Galileo GalileiGalileo Galilei Farhad Kamangar Computer Science and Engineering Department The University of Texas at Arlington

What Is An Image? Grayscale image: –Think of an image as a function f( x, y ) is the intensity at position ( x, y ) In practice, an image only is defined over a finite A color image is three functions

Images as A Matrix Digital images:: Sample the 2D space on a regular grid R G B row column

Neighborhood operations If we shuffle all pixels in an image then the image can not be recognized (However the histogram of the image will not change) Image information is stored in local neighborhoods. We can perform image operations on local neighborhoods. Example: Replace each pixel with the average of all the neighboring pixels.

Moving Average Source: S. Seitz f( x, y ) g( x, y )

Correlation Filtering an image means replacing each pixel in the original image with a linear combination of that pixel and its neighbors. The filter “template” or “kernel” or “mask” H[u,v] is the values for the weights in the linear combination. The kernel size is 2k+1 by 2k+1 and the center element is assumed to be element (0,0): g( x, y ) h(u,v) “kernel” or “mask” f( x, y )

Convolution Same as cross-correlation, except that the kernel is “flipped” (horizontally and vertically) This is called a convolution operation:

Convolution vs Correlation Cross correlation is: Not commutative Associative No Identity Convolution is: Commutative Associative Identity

Moving Average as Correlation g( x, y ) 1/9 h(u,v) “kernel” or “mask” f( x, y ) cross-correlation

What is the gradient? No Change in horizontal direction Change in vertical direction

Gradient

How do we calculate gradient on a digital image F(x,y)? We can subtract a pixel value from it’s neighbor. Question: Which neighbor?

Gradient Can we compute gradients with cross correlation? Correlation with [-.5, 0,.5] Correlation with [-.5; 0;.5] for derivative in y direction.

Gradient Convolution Masks

Roberts operator

Sobel Common Sobel operators are:

Prewit Operator

What is an Edge? Edge is Where Change Occurs The changes due to noise are not edges Edges in real images are not perfectly sharp Change is measured by derivative Biggest change, derivative has maximum magnitude Or 2 nd derivative is zero.

Edges Edge detection is the process of finding edges in an image A contour a curve that passes through neighboring edge points Edge following is the process of searching the image to determine contours

Derivative Around an Edge If the edge is noisy then derivative is high everywhere. Smooth before taking derivative.

Implementation of 1D Edge Detection

Efficient Implementation of Edge Detection

Use of Second Derivative Edge points are located at the peaks of of the first derivative peaks of the first derivative A threshold is used to detect an edge. Alternative approach is to use the zero crossing of the second derivative. The Laplacian may be used as a second derivative. The Laplacian is symmetric in all directions and is invariant to rotation in the image.

Laplacian Operator Two dimensional equivalent of the second derivative For digital images Mask

Laplacian of Gaussian The second derivative creates zero crossings in the presence of noise. The image should be smoothed before edge detection The Laplacian of a Gaussian LoG smoothes image by combining Gaussian filtering and Laplacian operator

Laplacian of Gaussian Laplacian of Gaussian operator Zero-crossings is the edge position

Discrete LoG

DoG vs LoG Difference of two Gaussians (DoG) can approximate Laplacian of Gaussian many filters with varying σ DoG is implemented by subtracting two Gaussian filters with different σ

Edge detection by subtraction Laplacian of Gaussian Gaussian delta function

Kirsch edge patterns

Canny Edge Detector Smooth the image to remove the noise (Gaussian filter) Find the gradients of the image Apply non-maximum suppression Apply double threshold Select edge pixels by hysteresis (suppress all edges that are weak and not connected to strong edges).

Canny Edge Detector Find points along the curve where the gradient magnitude is biggest.

Non-Maxima Supression Find points along the curve where the gradient magnitude is biggest.