Applications of Image Filters Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/04/10

Credit: S. Seitz Review: Image filtering

Image filtering Credit: S. Seitz

Image filtering Credit: S. Seitz

Filtering in spatial domain * =

Filtering in frequency domain FFT Inverse FFT =

Sharpening revisited What does blurring take away? original smoothed (5x5) – detail = sharpened = Let’s add it back: originaldetail + α

Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006 “Hybrid Images,”

Application: Hybrid Images A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006 “Hybrid Images,”

Today’s class How to use filters for – Matching – Denoising – Anti-aliasing Image representation with pyramids Texture – What is it? – How to represent it?

Matching with filters Goal: find in image

Matching with filters Goal: find in image Method 1: SSD Input 1- sqrt(SSD) Threshold at 0.8

Matching with filters Goal: find in image Method 1: SSD Method 2: Normalized cross-correlation Input Normalized X-Correlation Threshold at 0.5

Noise Gaussian Additive Noise + = Noisy Image

Gaussian noise Mathematical model: sum of many independent factors Assumption: independent, zero-mean noise Source: M. Hebert

Noise Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution Source: S. Seitz

Denoising Additive Gaussian Noise Gaussian Filter

Smoothing with larger standard deviations suppresses noise, but also blurs the image Reducing Gaussian noise Source: S. Lazebnik

Reducing salt-and-pepper noise by Gaussian smoothing 3x35x57x7

Alternative idea: Median filtering A median filter operates over a window by selecting the median intensity in the window Is median filtering linear? Source: K. Grauman

Median filter What advantage does median filtering have over Gaussian filtering? – Robustness to outliers Source: K. Grauman

Median filter Salt-and-pepper noise Median filtered Source: M. Hebert MATLAB: medfilt2(image, [h w])

Median vs. Gaussian filtering 3x35x57x7 Gaussian Median

Throw away every other row and column to create a 1/2 size image Subsampling by a factor of 2

1D example (sinewave): Source: S. Marschner Aliasing problem

Source: S. Marschner 1D example (sinewave): Aliasing problem

Sub-sampling may be dangerous…. Characteristic errors may appear: – “Wagon wheels rolling the wrong way in movies” – “Checkerboards disintegrate in ray tracing” – “Striped shirts look funny on color television” Source: D. Forsyth Aliasing problem

Aliasing in video Slide by Steve Seitz

Source: A. Efros Aliasing in graphics

Sampling and aliasing

When sampling a signal at discrete intervals, the sampling frequency must be  2  f max ; f max = max frequency of the input signal. This will allows to reconstruct the original perfectly from the sampled version good bad vvv Nyquist-Shannon Sampling Theorem

Anti-aliasing Solutions: Sample more often Get rid of all frequencies that are greater than half the new sampling frequency – Will lose information – But it’s better than aliasing – Apply a smoothing filter

Algorithm for downsampling by factor of 2 1.Start with image(h, w) 2.Apply low-pass filter im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1)) 3.Sample every other pixel im_small = im_blur(1:2:end, 1:2:end);

Anti-aliasing Forsyth and Ponce 2002

Subsampling without pre-filtering 1/4 (2x zoom) 1/8 (4x zoom) 1/2 Slide by Steve Seitz

Subsampling with Gaussian pre-filtering G 1/4G 1/8Gaussian 1/2 Slide by Steve Seitz

Gaussian pyramid Source: Forsyth

Laplacian filter Gaussian unit impulse Laplacian of Gaussian Source: Lazebnik

Laplacian pyramid Source: Forsyth

Computing Gaussian/Laplacian Pyramid Can we reconstruct the original from the laplacian pyramid?

Related idea: 2d wavelets

2d Wavelets Matlab: wavemenu

Image representation Pixels: great for spatial resolution, poor access to frequency Fourier transform: great for frequency, not for spatial info Pyramids/wavelets: balance between spatial and frequency information

Major uses of image pyramids Compression Object detection – Scale search – Features Detecting stable interest points Registration – Course-to-fine

Texture Source: Forsyth

Texture and Material

Texture and Orientation

Texture and Scale

What is texture? Regular or stochastic patterns caused by bumps, grooves, and/or markings

How can we represent texture? Measure frequencies at various orientations and scales

Overcomplete representation: filter banks LM Filter Bank Code for filter banks:

Filter banks Process image with each filter and keep responses (or squared/abs responses)

Representing texture Idea 1: take simple statistics (e.g., mean, std) of various absolute filter responses

Can you match the texture to the response? Mean abs responses Filters A B C 1 2 3

Representing texture by mean abs response Mean abs responses Filters

Representing texture Idea 2: take vectors of filter responses at each pixel and cluster them, then take histograms (more on in coming weeks)

Things to remember When matching using a filter, normalized cross correlation is preferred Use Gaussian or median filter for denoising Beware of aliasing – use lowpass filter to downsample Laplacian pyramids and wavelets provide spatial/frequency information Filter banks provide overcomplete representation, good for modeling/recognizing texture

Next class Edges and lines

Questions

Denoising “Salt and Pepper” Noise

Denoising 3x3 Median Filter “Salt and Pepper” Noise

Denoising Gaussian Additive Noise 3x3 Median Filter