Multiscale Analysis of Images Gilad Lerman Math 5467 (stealing slides from Gonzalez & Woods, and Efros)

The Multiscale Nature of Images

Recall: Gaussian pre-filtering Solution: filter the image, then subsample Filter size should double for each ½ size reduction.

Subsampling with Gaussian pre-filtering Solution: filter the image, then subsample Filter size should double for each ½ size reduction.

Image Pyramids Known as a Gaussian Pyramid [Burt and Adelson, 1983] In computer graphics, a mip map [Williams, 1983] A precursor to wavelet transform

A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose Figure from David Forsyth

Gaussian pyramid construction filter mask Repeat Filter Subsample Until minimum resolution reached can specify desired number of levels (e.g., 3-level pyramid) The whole pyramid is only 4/3 the size of the original image!

What does blurring take away? (recall) original

What does blurring take away? (recall) smoothed (5x5 Gaussian)

High-Pass filter smoothed – original

Band-pass filtering Gaussian Pyramid (low-pass images) Scale is confusing as there is down-sampling and up-sampling Laplacian Pyramid (subband images) Created from Gaussian pyramid by subtraction

Laplacian Pyramid Need this! Original image How can we reconstruct (collapse) this pyramid into the original image?

Image resampling (interpolation) So far, we considered only power-of-two subsampling What about arbitrary scale reduction? How can we increase the size of the image? d = 1 in this example 1 2 3 4 5 Recall how a digital image is formed It is a discrete point-sampling of a continuous function If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

Image resampling So far, we considered only power-of-two subsampling What about arbitrary scale reduction? How can we increase the size of the image? d = 1 in this example 1 2 3 4 5 Recall how a digital image is formed It is a discrete point-sampling of a continuous function If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

Image resampling So what to do if we don’t know Image reconstruction Answer: guess an approximation Can be done in a principled way: filtering 1 2 3 4 5 2.5 d = 1 in this example Image reconstruction Convert to a continuous function Reconstruct by cross-correlation:

bilinear interpolation Resampling filters What does the 2D version of this hat function look like? performs linear interpolation (tent function) performs bilinear interpolation Better filters give better resampled images Bicubic is common choice Why not use a Gaussian? What if we don’t want whole f, but just one sample?

Bilinear interpolation Smapling at f(x,y):

Applications (more efficient with wavelets) Coding High frequency coefficients need fewer bits, so one can collapse a compressed Laplacian pyramid Denoising/Restoration Setting “most” coefficients in Laplacian pyramid to zero Image Blending

Application: Pyramid Blending 1 1 1 Left pyramid blend Right pyramid

Image Blending

+ = Feathering Ileft Iright left right Encoding transparency 1 + Ileft Iright left right = Encoding transparency I(x,y) = (aR, aG, aB, a) Iblend = left Ileft + right Iright

Affect of Window Size 1 left 1 right

Affect of Window Size 1 1

Good Window Size “Optimal” Window: smooth but not ghosted 1 “Optimal” Window: smooth but not ghosted Burt and Adelson (83): Choose by pyramids…

Pyramid Blending initial registration has been applied… Fig. Image mosaics. The left half of image (a) is catinated with the right half of image (b) to give the mosaic in (c). Note that the boundary between regions is clearly visible. The mosaic in (d) was obtained by combining images separately in each spatial frequency band of their pyramid representations then expanding and summing these bandpass mosaics.

Pyramid Blending (Color)

Laplacian Pyramid: Blending General Approach: Build Laplacian pyramids LA and LB from images A and B Build a Gaussian pyramid GR from selected region R (black white corresponding images) Form a combined pyramid LS from LA and LB using nodes of GR as weights: LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j) Collapse the LS pyramid to get the final blended image

laplacian level 4 laplacian level 2 laplacian level left pyramid right pyramid blended pyramid

Blending Regions

Simplification: Two-band Blending Brown & Lowe, 2003 Only use two bands: high freq. and low freq. Blends low freq. smoothly Blend high freq. with no smoothing: use binary mask Can be explored in a project…