1. Multiscale Analysis of Images
Gilad Lerman
Math 5467
(stealing slides from
Gonzalez & Woods, and Efros)

2. The Multiscale Nature of Images

3. Recall: Gaussian pre-filtering
Solution: filter the image, then subsample
Filter size should double for each ½ size reduction.

4. Subsampling with Gaussian pre-filtering
Solution: filter the image, then subsample
Filter size should double for each ½ size reduction.

5. Image Pyramids Known as a Gaussian Pyramid [Burt and Adelson, 1983]
In computer graphics, a mip map [Williams, 1983]
A precursor to wavelet transform

6. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose
Figure from David Forsyth

7. Gaussian pyramid construction
filter mask
Repeat
Filter
Subsample
Until minimum resolution reached
can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!

8. What does blurring take away? (recall)
original

9. What does blurring take away? (recall)
smoothed (5x5 Gaussian)

10. High-Pass filter
smoothed – original

11. Band-pass filtering Gaussian Pyramid (low-pass images)
Scale is confusing as there is down-sampling and up-sampling
Laplacian Pyramid (subband images)
Created from Gaussian pyramid by subtraction

12. Laplacian Pyramid
Need this!
Original
image
How can we reconstruct (collapse) this pyramid into the original image?

15. Image resampling (interpolation)
So far, we considered only power-of-two subsampling
What about arbitrary scale reduction?
How can we increase the size of the image?
d = 1 in this
example 1 2 3 4 5
Recall how a digital image is formed
It is a discrete point-sampling of a continuous function
If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

16. Image resampling So far, we considered only power-of-two subsampling
What about arbitrary scale reduction?
How can we increase the size of the image?
d = 1 in this
example 1 2 3 4 5
Recall how a digital image is formed
It is a discrete point-sampling of a continuous function
If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale

17. Image resampling So what to do if we don’t know Image reconstruction
Answer: guess an approximation
Can be done in a principled way: filtering 1 2 3 4 5
2.5
d = 1 in this
example
Image reconstruction
Convert to a continuous function
Reconstruct by cross-correlation:

18. bilinear interpolation
Resampling filters
What does the 2D version of this hat function look like?
performs
linear interpolation
(tent function) performs
bilinear interpolation
Better filters give better resampled images
Bicubic is common choice
Why not use a Gaussian?
What if we don’t want whole f, but just one sample?

19. Bilinear interpolation
Smapling at f(x,y):

20. Applications (more efficient with wavelets)
Coding
High frequency coefficients need fewer bits, so one can
collapse a compressed Laplacian pyramid
Denoising/Restoration
Setting “most” coefficients in Laplacian pyramid to zero
Image Blending

21. Application: Pyramid Blending1 1 1
Left pyramid
blend
Right pyramid

22. Image Blending

23. + = Feathering Ileft Iright left right Encoding transparency1 +
Ileft
Iright
left
right =
Encoding transparency
I(x,y) = (aR, aG, aB, a)
Iblend = left Ileft + right Iright

24. Affect of Window Size 1
left 1
right

25. Affect of Window Size 1 1

26. Good Window Size “Optimal” Window: smooth but not ghosted1
“Optimal” Window: smooth but not ghosted
Burt and Adelson (83): Choose by pyramids…

27. Pyramid Blending initial registration has been applied…
Fig. Image mosaics. The left half of image (a) is catinated with the right half of
image (b) to give the mosaic in (c). Note that the boundary between regions is
clearly visible. The mosaic in (d) was obtained by combining images separately in
each spatial frequency band of their pyramid representations then expanding and
summing these bandpass mosaics.

28. Pyramid Blending (Color)

29. Laplacian Pyramid: Blending
General Approach:
Build Laplacian pyramids LA and LB from images A and B
Build a Gaussian pyramid GR from selected region R (black white corresponding images)
Form a combined pyramid LS from LA and LB using nodes of GR as weights:
LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j)
Collapse the LS pyramid to get the final blended image

30. laplacian
level 4
laplacian
level 2
laplacian
level
left pyramid
right pyramid
blended pyramid

31. Blending Regions

32. Simplification: Two-band Blending
Brown & Lowe, 2003
Only use two bands: high freq. and low freq.
Blends low freq. smoothly
Blend high freq. with no smoothing: use binary mask
Can be explored in a project…