1. Image Representation Gaussian pyramids Laplacian Pyramids
Image Processing - Lesson 10
Image Representation
Gaussian pyramids
Laplacian Pyramids
Wavelet Pyramids
Applications

2. Image Pyramids
Image features at different resolutions require filters at different scales.
Edges (derivatives):
f(x)
f (x)

3. Image Pyramids Image Pyramid = Hierarchical representation of an image
No details in image -
(blurred image)
low frequencies
Low
Resolution
Details in image -
low+high frequencies
High
Resolution
A collection of images at different resolutions.

4. Image pyramids
Gaussian Pyramids
Laplacian Pyramids
Wavelet/QMF

5. Image Pyramid
Low resolution
High resolution

6. Image Pyramid Frequency Domain
Low resolution
High resolution

7. Image Blurring = low pass filtering* = ~ =

8. * = * = * =

9. Image Pyramid
Low resolution
High resolution

10. Gaussian Pyramid
Level n
1 X 1
Level 1
2n-1 X 2n-1
Level 0
2n X 2n

11. Gaussian Pyramid

12. Gaussian Pyramid Burt & Adelson (1981) Normalized: Swi = 1
Symmetry: wi = w-i
Unimodal: wi ³ wj for 0 < i < j
Equal Contribution: for all j Swj+2i = constant w0 w1
w-1
w-2 w2

13. Gaussian Pyramid a + 2b + 2c = 1 a + 2c = 2b a > 0.25 b = 0.25
Burt & Adelson (1981) a b c
a + 2b + 2c = 1
a + 2c = 2b
a > 0.25
b = 0.25
c = a/2

14. g = [0.05 0.25 0.4 0.25 0.05] low_pass_filter = g’ * g =
For a = most similar to a Gauusian filter
g = [ ]
low_pass_filter = g’ * g = 1 2 3 4 5
0.05
0.1
0.15
0.2

15. Computational Aspects
Gaussian Pyramid -
Computational Aspects
Memory:
2NX2N (1 + 1/4 + 1/ ) = 2NX2N * 4/3
Computation:
Level i can be computed with a single convolution
with filter: hi = g * g * g * .....
i times
Example:
h2 = * = g g

16. MultiScale Pattern Matching
Option 1: Scale target and search for each in image.
Option 2: Search for original target in image pyramid.

17. Hierarchical Pattern Matching
search
search
search
search

18. Pattern matching using Pyramids - Example
image
pattern
correlation

19. Image pyramids
Gaussian Pyramids
Laplacian Pyramids
Wavelet/QMF

20. Laplacian Pyramid Motivation = Compression, redundancy removal.
compression rates are higher for predictable values.
e.g. values around 0.
G0, G1, .... = the levels of a Gaussian Pyramid.
Predict level Gl from level Gl+1 by Expanding Gl+1 to G’l
Gl+1
Expand
Reduce Gl
G’l
Denote by Ll the error in prediction:
Ll = Gl - G’l
L0, L1, .... = the levels of a Laplacian Pyramid.

21. What does blurring take away?
original

22. What does blurring take away?
smoothed (5x5 Gaussian)

23. What does blurring take away?
smoothed – original

24. - = = - - = Laplacian Pyramid Gaussian Laplacian Pyramid Pyramid
expand - =
expand - =
expand - =

25. Frequency
Domain
Gaussian
Pyramid
Laplacian
Pyramid

26. Laplace Pyramid -
No scaling

27. from: B.Freeman

28. Reconstruction of the original image from the Laplacian Pyramid
Gl = Ll + G’l
Laplacian
Pyramid
expand + =
expand + =
expand + =
Original
Image

29. Computational Aspects
Laplacian Pyramid -
Computational Aspects
Memory:
2NX2N (1 + 1/4 + 1/ ) = 2NX2N * 4/3
However coefficients are highly compressible.
Computation:
Li can be computed from G0 with a single convolution
with filter: ki = hi-1 - hi - =
hi-1 hi ki k1 k2 k3

30. Image Mosaicing
Registration

31. Image Blending

32. Blending

33. Multiresolution Spline
When splining two images, transition from one image to
the other should behave:
High Frequencies
Middle Frequencies
Low Frequencies

34. Multiresolution Spline
High Frequencies
Middle Frequencies
Low Frequencies

35. Multiresolution Spline - Example
Left Image
Right Image
Left + Right
Narrow Transition
Wide Transition
(Burt & Adelson)

36. Multiresolustion Spline - Using Laplacian Pyramid

37. Multiresolution Spline - Example
Left Image
Right Image
Left + Right
Narrow Transition
Wide Transition
Multiresolution Spline
(Burt & Adelson)

38. Multiresolution Spline - Example
Original - Left
Original - Right
Glued
Splined

39. laplacian level 4
laplacian level 2
laplacian level 0
left pyramid
right pyramid
blended pyramid

40. Multiresolution Spline

41. Multiresolution Spline - Example

42. The colored version
© prof. dmartin

43. Image pyramids
Gaussian Pyramids
Laplacian Pyramids
Wavelet/QMF

44. What is a good representation for image analysis?
Pixel domain representation tells you “where” (pixel location), but not “what”.
In space, this representation is too localized
Fourier transform domain tells you “what” (textural properties), but not “where”.
In space, this representation is too spread out.
Want an image representation that gives you a local description of image events—what is happening where.
That representation might be “just right”.

45. Space-Frequency Tiling
Standard basis
Spatial
Freq.
Fourier basis
Spatial
Freq.
Wavelet basis
Spatial

46. Space-Frequency Tiling
Standard basis
Spatial
Freq.
Fourier basis
Spatial
Freq.
Wavelet basis
Spatial

47. Various Wavelet basis

48. Wavelet bands are split recursively
Wavelet - Frequency domain
Wavelet bands are split recursively
image H L H L L H

49. Wavelet decomposition - 2D
Wavelet - Frequency domain
Wavelet decomposition - 2D
Frequency domain
Horizontal high pass
Vertical high pass
Horizontal low pass
Vertical low pass

50. Apply the wavelet transform separably in both dimensions
Wavelet - Frequency domain
Apply the wavelet transform separably in both dimensions
Horizontal high pass, vertical high pass
Horizontal high pass, vertical low-pass
Horizontal low pass, vertical high-pass
Horizontal low pass,
Vertical low-pass

51. Splitting can be applied recursively:

52. Pyramids in Frequency Domain
Gaussian Pyramid
Laplacian Pyramid

53. Wavelet Decomposition
Fourier Space

54. Wavelet Transform - Step By Step Example

55. Wavelet Transform - Example

56. Wavelet Transform - Example

57. Image Pyramids - Comparison
Image pyramid levels = Filter then sample.
Filters:
Gaussian Pyramid
Laplacian Pyramid
Wavelet Pyramid

58. Image Linear Transforms
Basis
Characteristics
Localized in space
Not localized in Frequency
Delta
Standard
Not localized in space
Localized in Frequency
Fourier
Sines+Cosines
Wavelet
Pyramid
Localized in space
Localized in Frequency
Wavelet Filters
Delta
Fourier
Wavelet
space
space
space

59. Convolution and Transforms in matrix notation (1D case)
transformed image
Vectorized image
Basis vectors (Fourier, Wavelet, etc)

60. = * Fourier Transform Fourier transform Fourier bases
pixel domain image
Fourier bases are global: each transform coefficient depends on all pixel locations.
From: B. Freeman

61. Transform in matrix notation (1D case)
Forward Transform:
transformed image
Vectorized image
Basis vectors (Fourier, Wavelet, etc)
Inverse Transform:
Basis vectors
Vectorized image
transformed image

62. = * Inverse Fourier Transform Fourier transform pixel domain image
Fourier bases
Every image pixel is a linear combination of the Fourier basis weighted by the coefficient.
Note that if U is orthonormal basis then

63. = * Convolution Convolution Result Circular Matrix of Filter Kernels
pixel image
From: B. Freeman

64. Pyramid = Convolution + Sampling*
Convolution
Result
Matrix of
Filter Kernels
pixel image
From: B. Freeman

65. Pyramid = Convolution + Sampling
Pyramid Level 1 = *
Pyramid Level 2 = *
Pyramid Level 2 = *

66. Pyramid = Convolution + Sampling
Pyramid Level 2 = *
Pyramid Level 2 = *

67. Pyramid as Matrix Computation - Example
- Next pyramid level
U2 =
- The combined effect of the two pyramid levels
U2 * U1 =
from: B.Freeman

68. = = * Gaussian Pyramid pixel image
Overcomplete representation. Low-pass filters, sampled appropriately for their blur.
Gaussian pyramid
From: B. Freeman

69. = * Laplacian Pyramid pixel image
Overcomplete representation. Transformed pixels represent bandpassed image information.
Laplacian pyramid
From: B. Freeman

70. = * Wavelet Transform Wavelet pyramid
Ortho-normal transform (like Fourier transform), but with localized basis functions.
pixel image
From: B. Freeman

71. Wavelet Shrinkage Denoising

72. Image statistics (or, mathematically, how can you tell image from noise?)
Noisy image
From: B. Freeman

73. Clean image
From: B. Freeman

74. Image domain histogram
From: B. Freeman

75. Wavelet domain image histogram
From: B. Freeman

76. Image domain noise histogram
From: B. Freeman

77. Wavelet domain noise histogram

78. Noise-corrupted full-freq and bandpass images
But want the bandpass image histogram to look like this
From: B. Freeman

79. Using Bayes theorem argmaxx P(x|y) = P(y|x) P(x) / P(y)
Constant w.r.t. parameters x.
argmaxx P(x|y) = P(y|x) P(x) / P(y)
The parameters you want to estimate
Likelihood function
What you observe
Prior probability
From: B. Freeman

80. Bayesian MAP estimator for clean bandpass coefficient values
Let x = bandpassed image value before adding noise.
Let y = noise-corrupted observation.
By Bayes theorem
P(x|y) = k P(y|x) P(x)
P(x)
maximum
P(y|x) y
P(y|x)
P(x|y)
P(x|y)
From: B. Freeman

81. Bayesian MAP estimator for clean bandpass coefficient values
P(y|x)
maximum
P(x|y)
From: B. Freeman

82. Bayesian MAP estimator for clean bandpass coefficient values
maximum
P(y|x)
P(x|y)
From: B. Freeman

83. MAP estimate, , as function of observed coefficient value, y
From: B. Freeman

84. Wavelet Shrinkage Pipe-line
Mapping functions
Transform W
xiw
yiw
Inverse
Transform WT

85. Simoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring
Noise removal results
Simoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring
From: B. Freeman

86. More results

87. More results