1. Wavelets (Chapter 7)

2. What are Wavelets?
Wavelets are functions that “wave” above and below the x-axis, have (1) varying frequency, (2) limited duration, and (3) an average value of zero.
This is in contrast to sinusoids, used by FT, which have infinite energy.
Sinusoid Wavelet

3. What are Wavelets? (cont’d)
Like sines and cosines in FT, wavelets are used as basis functions ψk(t) in representing other functions f(t):
Span of ψk(t): vector space S containing all functions f(t) that can be represented by ψk(t).

4. What are Wavelets? (cont’d)
There are many different wavelets:
Haar
Morlet
Daubechies

5. What are Wavelets? (cont’d)
(dyadic/octave grid)

6. What are Wavelets? (cont’d)j
scale/frequency
localization
space localization

7. Continuous Wavelet Transform (CWT)
Scale parameter
(measure of frequency)
Translation parameter, measure of time
Normalization
constant
Forward
CWT:
Continuous wavelet transform of the signal f(t)
Mother wavelet (window)
Scale = 1/j = 1/Frequency

8. CWT: Main Steps
Take a wavelet and compare it to a section at the start of the original signal.
Calculate a number, C, that represents how closely correlated the wavelet is with this section of the signal. The higher C is, the more the similarity.

9. CWT: Main Steps (cont’d)
3. Shift the wavelet to the right and repeat steps 1 and 2 until you've covered the whole signal.

10. CWT: Main Steps (cont’d)
4. Scale the wavelet and repeat steps 1 through Repeat steps 1 through 4 for all scales.

11. Coefficients of CTW Transform
Wavelet analysis produces a time-scale view of the input signal or image.
space

12. Continuous Wavelet Transform (cont’d)
Inverse CWT:
double integral!

13. FT vs WT
weighted by F(u)
weighted by C(τ,s)

14. Properties of Wavelets
Simultaneous localization in time and scale
- The location of the wavelet allows to explicitly represent the location of events in time.
- The shape of the wavelet allows to represent different detail or resolution.
space

15. Properties of Wavelets (cont’d)
Sparsity: for functions typically found in practice, many of the coefficients in a wavelet representation are either zero or very small.
Linear-time complexity: many wavelet transformations can be accomplished in O(N) time.

16. Properties of Wavelets (cont’d)
Adaptability: wavelets can be adapted to represent a wide variety of functions (e.g., functions with discontinuities, functions defined on bounded domains etc.).
Well suited to problems involving images, open or closed curves, and surfaces of just about any variety.
Can represent functions with discontinuities or corners more efficiently (i.e., some have sharp corners themselves).

17. Discrete Wavelet Transform (DWT)
(forward DWT)
(inverse DWT)
where

18. DFT vs DWT FT expansion: WT expansion one parameter basis or
two parameter basis

19. Multiresolution Representation using
fine
details
narrower, small translations j
coarse
details

20. Multiresolution Representation using
fine
details j
coarse
details

21. Multiresolution Representation using
fine
details
wider, large translations j
coarse
details

22. Multiresolution Representation using
high resolution
(more details) j …
low resolution
(less details)

23. Approximation Pyramid (revisited)
high resolution
j = J
low resolution
j=0
high resolution
j = J
low resolution
j=0
scale=1/j

24. Image Pyramids

25. Prediction Residual Pyramid (revisited)
Prediction residual pyramid can be represented more
efficiently.
In the absence of quantization errors, the approximation
pyramid can be reconstructed from the prediction residual
pyramid.

27. Subband coding
In subband coding, an image is decomposed into a set of bandlimited components, called subbands.
Since the bandwidth of the resulting subbands is smaller than that of the original image, the subbands can be downsampled without loss of information.

28. Perfect Reconstruction Filter
Z transform:
Goal: find H0, H1, G0 and G1 so that

29. Perfect Reconstruction Filter: ConditionsIf
Then

30. Perfect Reconstruction Filter Families
QMF: quadrature mirror filters
CQF: conjugate mirror filters

31. 2-D

32. Efficient Representation Using “Details”
details D3
details D2
details D1 L0

33. Efficient Representation Using Details (cont’d)
representation: L0 D1 D2 D3
in general: L0 D1 D2 D3…DJ
(analysis)
A wavelet representation of a function consists of
a coarse overall approximation
detail coefficients that influence the function at various scales.

34. Reconstruction (synthesis)
H3=L2+D3
H2=L1+D2
details D3
details D2
H1=L0+D1
details D1 L0

35. Example - Haar Wavelets
Suppose we are given a 1D "image" with a resolution of 4 pixels:
[ ]
The Haar wavelet transform is the following:
L0 D1 D2 D3

36. Example - Haar Wavelets (cont’d)
Start by averaging the pixels together (pairwise) to get a new lower resolution image:
To recover the original four pixels from the two averaged pixels, store some detail coefficients.

37. Example - Haar Wavelets (cont’d)
Repeating this process on the averages gives the full decomposition:

38. Example - Haar Wavelets (cont’d)
The Harr decomposition of the original four-pixel image is:
We can reconstruct the original image to a resolution by adding or subtracting the detail coefficients from the lower-resolution versions.
1 -1 2

39. Example - Haar Wavelets (cont’d)
Note small magnitude
detail coefficients! Dj
Dj-1
How to
compute Di ? D1 L0

40. Multiresolution Conditions
If a set of functions can be represented by a weighted sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j+1t - k):
high
resolution j
scale/frequency
localization
low
resolution
time localization

41. Multiresolution Conditions (cont’d)
If a set of functions can be represented by a weighted sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j+1t - k):
Vj: span of ψ(2jt - k):
Vj+1: span of ψ(2j+1t - k):

42. Nested Spaces Vj Basis functions: Vj : space spanned by ψ(2jt - k)
f(t) ϵ Vj
ψ(2t - k) V1 … Vj
ψ(2jt - k)
Multiresolution conditions  nested spanned spaces:
i.e., if f(t) ϵ V j then f(t) ϵ V j+1

43. How to compute Di ? f(t) ϵ Vj IDEA: define a set of basis
Functions that span the
differences between Vj

44. Orthogonal Complement Wj
Let Wj be the orthogonal complement of Vj in Vj+1
- i.e., all functions in Vj that are orthogonal to Wj
Vj+1 = Vj + Wj

45. How to compute Di ? (cont’d)
If f(t) ϵ Vj+1, then f(t) can be represented using basis functions φ(t) fromVj+1:
Vj+1
Alternatively, f(t) can be represented using two basis functions,
φ(t) from Vj and ψ(t) from Wj:
Vj+1 = Vj + Wj

46. How to compute Di ? (cont’d)
Think of Wj as a means to represent the parts of a function in Vj+1 that cannot be represented in Vj
differences
between
Vj and Vj+1
Vj, Wj

47. How to compute Di ? (cont’d)
 using recursion on Vj:
Vj+1 = Vj + Wj
Vj+1 = Vj-1+Wj-1+Wj = …= V0 + W0 + W1 + W2 + … + Wj
if f(t) ϵ Vj+1 , then:
V W0, W1, W2, …
basis functions basis functions

48. Wavelet expansion (Section 7.2)
Efficient wavelet decompositions involves a pair of waveforms (mother wavelets):
The two shapes are translated and scaled to produce wavelets (wavelet basis) at different locations and on different scales.
encode low
resolution info
encode details or
high resolution info
φ(t) ψ(t)
φ(t-k) ψ(2jt-k)

49. Wavelet expansion (cont’d)
f(t) is written as a linear combination of φ(t-k) and ψ(2jt-k) :
scaling function wavelet function
Note: in Fourier analysis, there are only two possible values of k ( i.e., 0 and π/2); the values j correspond to different scales (i.e., frequencies).

50. 1D Haar Wavelets φ(t) ψ(t) Haar scaling and wavelet functions:
computes average
computes details

51. 1D Haar Wavelets (cont’d)
Think of a one-pixel image as a function that is constant over [0,1)
We will denote by V0 the space of all such functions.
Example:

52. 1D Haar Wavelets
Think of a two-pixel image as a function having two constant pieces over the intervals [0, 1/2) and [1/2,1)
We will denote by V1 the space of all such functions.
Note that
Examples: ½ + =

53. 1D Haar Wavelets (cont’d)
V j represents all the 2j-pixel images
Functions having constant pieces over 2j equal-sized intervals on [0,1).
Note that
width: 1/2j
Examples:
ϵ Vj
ϵ Vj

54. 1D Haar Wavelets (cont’d)
V0, V1, ..., V j are nested
i.e., VJ … V2 V1
fine details
coarse details

55. 1D Haar Wavelets (cont’d)
Mother scaling function:
Let’s define a basis for V j : 1
note alternative notation:

56. 1D Haar Wavelets (cont’d)

57. 1D Haar Wavelets (cont’d)
Suppose Wj is the orthogonal complement of Vj in Vj+1
i.e., all functions in Vj which are orthogonal to Wj

58. 1D Haar Wavelets (cont’d)
Mother wavelet function:
Note that φ(x) . ψ(x) = 0 (i.e., orthogonal) 1 -1 / 1 1
= 0 . -1 /

59. 1D Haar Wavelets (cont’d)
Mother wavelet function:
Let’s define a basis ψ ji for Wj : 1 -1
note alternative notation:

60. 1D Haar Wavelets (cont’d)
basis for V 1 :
Note
that inner
product
is zero!
basis W 1 :
j=1

61. 1D Haar Wavelets (cont’d)
Basis functions ψ ji of W j
Basis functions φ ji of V j
form a basis in V j+1

62. 1D Haar Wavelets (cont’d)
V3 = V2 + W2

63. 1D Haar Wavelets (cont’d)
V2 = V1 + W1

64. 1D Haar Wavelets (cont’d)
V1 = V0 + W0

65. 1D Haar Wavelets (cont’d)

66. Example - Haar basis (revisited)

67. Decomposition of f(x)
f(x)=
φ0,2(x)
φ1,2(x)
φ2,2(x)
φ3,2(x) V2

68. Decomposition of f(x) (cont’d)
V1and W1
V2=V1+W1

69. Example - Haar basis (revisited)

70. Decomposition of f(x) (cont’d)
V0 ,W0 and W1
V2=V1+W1=V0+W0+W1

71. Example - Haar basis (revisited)

72. Example

73. Example (cont’d)

74. Filter banks (analysis)
The lower resolution coefficients can be calculated from the higher resolution coefficients by a tree-structured algorithm (filter bank).
Example:
ψ(2t - k)
a1k (j=1)
ψ(t - k)
a0k (j=0)

75. Filter banks (analysis) (cont’d)
The lower resolution coefficients can be calculated from the higher resolution coefficients by a tree-structured algorithm (filter bank).
Subband
encoding!
h0(-n) is a lowpass filter and h1(-n) is a highpass filter

76. Example - Haar basis (revisited)
[ ]
low-pass,
down-sampling
high-pass,
down-sampling
(9+7)/2 (3+5)/ (9-7)/2 (3-5)/2
V1 basis functions

77. Filter banks (analysis) (cont’d)

78. Example - Haar basis (revisited)
[ ]
low-pass,
down-sampling
high-pass,
down-sampling
V1 basis functions
(8+4)/ (8-4)/2

79. Convention for illustrating 1D Haar wavelet decomposition
x x x x x x … x x
average
detail …
re-arrange:
V1 basis functions
re-arrange:

80. Convention for illustrating 1D Haar wavelet decomposition (cont’d)
average
x x x x x x … x x
detail …

81. Orthogonality and normalization
The Haar basis forms an orthogonal basis
It can become orthonormal through the following normalization:
since

82. Examples of lowpass/highpass analysis filters
Haar h1 h0
Daubechies h1

83. Filter banks (synthesis)
The higher resolution coefficients can be calculated from
the lower resolution coefficients using a similar structure.

84. Filter banks (synthesis) (cont’d)

85. Examples of lowpass/highpass synthesis filters
Haar (same as
for analysis): g1 g0
Daubechies g1 +

86. 2D Haar Wavelet Transform
The 2D Haar wavelet decomposition can be computed using 1D Haar wavelet decompositions (i.e., 2D Haar wavelet basis is separable).
Two decompositions
Standard decomposition
Non-standard decomposition
Each decomposition corresponds to a different set of 2D basis functions.

87. Standard Haar wavelet decomposition
Steps
(1) Compute 1D Haar wavelet decomposition of each row of the original pixel values.
(2) Compute 1D Haar wavelet decomposition of each column of the row-transformed pixels.

88. Standard Haar wavelet decomposition (cont’d)
average
(1) row-wise Haar decomposition:
detail
re-arrange terms … …
x x x … x
… …
x x x x …
… …
… … …

89. Standard Haar wavelet decomposition (cont’d)
average
(1) row-wise Haar decomposition:
detail
from previous slide:
row-transformed result … … … …
… …
… … …

90. Standard Haar wavelet decomposition (cont’d)
average
detail
(2) column-wise Haar decomposition:
row-transformed result
column-transformed result … … … … …
… …
… … … …

91. Example …
… …
row-transformed result …
… …
re-arrange terms

92. Example (cont’d)
column-transformed result … …
… … …

93. 2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.
φ0,0(x)
ψ0,0(x)
ψ0,1(x)
ψ1,1(x)
Example:
V2=V0+W0+W1

94. 2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.
φ00(x), φ00(x)
ψ00(x), φ00(x)
ψ01(x), φ00(x)

95. 2D Haar basis of standard decompositionV2

96. Non-standard Haar wavelet decomposition
Alternates between operations on rows and columns.
(1) Perform one level decomposition in each row (i.e., one step of horizontal pairwise averaging and differencing).
(2) Perform one level decomposition in each column from step 1 (i.e., one step of vertical pairwise averaging and differencing).
(3) Repeat the process on the quadrant containing averages only (i.e., in both directions).

97. Non-standard Haar wavelet decomposition (cont’d)
one level, horizontal
Haar decomposition:
one level, vertical
Haar decomposition: … …
x x x … x
… …
x x x x … …
… …
… … … …
Note: averaging/differencing
of detail coefficients shown

98. Non-standard Haar wavelet decomposition (cont’d)
re-arrange terms … …
one level, vertical
Haar decomposition
on “green” quadrant
one level, horizontal
Haar decomposition
on “green” quadrant
… … … … … … …
… … …

99. Example
re-arrange terms … …
… … … …

100. Example (cont’d) … …
… … …

101. 2D Haar basis for non-standard decomposition
Defined through 2D scaling and wavelet functions:

102. 2D Haar basis for non-standard decomposition (cont’d)
Three sets of detail coefficients (i.e., subband encoding)
LL: average LL
Detail coefficients
LH: intensity variations along columns (horizontal edges)
HL: intensity variations along
rows (vertical edges)
HH: intensity variations along
diagonals

103. 2D Haar basis for non-standard decomposition (cont’d)V2

104. Forward/Inverse DWT (using textbook’s notation)
i=H,V,D
H LH
V HL
D HH LL

105. 2D DWT using filter banks (analysis)LH HH
H LH
V HL
D HH LL HL

106. Illustrating 2D wavelet decompositionLH HH LL LH HH HL LL HL
The wavelet transform can be applied again on the lowpass-lowpass version of the image, yielding seven subimages.

107. 2D IDWT using filter banks (synthesis)
H LH
V HL
D HH

108. Applications Noise filtering Image compression Image fusion
Fingerprint compression
Image fusion
Recognition
G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by Fusing Thermal Infrared and Visible Imagery", Image and Vision Computing, vol. 24, no. 7, pp , 2006.
Image matching and retrieval
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast Multiresolution Image Querying", SIGRAPH, 1995.

109. Image Querying “query by content” or “query by example”
Typically, the K best matches are reported.

110. Fast Multiresolution Image Querying
painted
low resolution
target
queries