1. Extensions of wavelets
ECE 802

2. M-Band Wavelet Systems
Generalization of dyadic wavelets
Scale factor of M
More flexible tiling of the time-frequency plane

3. Properties Scaling Equation: Existence and Orthogonality:
M-1 wavelets:

4. MRA
At each scale j, there are M-1 wavelet functions and one scaling function
If the wavelets are orthogonal to the scaling function at the same scale

5. Analysis and Synthesis
The expansion is
The filter bank structure will now have M branches
Gives a mixture of a logarithmic and linear frequency resolution.
Easier to design for M=2k

6. Wavelet Packets
M=2 results in a logarithmic frequency resolution. The low frequencies have narrow bandwidths and the high frequencies have wide bandwidths.
Wavelet packet system proposed by Coifman
Adjustable resolution of frequencies at high frequencies
Computational complexity O(NlogN)

7. Wavelet Packet Decomposition
In order to have higher resolution decomposition at high frequencies, iterate the highpass wavelet branch
Split both the lowpass and highpass bands at all stages
Evenly spaced frequency resolution
In DWT we consider the outputs of each channel.
In WPD, we have more outputs than inputs redundant system
Choose an independent set as basis (not one unique basis)

8. Optimization Criteria
Search based on minimizing a cost function on the transform coefficients.
Binary search algorithm for additive cost function
How do we choose the ‘best’ basis?
Shannon entropy
Thresholding the coefficients
Log Energy
Norm of the coefficients
Two approaches:
Choose a particular decomposition based on the signal class
Adapt the decomposition to each signal

9. Complexity
P(J): The number of J-scale orthonormal wavelet packet transforms
P(1)=1
P(J)=P(J-1)2+1
Application: FBI standard for fingerprint image compression

10. Haar Wavelet Packets

11. Wavelet Packet Tree

12. Optimization Functions
Shannon Entropy:
Norm:
Log-Energy:
Threshold Entropy: Number of times the coefficient is larger than a threshold

13. Example: Minimum Entropy Decomposition
Start with a constant original signal. w00 = ones(1,16)*0.25;
Compute entropy of original signal.
e00 = wentropy(w00,'shannon') e00 =
Then split w00 using the haar wavelet.
[w10,w11] = dwt(w00,'db1');
Compute entropy of approximation at level 1.
e10 = wentropy(w10,'shannon') e10 =
The detail of level 1, w11, is zero; the entropy e11 is zero. Due to the additivity property the entropy of decomposition is given by e10+e11= This has to be compared to the initial entropy e00= We have e10 + e11 < e00, so the splitting is interesting.
Now split w10 (not w11 because the splitting of a null vector is without interest since the entropy is zero).
[w20,w21] = dwt(w10,'db1');
We have w20=0.5*ones(1,4) and w21 is zero. The entropy of the approximation level 2 is
e20 = wentropy(w20,'shannon') e20 =
Again we have e < e10, so splitting makes the entropy decrease.
Then
[w30,w31] = dwt(w20,'db1'); e30 = wentropy(w30,'shannon')     e30 = [w40,w41] = dwt(w30,'db1')     w40 =     w41 = 0 e40 = wentropy(w40,'shannon')     e40 = 0
Perform wavelet packets decomposition of the signal s.
t = wpdec(s,4,'haar','shannon');

14. Best Tree

15. Overcomplete Representations, Frames, Redundant Transforms
There are many cases where a single basis is not effective for signal representation.
Example: Fourier basis is good for sinusoids, but bad for transients
Efficiency of the transform can be improved by combining several basis systems.
Combination of basis systemsOvercomplete
Collection of basis systems is called a dictionary.

16. Desired Criteria Sparsity: Efficient representation
Separation: Better ability to separate a mixture of signals
Superresolution: Higher resolution or detail compared to a single basis
Stability: Robust under noise, the selected atoms do not change
Speed

17. Definitions
Frame: Generalization of a basis, a collection of functions that span the vector space, but are not linearly independent
The frame condition:
0<A<B<∞
If A=B, tight frame
If A=B=1, orthonormal basis

18. Frame Examples
Tight Frame: 4 basis functions in 3-dimensional space

19. Matching Pursuit
Matching pursuit (MP) algorithm finds a sub-optimal solution to the problem of an adaptive approximation of a signal in a redundant set (dictionary) of functions.
Look for a linear expansion of a signal in terms of elements (atoms) of a dictionary.

20. Algorithm [Mallat, Zhang 1993]
At each step, try to find the element of the dictionary that ‘best’ fits the signal.
Energy Conservation:
For a complete dictionary as M∞, the residue should go to zero.
Stopping Criteria: Threshold the residue or pre-determine M

21. Dictionary
Commonly used dictionary: Gabor functions, dictionary of time-frequency atoms
General and compact model for oscillations
Compact time-frequency localization
Restrict the search to a range of time, frequency, and scale values

22. Applications
EEG Spike Parametrization:

23. Extensions
Multichannel MP: Jointly represent a class of signals using the same elements of the dictionary
Orthogonal Matching Pursuit (OMP):
Efficient greedy algorithm
Applies Gram-Schmidt orthogonalization to the selected atoms before computing the residue
The selected atoms are orthogonalized with respect to the residue
Faster convergence than MP

24. Basis Pursuit [Chen, Donoho]
Convex optimization: Find the representation that minimizes the l1 norm of the coefficients
Solved using linear programming
Nearly linear time
l1 norm guarantees sparsity, l2 norm does not (Method of Frames)

25. Examples
FM-Cosine signal:

26. Comparisons MP and OMP are iterative algorithms.
MP starts from an ‘empty’ signal model and builds it up one atom at a time
BP starts from a ‘full’ model and iteratively improves the full model.
Wavelet Packet Decomposition (Best Orthogonal Basis) focuses only on the orthogonal bases.
MOF l2 solution, not sparse, can be noisy.