1. Wavelet-Based Denoising Using Hidden Markov Models
M. Jaber Borran and Robert D. Nowak
Rice University

2. Some properties of DWT Primary Secondary Locality  Match more signals
Multiresolution
Compression  Sparse DWT’s
Secondary
Clustering  Dependency within scale
Persistence  Dependency across scale

3. Probabilistic Model for an Individual Wavelet Coefficient
Compression  many small coefficients
few large coefficients S W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)

4. Probabilistic Model for a Wavelet Transform
Ignoring the dependencies 
Independent Mixture (IM) Model
Clustering 
Hidden Markov Chain Model
Persistence 
Hidden Markov Tree Model

5. Parameters of HMT Model
pmf of the root node
transition probability
(parameters of the)
conditional pdfs
e.g. if Gaussian Mixture is used
q : Model Parameter Vector

6. Dependency between Signs of Wavelet Coefficients
Signal
Wavelet t T w1
T/2 w2

7. New Probabilistic Model for Individual Wavelet Coefficients
Use one-sided functions as conditional probability densities S W
pS(1)
fW|S(w|1)
pS(2)
fW|S(w|2)
fW (w)
pS(3)
fW|S(w|3)
pS(4)
fW|S(w|4)

8. Proposed Mixture PDF
Use exponential distributions as components of the mixture distribution
m even
m odd

9. PDF of the Noisy Wavelet Coefficients
Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance s2, then we have
Noisy wavelet coefficient,
m even
m odd

10. Training the HMT Model y: Observed noisy wavelet coefficients
s: Vector of hidden states
q: Model parameter vector
Maximum likelihood parameter estimation:
Intractable, because s is unobserved (hidden).

11. Model Training Using Expectation Maximization Algorithm
Define the set of complete data, x = (y,s)
and then,

12. EM Algorithm (continued)
State a posteriori probabilities are calculated using Upward-Downward algorithm
Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U.
Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.

13. Denoising
MAP estimate:

14. Denoising (continued)
Conditional Mean estimate:

18. Conclusion
We observed a high correlation between the signs of the wavelet coefficients in adjacent scales.
We used one-sided distributions as mixture components for individual wavelet coefficients.
We used hidden Markov tree model to capture the dependencies.
The proposed method achieves better MSE in denoising and the denoised signals are much smoother.