Wavelet-Based Denoising Using Hidden Markov Models M. Jaber Borran and Robert D. Nowak Rice University

Some properties of DWT Primary Secondary Locality  Match more signals Multiresolution Compression  Sparse DWT’s Secondary Clustering  Dependency within scale Persistence  Dependency across scale

Probabilistic Model for an Individual Wavelet Coefficient Compression  many small coefficients few large coefficients S W pS(1) fW|S(w|1) pS(2) fW|S(w|2) fW (w)

Probabilistic Model for a Wavelet Transform Ignoring the dependencies  Independent Mixture (IM) Model Clustering  Hidden Markov Chain Model Persistence  Hidden Markov Tree Model

Parameters of HMT Model pmf of the root node transition probability (parameters of the) conditional pdfs e.g. if Gaussian Mixture is used q : Model Parameter Vector

Dependency between Signs of Wavelet Coefficients Signal Wavelet t T w1 T/2 w2

New Probabilistic Model for Individual Wavelet Coefficients Use one-sided functions as conditional probability densities S W pS(1) fW|S(w|1) pS(2) fW|S(w|2) fW (w) pS(3) fW|S(w|3) pS(4) fW|S(w|4)

Proposed Mixture PDF Use exponential distributions as components of the mixture distribution m even m odd

PDF of the Noisy Wavelet Coefficients Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance s2, then we have Noisy wavelet coefficient, m even m odd

Training the HMT Model y: Observed noisy wavelet coefficients s: Vector of hidden states q: Model parameter vector Maximum likelihood parameter estimation: Intractable, because s is unobserved (hidden).

Model Training Using Expectation Maximization Algorithm Define the set of complete data, x = (y,s) and then,

EM Algorithm (continued) State a posteriori probabilities are calculated using Upward-Downward algorithm Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U. Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.

Denoising MAP estimate:

Denoising (continued) Conditional Mean estimate:

Conclusion We observed a high correlation between the signs of the wavelet coefficients in adjacent scales. We used one-sided distributions as mixture components for individual wavelet coefficients. We used hidden Markov tree model to capture the dependencies. The proposed method achieves better MSE in denoising and the denoised signals are much smoother.