Wavelet-Based Denoising Using Hidden Markov Models ELEC 631 Course Project Mohammad Jaber Borran

Some properties of DWT Primary Secondary Locality  Match more signals Multiresolution Compression  Sparse DWT’s Secondary Clustering  Dependency within scale Persistence  Dependency across scale

Probabilistic Model for an Individual Wavelet Coefficient Compression  many small coefficients few large coefficients S W pS(1) fW|S(w|1) pS(2) fW|S(w|2) fW (w)

Probabilistic Model for a Wavelet Transform Persistence  Hidden Markov Tree Model t f Ignoring the dependencies  Independent Mixture (IM) Model t f Clustering  Hidden Markov Chain Model

Parameters of HMT Model pmf of the root node transition probability (parameters of the) conditional pdfs e.g. if Gaussian Mixture is used q : Model Parameter Vector

Dependency between Signs of Wavelet Coefficients Signal Wavelet t T w1 T/2 w2

New Probabilistic Model for Individual Wavelet Coefficients Use one-sided functions as conditional probability densities S W pS(1) fW|S(w|1) pS(2) fW|S(w|2) fW (w) pS(3) fW|S(w|3) pS(4) fW|S(w|4)

Proposed Mixture PDF Use exponential distributions as components of the mixture distribution If m is even: If m is odd:

PDF of the Noisy Wavelet Coefficients Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance s2, then we have Noisy wavelet coefficient, If m is even: If m is odd:

Training the HMT Model y: Observed noisy wavelet coefficients s: Vector of hidden states q: Model parameter vector Maximum likelihood parameter estimation: Intractable, because s is unobserved (hidden).

Model Training Using Expectation Maximization Algorithm Define the set of complete data, x = (y,s) and then,

EM Algorithm (continued) State a posteriori probabilities are calculated using Upward-Downward algorithm Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U. Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.

Denoising MAP estimate:

Denoising (continued) Conditional mean estimate:

Conclusion Mixture distributions for individual wavelet coefficients can effectively model the non–Gaussian nature of the coefficients. Hidden Markov Models can serve as a powerful tool for wavelet-based statistical signal processing. One-sided exponential distributions for mixture components along with hidden Markov Tree model can achieve better performance in denoising.