1. Directional Multiscale Modeling of Images
Duncan Po and Minh N. Do
University of Illinois at Urbana-Champaign

2. Motivation: Image Modeling
A randomly
generated image
A “natural” image
A simple and accurate image model is the key in many
image processing applications.

3. Background: Multiscale Modeling
Initially: Wavelet transform as a good decorrelator
Later: Incorporate dependencies across scale and space

4. New Image Representations
Idea: Efficiently represent smooth contours by directional and elongated basis elements
Idea: Successive refinement for edges in both location and direction

5. The Contourlet Transform
Contourlet transform (Do and Vetterli, 2002): extension of the wavelet transform using directional filter banks
Properties: sparse representation for smooth contours, efficient FB algorithms, tree data structure, …

6. The Contourlet Transform
Wavelet
Contourlet

7. Our Goals
Study statistics and properties of contourlet coefficients of natural images.
- Good understanding of properties would provide key
insights in developing contourlet-based applications.
Based on statistics and properties, develop a suitable model.
- Key Idea: Model all three fundamental parameters
of visual information: scale, space, and direction.
Apply model to applications.
- denoising and texture retrieval.

8. Marginal Distribution (Peppers)
Kurtosis = >> 3  non-Gaussian!!

9. Structure of Contourlet Coefficients
Each coefficient (X) has:
Parent (PX), Neighbors (NX), Cousins (CX)
We refer to all of them as Generalized Neighbors

10. Joint Statistics: Conditional Distributions (Peppers)
Parent
Neighbor
Cousin

11. Joint Statistics: Conditional Distributions on Far Neighbors (3 Coefficients away)
Peppers
Goldhill

12. Joint Statistics: Conditional Distributions (Peppers)
P(X| PX = px)
Kurtosis = 3.90
P(X| NX = nx)
Kurtosis = 2.90
P(X| CX = cx)
Kurtosis = 2.99

13. Joint Statistics: Quantitative
Use mutual information (Liu and Moulin, 2001)
A measure of how much information X conveys about Y
In estimation, quantifies how easy to estimate X given Y.
In compression, if it takes m bits to encode X, then given Y, it takes only m-I(X,Y) bits to encode X.

14. Joint Statistics: Quantitative
Histogram estimator for mutual information (Moddemeijer, 1989)
Multiple variables: use sufficient statistics (Liu and Moulin, 2001)

15. Results: I(X;.) PX NX CX /NX /CX all Lena 0.11 0.23 0.19 0.24 0.26
0.21
Barbara
0.14
0.58
0.39
0.59
0.40
0.56
Peppers
0.10
0.17
0.20
0.16

16. Lena PX NX CX 9-7,CD 0.11 0.23 0.19 Haar, CD 0.18 0.33 0.32 9-7, Haar
0.24
0.22
9-7, PKVA
0.15
4 directions
8 directions
16 directions NX
0.26
0.23
0.20 CX
0.14
0.19

17. Average Mutual Information against Individual Generalized Neighbors
I(X;PX)
I(X;NX)
I(X;CX)
Lena
0.11
0.09
0.08
Barbara
0.14
0.31
0.20
Peppers
0.07
0.06

18. Summary
Contourlet coefficients of natural images exhibit the following properties:
non-Gaussian marginally.
dependent on generalized neighborhood.
conditionally Gaussian conditioned on generalized neighborhood.
parents are (often) the most influential.
Next Step: Develop a simple statistical model that takes into account these properties

19. Hidden Markov Tree (HMT) Model
Developed for wavelets (Crouse et. al., 1998)
Transition Matrix A
State S
Transition Matrix
Coefficient u
State 1
State 2

20. Contourlet HMT Model Each tree has the following parameters
root state probabilities
state transition probability matrix between subband k in scale j and its parent subband in scale j-1
Gaussian standard deviations of subband q in scale p

21. Contourlet HMT Model
Wavelets
Contourlets

22. Denoising: Bayesian Estimation

23. Denoising Results: PSNR
Image
Noise Std. Dev.
Noisy
Wiener2
Wavelet Threshold
Wavelet HMT
Contourlet HMT
Lena 10
28.13
33.03
32.00
33.84
33.38 30
18.88
27.40
26.67
28.35
28.18 50
14.63
24.75
24.20
25.89
26.04
Barbara
28.11
31.38
29.90
31.36
29.18
18.72
24.95
23.76
25.11
25.27
14.48
22.57
21.96
23.71
23.74
Zelda
34.06
33.37
35.33
33.45
18.83
28.67
28.24
30.67
30.00
14.61
25.78
26.05
27.63
27.07

24. Denoising Results: Zelda
Noisy, w = 50
PSNR = 14.61
Wiener2, 5X5
PSNR = 25.78
Original
Wavelet Thresholding
T = 3w , PSNR = 26.05
Wavelet HMT
PSNR = 27.63
Contourlet HMT
PSNR = 27.07

25. Denoising Results: Barbara
Noisy, w = 51
PSNR = 14.48
Wiener2, 5X5
PSNR = 22.57
Original
Wavelet Thresholding
T = 3w , PSNR = 21.96
Wavelet HMT
PSNR = 23.71
Contourlet HMT
PSNR = 23.74

26. Contourlet Texture Retrieval System

27. Texture Retrieval: Use Kullback-Liebler Distance

28. Retrieval Results Average retrieval rates:
Wavelet HMT: 90.87% Contourlet HMT: 93.29%
Wavelets retrieve better (>5%)
Contourlets retrieve better (>5%)

29. Conclusions
Contourlets: new true two dimensional transform allows modeling of all three visual parameters: scale, space, and direction
Statistical measurements show:
Strong intra-subband, inter-scale, and inter-orientation dependencies
Conditioned on their neighborhood, coefficients are approximately Gaussian
Contourlet hidden Markov tree model
Promising results in denoising and retrieval