1. EE565 Advanced Image Processing Copyright Xin Li 20081 Why do we Need Image Model in the first place? Any image processing algorithm has to work on a collection (class) of images instead of a single one Mathematical model gives us the abstraction of common properties of the images within the same class Model is our hypothesis and images are our observation data In physics, can F=ma explain the relationship between force and acceleration?  In image processing, can this model fit this class of images?

2. EE565 Advanced Image Processing Copyright Xin Li 20082 Statistical vs. Deterministic They are different languages invented by mathematicians to facilitate the communication of scientific results (just like English vs. Chinese spoken by people in different countries) None is better than other – pick up the one you feel most comfortable with We adopt a statistical language most of the time in this class

3. EE565 Advanced Image Processing Copyright Xin Li 20083 The Curse of Dimensionality Even for a small-size image such as 64- by-64, we need to model it by a random process in 4096-dimensional space (R 4096 ) whose covariance matrix is sized by 4096-by-4096 More importantly, we ask ourselves: do we need to consider all pixels simultaneously?

4. EE565 Advanced Image Processing Copyright Xin Li 20084 A Simple Idea: Locality The conditional pdf is determined by a local neighborhood N past samples

5. EE565 Advanced Image Processing Copyright Xin Li 20085 Parametric vs. Nonparametric non-parametric sampling Input image XkXk 1 234 5 678 Parametric model

6. EE565 Advanced Image Processing Copyright Xin Li 20086 Spatial vs. Wavelet

7. EE565 Advanced Image Processing Copyright Xin Li 20087 Complete vs. Overcomplete

8. EE565 Advanced Image Processing Copyright Xin Li 20088 Marginal PDF of wavelet coefficients where Laplacian Gaussian P: shape parameter : variance parameter

9. EE565 Advanced Image Processing Copyright Xin Li 20089 Joint PDF of Wavelet Coefficients Neighborhood I(Q): {Left,Up,cousin and aunt} X= Y= Joint pdf of two correlated random variables X and Y Can you use this model to interpret why EZW works?

10. EE565 Advanced Image Processing Copyright Xin Li 200810 Good Bad Spatially Fixed vs. Adaptive Models

11. EE565 Advanced Image Processing Copyright Xin Li 200811 Locality Revisited Input image N past samples The definition of local neighborhood has to be relative

12. EE565 Advanced Image Processing Copyright Xin Li 200812 Application I: Image Denoising Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering From local to nonlocal denoising

13. EE565 Advanced Image Processing Copyright Xin Li 200813 Linear Frequency Weighting FT Power spectrum |X| 2

14. EE565 Advanced Image Processing Copyright Xin Li 200814 Spatially Adaptive Wiener Filtering of Wavelet Coefficients Basic assumption: image source is modeled by a nonstationary Gaussian process Signal variance is locally estimated from the windowed noisy observation data T T N=T 2 Recall

15. EE565 Advanced Image Processing Copyright Xin Li 200815 Wavelet Thresholding DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

16. EE565 Advanced Image Processing Copyright Xin Li 200816 Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

17. EE565 Advanced Image Processing Copyright Xin Li 200817 Translation Invariant Denoising Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

18. EE565 Advanced Image Processing Copyright Xin Li 200818 Further Improvements Gaussian scalar mixture (GSM) based denoising (Portilla et al.’ 2003) Instead of estimating the variance, it explicitly addresses the issue of uncertainty with variance estimation Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band coefficients (refer to the posted paper)

19. EE565 Advanced Image Processing Copyright Xin Li 200819 Nonlocal Patch-based Denoising WD T T -1 ThresholdingWD = Noisy patches Denoised patches

20. EE565 Advanced Image Processing Copyright Xin Li 200820 Application II: Texture Syntehsis Spatial-domain models Parametric autoregressive model Nonparametric resampling based Wavelet-domain models Histogram matching based Parametric models based joint-statistics

21. EE565 Advanced Image Processing Copyright Xin Li 200821 Spatial-Domain Parametric Texture Synthesis

22. EE565 Advanced Image Processing Copyright Xin Li 200822 Nonparametric Texture Synthesis

23. EE565 Advanced Image Processing Copyright Xin Li 200823 Wavelet-domain Histogram Matching

24. EE565 Advanced Image Processing Copyright Xin Li 200824 Wavelet-Domain Parametric Texture Models original synthesized

25. EE565 Advanced Image Processing Copyright Xin Li 200825 Other Applications Interpolation Spatial-domain covariance-based models PDE-based (nonlinear diffusion) models Coding Statistical modeling of wavelet coefficients Dual to wavelet-based image denoising Data hiding DCT-domain human vision model

26. EE565 Advanced Image Processing Copyright Xin Li 200826 Summary on Theory Image models are at the foundation of any image processing algorithm Statistical models help us deal with the uncertainty in observation data Appropriate image representation (e.g., prediction/transform) facilitates the modeling task Spatial adaptation is important – to have a good model for a wide class of images Localized models are popular and powerful but nonlocal models might prevail later

27. EE565 Advanced Image Processing Copyright Xin Li 200827 Summary on Practice MATLAB provides a user friendly platform for testing your ideas You can see what you have done Experimental efficiency is important Avoid loops and test small-size images C/C++ programming skills are a plus Efficient implementation could make a difference

28. EE565 Advanced Image Processing Copyright Xin Li 200828 Beyond Image Processing I will discuss something important than Wiener filtering or wavelet coding It is about you and your career If you are a MS student, your master thesis will be your selling point in your job hunting If you are a PhD student, you need to have a desire for first-class research It all depends on your perspective - how you want to look at it

29. EE565 Advanced Image Processing Copyright Xin Li 200829 Where is Your Talent? Outsider advantage: EZW, Turbo codes, Youtube, … A tradeoff among mathematical capabilities, physics intuitions, programming skills, management style … Selling your work could be even more important than doing the work itself

30. EE565 Advanced Image Processing Copyright Xin Li 200830 Follow your heart and enjoy what you do! Final Words