1. Discriminative Approach for Transform Based Image Restoration
SIAM – Imaging Science, July 2008
Discriminative Approach for Transform Based Image Restoration
Yacov Hel-Or Doron Shaked Gil Ben-Artzi
The Interdisciplinary Center
Israel
Bar-Ilan Univ.
Israel
HP Las

2. Motivation – Image denoising
- Can we clean Lena?

3. Broader Scope inverse problem Inpainting De-blurring De-noising
De-mosaicing
All the above deal with degraded images.
Their reconstruction requires solving an
inverse problem

4. Key point: Stat. Prior of Natural Images
likelihood
prior
Bayesian estimation:

5. Problem: P(x) is complicated to model
Defined over a huge dimensional space.
Sparsely sampled.
Known to be non Gaussian.
A prior p.d.f. of a 2x2 image patch
form Mumford & Huang, 2000

6. The Wavelet Transform Marginalizes Image Prior
Observation1: The Wavelet transform tends to de-correlate pixel dependencies of natural images.
W.T.

7. Observation2: The statistics of natural images are homogeneous.
Share the same statistics

8. Donoho & Johnston 94 Wavelet Shrinkage Denoising: Unitary Case
Degradation Model:
MAP estimation in the transform domain

9. Modify coefficients via scalar mapping functions
The Wavelet domain diagonalizes the system.
The estimation of a coefficient depends solely on its own measured value
This leads to a very useful property:
Modify coefficients via scalar mapping functions
where Bk stands for the k’th band

10. Shrinkage Pipe-line  x y + x= BTkk(Bky) k(Bky) Bky BTkk(Bky) BT1 B2
BT3
BT3 B2 y
x= BTkk(Bky)  
k(Bky)
Bky
BTkk(Bky)
xiB
yiB
Image
domain
Transform
domain
Image
domain
Result

11. Wavelet Shrinkage as a Locally Adaptive Patch Based Method
KxK
DCT
xiB
yiB
DCT-1
xiB
yiB
xiB
yiB

12. Can be viewed as shrinkage de-noising in a Unitary Transform (Windowed DCT).
KxK bands
WDCT
Unitary Transform
xiB
WDCT-1
yiB

13. Alternative Approach: Sliding Window
KxK
DCT
xiB
yiB
DCT-1
xiB
yiB
xiB
yiB

14. Can be viewed as shrinkage de-noising in a redundant transform (U. D
Can be viewed as shrinkage de-noising in a redundant transform (U.D. Windowed DCT).
UWDCT
Redundant Transform
xiB
UWDCT-1
yiB

15. How to Design the Mapping Functions?
Descriptive approach: The shape of the mapping function j depends solely on Pj and the noise variance .
noise variance ()
Modeling
marginal p.d.f.
of band j
MAP
objective yw

16. Commonly Pj(yB) are approximated by GGD:
for p<1
from: Simoncelli 99

17. Hard Thresholding Soft Thresholding Linear Wiener Filtering
MAP estimators for GGD model with three different exponents. The noise is additive Gaussian, with variance one third that of the signal.
from: Simoncelli 99

18. Due to its simplicity Wavelet Shrinkage became extremely popular:
Thousands of applications.
Thousands of related papers
What about efficiency?
Denoising performance of the original Wavelet Shrinkage technique is far from the state-of-the-art results.
Why?
Wavelet coefficients are not really independent.

19. Redundant Representation Joint (Local) Coefficient
Recent Developments
Since the original approach suggested by D&J significant improvements were achieved:
Original Shrinkage
Redundant Representation
Joint (Local) Coefficient
Modeling
Overcomplete transform
Scalar MFs
Simple
Not considered state-of-the-art
Multivariate MFs
Complicated
Superior results

20. What’s wrong with existing redundant Shrinkage?
Mapping functions:
Naively borrowed from the unitary case.
Independence assumption:
In the overcomplete case, the wavelet coefficients are inherently dependent.
Minimization domain:
For the unitary case MFs are optimized in the transform domain. This is incorrect in the overcomplete case (Parseval is not valid anymore).
Unsubstantiated
Improvements are shown empirically.

21. Questions we are going to address
How to design optimal MFs for redundant bases.
What is the role of redundancy.
What is the role of the domain in which the MFs are optimized.
We show that the shrinkage approach is comparable to state-of-the-art approaches where MFs are correctly designed.

22. Optimal Mapping Function:
Traditional approach: Descriptive
Modeling marginal p.d.f. of band k
MAP
objective x

23. Optimal Mapping Function:
Suggested approach: Discriminative
Off line: Design MFs with respect to a given set of examples: {xei} and {yei}
On line: Apply the obtained MFs to new noisy signals.
Denoising
Algorithm

24. Option 1: Transform domain – independent bands+ B1 Bk B1
BTk + B1 Bk B1
BTk

25. Option 2: Spatial domain – independent bands+ B1 Bk B1
BTk + B1 Bk B1
BTk

26. Option 3: Spatial domain – joint bands+ B1 Bk B1
BTk + B1 Bk B1
BTk

27. The Role of Optimization Domain
Theorem 1: For unitary transforms and for any set of {k}:
Theorem 2: For over-complete (tight-frame) and for any set of {k}: =

28. Unitary v.s. Overcomplete Spatial v.s. Transform Domain
Spatial domain
> =
> =
Transform domain

29. Is it Justified to optimized in the transform domain?
In the transform domains we minimize an upper envelope.
It is preferable to minimize in the spatial domain.

30. Optimal Design of Non-Linear MF’s
Problem: How to optimize non-linear MFs ?
Solution: Span the non-linear {k} using a linear sum of basis functions.
Finding {k} boils down to finding the span coefficients (closed form).
Mapping functions
k(y) y
For more details: see Hel-Or & Shaked: IEEE-IP, Feb 2008

31. z=[0,,0,1-r(z),r(z),0,]q = Sq(z)q z=r(z) qj+(1-r(z)) qj-1
Let zR be a real value in a bounded interval [a,b).
We divide [a,b) into M segments q=[q0,q1,...,qM]
w.l.o.g. assume z[qj-1,qj)
Define residue r(z)=(z-qj-1)/(qj-qj-1) q0 q1 qM
qj-1 qj a
r(z) b z
z=[0,,0,1-r(z),r(z),0,]q = Sq(z)q
z=r(z) qj+(1-r(z)) qj-1

32. The Slice Transform We define a vectorial extension: We call this the
Slice Transform (SLT) of z.
  
  
ith row

33. The SLT Properties z’ z =Sq(z) p q
Substitution property: Substituting the boundary vector q with a different vector p forms a piecewise linear mapping. q0 q1 q2 q3 q4 z p0 p1 p2 p3 p4 z’ z
=Sq(z) p q z’ z q1 q2 q3 z q4

34. Back to the MFs Design We approximate the non-linear {k} with
piece-wise linear functions:
Finding {pk} is a standard LS problem with a closed form solution!

35. Results

36. Training Images

37. Tested Images

38. Simulation setup Transform used: Undecimated DCT
Noise: Additive i.i.d. Gaussian
Number of bins in SLT: 15
Number of bands: 3x x10

39. Option 1 Option 2 Option 3 MFs for UDCT 8x8 (i,i) bands, i=1..4, =20
MFs are non monotonic anymore !
Option 3

40. Why considering joint band dependencies produces non-monotonic MFs ?
noisy image
Unitary MF
image space
Redundant MF

41. Comparing psnr results for 8x8 undecimated DCT, sigma=20.

42. 8x8 UDCT
=10

43. 8x8 UDCT
=20

44. 8x8 UDCT
=10

45. Comparison with BLS-GSM

46. Comparison with BLS-GSM

47. Other Degradation Models

48. JPEG Artifact Removal

49. JPEG Artifact Removal

50. Image Sharpening

51. Image Sharpening

52. Conclusions
New and simple scheme for over-complete transform based denoising.
MFs are optimized in a discriminative manner.
Linear formulation of non-linear minimization.
Eliminating the need for modeling complex statistical prior in high-dim. space.
Seamlessly applied to other degradation problems as long as scalar MFs are used for reconstruction.

53. Recent Results
What is the best transform to be used (for a given image or for a given set)?

54. The End
Thank You