1. Wavelet-Domain Video Denoising Based on Reliability Measures Vladimir Zlokolica, Aleksandra Piˇzurica and Wilfried Philips Circuits and Systems for Video Technology 2006, Transaction on IEEE Journals

2. Outline Introduction Proposed Video Denoising Method – Noise Estimation[36] – Reliability of MV Estimates – Motion Estimation – Recursive Temporal Filtering (RTF) – Adaptive Spatial Filtering Experimental Results Conclusions

3. Video denoising: spatial-temporal filters. Filtering: – Nonseparable (fully 3-D) [2]–[6] – Separable (2-D +1-D) [7]–[14] Combined: weighting for spatial and temporal? Spatial-first: ringing or blurring at high noise levels. Temporal-first We adopt the temporal-first approach, and develop a robust motion estimation method. Introduction

4. Proposed Video Denoising Method

5. Noise Estimation Assume that most frequent gradient amplitude is predominately caused by noise: It can be related to the input noise level: Finally, we recursively update estimated σ s : [36]V. Zlokolica, A. Pizurica, and W. Philips, “Wavelet domain noise-robust motion estimation and noise estimation for video denoising,”presented at the 1st Int. Workshop Video Process. Quality Metrics Consum. Electron., Scotssdale, AZ, Jan. 2005, Paper no. 200.

6. Parameters k 1 = 0.001, k 2 = 1.069 and k 3 =2.213. value is almost independent of image context and strongly corresponds to noise level. Noise Estimation [36]V. Zlokolica, A. Pizurica, and W. Philips, “Wavelet domain noise-robust motion estimation and noise estimation for video denoising,”presented at the 1st Int. Workshop Video Process. Quality Metrics Consum. Electron., Scotssdale, AZ, Jan. 2005, Paper no. 200.

7. Proposed Video Denoising Method

8. The proposed method uses a nondecimated wavelet transform[35]. Denote wavelet bands: WB={LL,HL,LH,HH} The spatial position as: r = (x,y) The decomposition level: l = 1,…,N (1 denotes the finest scale and N the coarsest) WB n : noisy band; WB tf : temporally filtered; WB stf : spatio-temporally filtered band.

9. Reliability of MV Estimates We define the MAD for each block s in the wavelet band WB (l) (r,t), as follows: – B s : the set of r belonging to the given 8x8 block. – WB: {LL,HL,LH,HH} – N: the maximum decomposition level.

10. Reliability of MV Estimates

11. Motion Estimation Wavelet-Domain Three-Step Method: – Estimates first MV field at the roughest scale and in the following steps refines the MV field: v pi ∈ {0,s,s’,t,t’}; P(0) = 0, P(v pi ) = 2.5 v (1) cx, v (1) cy ∈ {-8,-4,0,4,8}; v (2) cx, v (2) cy ∈ {-4,-2,0,2,4}; v (3) cx, v (3) cy ∈ {-2,-1,0,1,2};

12. Motion Estimation The cost function: – Where the constants C 1 and C 2 are optimized to obtain a noise robust and smooth MV field: C 1 = 1, C 2 = 1.45 – Assign more weight to the cost function for higher θ H and θ V for the tested nonzero correction.

13. Proposed Video Denoising Method

14. Recursive Temporal Filtering (RTF) Wavelet domain temporal filtering: When WB (l) tf (r-v b,t-1) has not all been filtered, noisy wavelet coefficient will propagate.

15. Recursive Temporal Filtering (RTF) To solve this problem, we update α (l) WB (s,t,σ n,v b ) with a correction function: – When α (l) WB (r-v b,t-1) → 0, α (l) WB * (r,t) →0.5: Both frames are noisy, perform simple averaging. – When α (l) WB (r-v b,t-1) → 1, α (l) WB * (r,t) → α (l) WB (r,t). – Furthermore, we apply α = 1 at least two time- recursions with reliable MVs have been applied in the last two frames.

16. Proposed Video Denoising Method

17. Adaptive Spatial Filtering Let δ(r c ) denote the neighborhood surrounding the central pixel r c : – Where T = MAD (l) WB (s,t,v b ), k m = 1. – The lower MAD the for the corresponding wavelet band WB (l) and block s, the less we will average.

18. Experimental Results Fig. 4. Results for the 29th frame of “Bicycle” sequence with added Gaussian noise (σ n = 15), processed by (c) WRTF filter and (d) 3RDS filter [16]. (a) Original image frame. (b) Noisy image frame. (c) WRTF (d) 3RDS filter [16]

19. Experimental Results Fig. 7. Results for the 75th frame of the processed “Flower Garden” sequence with added Gaussian noise (σ n = 15), by (c) the 3DWTF algorithm, and (d) the WRSTF algorithm. (a) Original image frame. (b) Noisy image frame. (c) 3DWTF (d) WRSTF

20. Experimental Results (a) (c) (b) (d)

21. Experimental Results

23. Conclusions We have proposed a new method for motion estimation and image sequence denoising in the wavelet domain. By robustly estimating motion and compensating, we efficiently remove noise without introducing visual artifacts. In future work, we intend to refine our motion estimation framework in order to deal with occlusion and moving block edges.