1. המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory הטכניון - מכון טכנולוגי לישראל הפקולטה להנדסת חשמל Technion - Israel institute of technology department of Electrical Engineering 2007 winter Middle presentation (part B) Performed by: Cohen Ido, Volokitina Irina Instructor: Rivkin Ina, Technion Almog Asaf, Intel Denoising video in real time

2. Agenda Motivation and background. Optional concepts and solutions (Why bi-lateral filter?). Validation’s expectations and destination for middle presentation. Results’ validation (and more). Implementation in hardware (gidel and synplify). Time line

3. Image noise – definition and more The term noise usually refers to the high frequency random perturbations. corresponds to visible grain or particles present in the image. Generally caused by the electronic noise in the input device sensor and circuitry (e.g. scanner, digital camera).

4. Type of noise

5. The solution is DENOISING Removing noise from data is often the first step in data analysis. Denoising techniques should not only reduce the noise, but do so without blurring or changing the location of the edges.

6. Optional concepts and solutions Diffused image. Bilateral filter.

7. Bilateral Simple implementation. Noniterative. Local. YUV. (CIE) Diffusion Complicating implementation. Iterative. Not local. RGB. Bilateral and diffusion filtering comparison Conclusion: local and noniterative characterizations make us to choose in bilateral algorithm

8. Bilateral filter Bilateral filtering smooths images while preserving edges, by means of a nonlinear combination of nearby image values. The bilateral filter can enforce the perceptual metric underlying the CIE-Lab color space, and smooth colors and preserve edges in a way that is tuned to human perception.

9. VALIDATION Implement the algorithm with high level design tool MATLAB, and compare the denoising image and the original one.

10. Validation the solution’s correctness The indication for “success” or “fail” is MSE. The destination is to reduce the MSE by 50%.

11. Validation Process Noise (normal distribution) denoising unit (bilateral) + Input File (without noise) Output File MSE measurement RGBRGB RGBRGB RGBRGB RGBRGB RGBRGB RGBRGB RGBRGB configuration noisemaker MSE redaction percentage

12. Implementation and validation process MSE tester. Normal distribution noisemaker. Denoising unit. Stages in the implementation rgb2yuv function and opposite. Adaptive LPF. Fir select. Adaptive HPF.

13. Implementations stages Denoising unit (bilateral) RGB YUVYUV RGB RGBRGBRGBRGB YUV

14. Implementations stages Denoising unit (bilateral) RGB YUVYUV RGB RGBRGBRGBRGB YUV Adaptive LPF YUV configuration

15. Implementations stages Denoising unit (bilateral) RGB YUVYUV RGB RGBRGBRGBRGB YUV Adaptive LPF YUV configuration Fir select MUXMUX

16. Implementations stages Denoising unit (bilateral) RGB YUVYUV RGB RGBRGBRGBRGB YUV Adaptive LPF YUV configuration Fir select MUXMUX Adaptive HPF

17. expectations rgb2yuv and opposite – changing up to 10% in MSE. LPF – reducing MSE by at least 35% Adding fir select – reducing MSE by 5-10% HPF – reducing 10% MSE. Destination: reduction the MSE by 50%

18. Time Lines 3.1.07 1 week 10.1.07 Implement MSE tester, noisemaker, rbg2yuv, yuv2rgb 2 week 24.1.07 Implement LPF, fir sector 1 week 31.1.07 Implement HPF The target: till end of January we should validate the given bilateral filtering algorithm and reduce MSE by 50% Middle presentation

19. Results rgb2yuv and opposite – changing up to 10% in MSE => less than 1%. LPF – reducing MSE by at least 35% => 44% Adding fir select – reducing MSE by 5-10% => 20% × HPF – reducing 10% MSE => add about 10% to MSE. Bottom line: reduction the MSE by 50% (and more) achieved

20. Original

21. noisy

22. filtered

23. Suggested alternative solution Filtering and rescaling

24. original

25. noisy

26. Filtered 7x7 (LPF)

27. Filter 3x3 (LPF)

28. Filtered 3x3 (HPF)

29. final

30. Comparing Multi thresholds LPF (7x7) About 70% MSE reduction Better image quality Single threshold LPF (3x3) About 70% MSE reduction Fine image quality Original waySuggested way

31. Conclusion LPF (7x7) has better results but it is not likely to implement in video stream. Multi thresholds, different kernels technique gives us much better image quality.

32. Goals of project Implement denoise bilateral algorithm Denoise bilateral algorithm Noise No Noise = Video in Video out configuration

33. System Block Diagram Bilateral filterYCbCr RGB Denoise algorithm RGB YCbCr configuration Video in Video out Synplify implemented GIDEL implemented

34. Bilateral filter block Diagram HPF (3X3) LPF (3X3) MEMORYMEMORY Image analysis & Fir select Controller YCbCr configuration X X + Synplify implemented GIDEL implemented

35. Memory block Diagram MEMORYMEMORY YCbCr L[n] L[n-2] L[n-1] L[n] SYNPLIFY implemented YCbCr

36. HPF/LPF block HPF/LPF SYNPLIFY implemented L[n-2] L[n-1] L[n] FIR L[n-2] L[n-1] L[n] + L[n-1]

37. Image analysis & Fir select block diagram L[n-2] L[n-1] L[n] MIN last 3 clock MAX last 3 clock - >=< SYNPLIFY implemented

38. Controller block diagram SYNPLIFY implemented HPF coefficients LPF coefficients Threshold & weight parameter User definition

39. Time Lines 14.1.07 2 weeks 02.2007 Implement of RGB YCbCr 2 weeks 02.2007 Implement MEMORY 1 week 03.2007 Implement Controller The target: till end of March we should validate the given bilateral filtering algorithm and reduce MSE by 50% Middle presentation Implement HPF&LPF 03.2007 2 weeks Implement of Image analysis & Fir select Integration 3 weeks 04.2007 Final presentation