1. Image Processing Xuejin Chen xjchen99@ustc.edu.cn Ref: http://fourier.eng.hmc.edu/e161/lectures/gradient/node10.html

2. Image processing operations Original image Increased contrast Change in hue Posterized (quantized colors) Blurred Rotated

3. Point operators Only depends on the pixel value – Plus, potentially, some globally collected information or parameters Brightness scaling Image composition – Matting – Blending Histogram equalization

4. Linear Filter Smoothing – Box, Bilinear, Gaussian

5. Linear Filter Smoothing – Box, Bilinear, Gaussian Edge – Sobel

6. Linear Filter Smoothing – Box, Bilinear, Gaussian Edge – Sobel Corner

7. Separable Filter Convolution of K-size kernel requires K 2 operations Can be sped up to 2K operations by – First performing a 1D horizontal convolution – Followed by a 1D vertical convolution

8. Separable Filter

9. Fourier transformation

10. Band-pass filters More sophisticated kernels – 1 Smoothing with a Gaussian filter – 2 Taking first or second derivatives Sobel Laplacian Coner

11. Band-pass filters Undirected second derivatives: Laplacian operator Laplacian of Gaussian (LoG) filter – Five point Laplacian

12. Steerable Filters Directional/Oriented filter – Sobel – Directional derivative A whole family of filters can be evaluated with very little cost by first convolving the image with (Gx, Gy)

13. Steerable Filters Second-order filter For directional Gaussian derivatives, it is possible to steer any order of derivative with a relatively small number of basis functions.

14. Steerable Filters Second-order filter Original image orientation map Original image with oriented structures enhanced.

15. Steerable Filters Fourth-order steerable filter test image -bars (lines) -step edges -different orientations average oriented energy dominant orientation oriented energy as a function of angle (Freeman and Adelson 1991)

16. Summed Area Table (Integral Image) Repeatedly convolved with different box filters – different sizes at – different locations Precompute the summed area table (Crow1984)

17. Summed Area Table (Integral Image) Compute the sum of any rectangle area easily Recursive filtering

18. Band-pass filters Sobel, Corner More sophisticated kernel: – Smooth image with a Gaussian filter – Take the first or second derivatives Laplacian Oriented Undirected

19. Laplacian of Gaussian (LoG)

20. LoG Discrete convolution kernel – Can be any size – Sum_elements = Zero

21. Laplacian of Gaussian (LoG)

22. Difference of Gaussian (DoG) Gaussian DoG

23. Difference of Gaussian (DoG)

25. LoG and DoG

28. DoG LoG

29. Laplacian for Edge Zero-crossing detection LoG DoG

30. Pyramids Change resolution – Upsampling (Interpolation) – Downsmapling (Decimation)

31. Interpolation Interpolation kernel h() with sampling rate r Bilinear

32. Interpolation Interpolation kernel h() with sampling rate r Bicubic interpolation

33. Bicubic Interpolation a specifies the derivative at x=1 Usually a=-1, best matches the frequency characteristics of a sinc function – A small amount of sharpening – Ringing (does not linearly interpolate straight lines Quadratic reproducing spline a=-0.5

34. Bicubic Interpolation BilinearCubic a=-1 Cubic a=-0.5 windowed sinc

35. Windowed sinc function Best quality interpolator (Usually) – Both preserves details in the lower resolution image and avoids aliasing

36. Windowed sinc function Best quality interpolator (Usually) – Both preserves details in the lower resolution image and avoids aliasing Ringing effect – Instead, repeatedly interpolate images by a small fractional amount

37. Decimation (Downsampling) Same kernel h(k,l) for both interpolation and decimation Avoid aliasing – Convolve the image with a low-pass filter

38. Decimation (Downsampling) Linear Binomial – Separating the high and low frequencies, – but leaves a fair amount of high-frequency detail, which can lead to aliasing after downsampling

39. Decimation (Downsampling) Linear Binomial Cubic – a=-1, – a=-0.5 Windowed sinc QMF-9 Jpeg2000 Sample rate = 2

40. Decimation (Downsampling) Cubic a=-1 – Sharpest but ringing QMF-9 and Jpeg2000 – Wavelet analysis filters – Useful for compression – More aliasing

41. Multi-resolution Representations Image pyramid – Accelerate coarse-to-fine search algorithms – Look for objects or patterns at different scales – Perform multi-resolution blending operations

42. Multi-resolution Representations Laplacian pyramid [Burt and Adelson’s (1983a)] – Best known and most widely used in computer vision

43. Laplacian Pyramid First: blur and subsample the original image with sample rate r=2 Five-tap kernel Octave pyramid

44. Laplacian pyramid First: blur and subsample the original image by sample rate = 2 Gaussian pyramid: Repeated convolutions of the binomial kernel converge to a Gaussian

45. Laplacian Pyramid

46. Actual computation of high-pass filter Results in perfect reconstruction when Q=I Laplacian image Gaussian image

47. Application: Image Blending regular splice pyramid blend

48. Low frequency part Medium frequency part High frequency part

51. Image Blending