1. Dr. J. Shanbehzadeh Shanbehzadeh@gmail.com M.HosseinKord Science and Research Branch of Islamic Azad University Machine Vision 1/49 slides

2. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Table of Contents 2 7-1) Gaussian Pyramids 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-5) Optical flow using Pyramids 7-1-1) Reduce 7-1-2) Expand 7-3-1) Image compression 7-3-2) Image composting 7-4) Interpolation

3. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 3 Original Img Highest Resolution Lowest Resolution...... 7-1)Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

4. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-1) Reduce 4 Level l Level l-1 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

5. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-1) Reduce- Convolution 5 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

6. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-1) Reduce (1D): Example 6 i = 2 Convolution Mask: [w(-2), w(-1), w(0), w(1), w(2)] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

7. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-1) Reduce (1D) 7 g l = REDUCE (g l-1 ) Convolution Mask: [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

8. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-2) Expand 8 n=1n=2 Notice: 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

9. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-2) Expand(1D) 9 i = 4 [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] Involved weights  [c, a, c] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

10. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 7-1-2) Expand(1D) 10 i = 3 [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] Involved weights  [b, b] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

11. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Expand 11 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

12. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Convolution Mask 12 Separable Symmetric 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

13. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Convolution Mask 13 The sum of mask should be 1. All nodes at a given level must contribute the same total weight to the nodes at the next higher level. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

14. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Convolution Mask 14 a + 2b + 2c = 1 a + 2c = 2b b= ¼ c ac b b 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

15. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Convolution Mask 15 a= 0.5 TRINGULAR 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids a= 0.4 GAUSSIAN

16. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Gaussian Mask 16 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

17. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Gaussian Pyramid 17 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

18. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Gaussian Pyramid 18 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

19. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramids

20. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramids 20 Similar to edge detected images. Most pixels are zero. Can be used for image compression. L 1 = g 1 – EXPAND[g 2 ] L 2 = g 2 – EXPAND[g 3 ] L 3 = g 3 – EXPAND[g 4 ] L 4 = g 4 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

21. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramids 21 Lower in size and resolution Gaussian Pyramid Laplacian Pyramid 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids L 1 = g 1 – EXPAND[g 2 ] L 2 = g 2 – EXPAND[g 3 ] L 3 = g 3 – EXPAND[g 4 ] L 4 = g 4

22. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Applications of Laplacian pyramids

23. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Image compression 23 Compute Gaussian pyramid Compute Laplacian pyramid Code Laplacian pyramid 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

24. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 24 Decode Laplacian pyramid. Compute Gaussian pyramid from Laplacian pyramid. g1 is reconstructed image. Image compression 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

25. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Image Compression (Entropy) 25 7.6 4.4 5.0 5.6 6.2 0.77 1.9 3.3 4.2 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

26. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Image Compression 26 1.58 0.73 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

27. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Combining Apple & Orange 27 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

28. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Algorithm 28 Generate Laplacian pyramid Lo of orange image. Generate Laplacian pyramid La of apple image. Generate Laplacian pyramid Lc by – copying left half of nodes at each level from apple and – right half of nodes from orange pyramids. Reconstruct combined image from Lc. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

29. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Interpolation

30. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Interpolation 30 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

31. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 1 ‐ D Interpolation 31 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids 1=< x =<2

32. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. 2 ‐ D Interpolation 32 Bilinear 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

33. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Bi ‐ linear Interpolation 33 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

34. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Optical flow using Laplacian Pyramid

35. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Why Lucas Kanade with Pyramids ? 35 Horn-Schunck and Lucas-Kanade optical method works only for small motion. If object moves faster, the brightness changes rapidly, 2x2 or 3x3 masks fail to estimate spatiotemporal derivatives. Pyramids can be used to compute large optical flow vectors. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

36. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Lucas Kanade with Pyramids 36 LK for highest level of Laplacian pyramid  ui, vi Do the interpolation  u*i-1, v*i-1 Multiply by 2  u*i-1, v*i-1 Calculate ft according to displacement of u*i-1, v*i-1 LK for level l-1 of Laplacian pyramid  u’i-1, v’i-1 Accurate value of Optical flow is  ui-1 = u*i-1 + u’i-1 vi-1 = v*i-1 + v’i-1 Lucas Kanade Interpolation 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

37. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramid 37 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

38. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramid 38 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

39. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramid 39 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

40. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramid 40 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

41. R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, 2008. Laplacian Pyramid 41 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids