Dr. J. Shanbehzadeh M.HosseinKord Science and Research Branch of Islamic Azad University Machine Vision 1/49 slides

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Table of Contents 2 7-1) Gaussian Pyramids 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-5) Optical flow using Pyramids 7-1-1) Reduce 7-1-2) Expand 7-3-1) Image compression 7-3-2) Image composting 7-4) Interpolation

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Original Img Highest Resolution Lowest Resolution )Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Reduce 4 Level l Level l-1 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Reduce- Convolution 5 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Reduce (1D): Example 6 i = 2 Convolution Mask: [w(-2), w(-1), w(0), w(1), w(2)] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Reduce (1D) 7 g l = REDUCE (g l-1 ) Convolution Mask: [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Expand 8 n=1n=2 Notice: 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Expand(1D) 9 i = 4 [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] Involved weights  [c, a, c] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ) Expand(1D) 10 i = 3 [w(-2), w(-1), w(0), w(1), w(2)] [ c, b, a, b, c ] Involved weights  [b, b] 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Expand ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Convolution Mask 12 Separable Symmetric 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Convolution Mask 13 The sum of mask should be 1. All nodes at a given level must contribute the same total weight to the nodes at the next higher level. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Convolution Mask 14 a + 2b + 2c = 1 a + 2c = 2b b= ¼ c ac b b 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Convolution Mask 15 a= 0.5 TRINGULAR 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids a= 0.4 GAUSSIAN

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Gaussian Mask ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Gaussian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Gaussian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramids 20 Similar to edge detected images. Most pixels are zero. Can be used for image compression. L 1 = g 1 – EXPAND[g 2 ] L 2 = g 2 – EXPAND[g 3 ] L 3 = g 3 – EXPAND[g 4 ] L 4 = g 4 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramids 21 Lower in size and resolution Gaussian Pyramid Laplacian Pyramid 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids L 1 = g 1 – EXPAND[g 2 ] L 2 = g 2 – EXPAND[g 3 ] L 3 = g 3 – EXPAND[g 4 ] L 4 = g 4

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Applications of Laplacian pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Image compression 23 Compute Gaussian pyramid Compute Laplacian pyramid Code Laplacian pyramid 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Decode Laplacian pyramid. Compute Gaussian pyramid from Laplacian pyramid. g1 is reconstructed image. Image compression 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Image Compression (Entropy) ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Image Compression ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Combining Apple & Orange ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Algorithm 28 Generate Laplacian pyramid Lo of orange image. Generate Laplacian pyramid La of apple image. Generate Laplacian pyramid Lc by – copying left half of nodes at each level from apple and – right half of nodes from orange pyramids. Reconstruct combined image from Lc. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Interpolation

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Interpolation ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ‐ D Interpolation ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids 1=< x =<2

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, ‐ D Interpolation 32 Bilinear 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Bi ‐ linear Interpolation ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Optical flow using Laplacian Pyramid

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Why Lucas Kanade with Pyramids ? 35 Horn-Schunck and Lucas-Kanade optical method works only for small motion. If object moves faster, the brightness changes rapidly, 2x2 or 3x3 masks fail to estimate spatiotemporal derivatives. Pyramids can be used to compute large optical flow vectors. 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Lucas Kanade with Pyramids 36 LK for highest level of Laplacian pyramid  ui, vi Do the interpolation  u*i-1, v*i-1 Multiply by 2  u*i-1, v*i-1 Calculate ft according to displacement of u*i-1, v*i-1 LK for level l-1 of Laplacian pyramid  u’i-1, v’i-1 Accurate value of Optical flow is  ui-1 = u*i-1 + u’i-1 vi-1 = v*i-1 + v’i-1 Lucas Kanade Interpolation 7-1) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids

R. C. Gonzalez, and R. E. Woods, Digital Image Processing, New Jersey: Prentice Hall, 3 rd edition, Laplacian Pyramid ) Gaussian Pyramids 7-1-1) Reduce 7-1-2) Expand 7-1-3) Convolution Mask 7-2) Laplacian Pyramids 7-3) Applications of Laplacian pyramids 7-3-1) Image compression 7-3-2) Image composting 7-4) interpolation 7-5) Optical flow using Pyramids