1. Blending and Compositing Computational Photography Derek Hoiem, University of Illinois 09/23/14

2. hybridImage.m

3. pyramids.m

4. Hybrid results Pooja Bag

5. Donald Cha

6. Hao Gao

7. This Class How do I put an object from one image into another?

8. Image Compositing Some slides from Efros/Seitz

9. News Composites Original “Enhanced” Version http://www.guardian.co.uk/world/2010/sep/ 16/mubarak-doctored-red-carpet-picture

10. News Composites Original “Enhanced” Version Walski, LA Times, 2003

11. Three methods 1.Cut and paste 2.Laplacian pyramid blending 3.Poisson blending

12. Method 1: Cut and Paste

13. Method: Segment using intelligent scissors Paste foreground pixels onto target region

14. Method 1: Cut and Paste Problems: Small segmentation errors noticeable Pixels are too blocky Won’t work for semi-transparent materials

15. Feathering Near object boundary pixel values come partly from foreground and partly from background

16. Method 1: Cut and Paste (with feathering)

17. Alpha compositing + Output = foreground*mask + background*(1-mask)

18. Alpha compositing with feathering Output = foreground*mask + background*(1-mask)

19. Another example (without feathering) Mattes Composite by David Dewey Composite

20. Proper blending is key

21. Alpha Blending / Feathering 0 1 0 1 + = I blend =  I left + (1-  )I right

22. Affect of Window Size 0 1 left right 0 1

23. Affect of Window Size 0 1 0 1

24. Good Window Size 0 1 “Optimal” Window: smooth but not ghosted

25. How much should we blend?

26. Method 2: Pyramid Blending

27. 0 1 0 1 0 1 Left pyramidRight pyramidblend At low frequencies, blend slowly At high frequencies, blend quickly

28. laplacian level 4 laplacian level 2 laplacian level 0 left pyramidright pyramidblended pyramid

29. Method 2: Pyramid Blending Burt and Adelson 1983

30. Laplacian Pyramid Blending Implementation: 1.Build Laplacian pyramids for each image 2.Build a Gaussian pyramid of region mask 3.Blend each level of pyramid using region mask from the same level 4.Collapse the pyramid to get the final blended image Burt and Adelson 1983 Region mask at level i of Gaussian pyramid Image 1 at level i of Laplacian pyramid Pointwise multiply

31. Simplification: Two-band Blending Brown & Lowe, 2003 – Only use two bands: high freq. and low freq. – Blends low freq. smoothly – Blend high freq. with no smoothing: use binary alpha

32. Low frequency High frequency 2-band Blending

33. Linear Blending

34. 2-band Blending

35. Blending Regions

36. © Chris Cameron

37. Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background Perez et al. 2003

38. Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background Perez et al. 2003 Project 3!

39. Method 3: Poisson Blending A good blend should preserve gradients of source region without changing the background Treat pixels as variables to be solved – Minimize squared difference between gradients of foreground region and gradients of target region – Keep background pixels constant Perez et al. 2003

40. Example Source: Evan Wallace Gradient Visualization

41. Source: Evan Wallace + Specify object region

42. Gradient-domain editing Creation of image = least squares problem in terms of: 1) pixel intensities; 2) differences of pixel intensities Least Squares Line Fit in 2 Dimensions Use Matlab least-squares solvers for numerically stable solution with sparse A

43. Examples 1.Line-fitting: y=mx+b

44. Examples 2. Gradient domain processing 20 8020 8020 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 10 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 source image background image target image 10 v1v1 v3v3 v2v2 v4v4 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16

45. Other results Perez et al. 2003

46. What do we lose? Perez et al. 2003 Foreground color changes Background pixels in target region are replaced

47. Blending with Mixed Gradients Use foreground or background gradient with larger magnitude as the guiding gradient Perez et al. 2003

48. Project 3: Gradient Domain Editing General concept: Solve for pixels of new image that satisfy constraints on the gradient and the intensity – Constraints can be from one image (for filtering) or more (for blending)

49. Project 3: Reconstruction from Gradients 1.Preserve x-y gradients 2.Preserve intensity of one pixel Source pixels: s Variable pixels: v 1.minimize (v(x+1,y)-v(x,y) - (s(x+1,y)-s(x,y))^2 2.minimize (v(x,y+1)-v(x,y) - (s(x,y+1)-s(x,y))^2 3.minimize (v(1,1)-s(1,1))^2

50. Project 3 (extra): Color2Gray rgb2gray ? Gradient-domain editing

51. Project 3 (extra): NPR Preserve gradients on edges – e.g., get canny edges with edge(im, ‘canny’) Reduce gradients not on edges Preserve original intensity Perez et al. 2003

52. DVS Camera Links Pencil balance – https://www.youtube.com/watch?v=XVR5wEYkEG k https://www.youtube.com/watch?v=XVR5wEYkEG k Quad Copter https://www.youtube.com/watch?v=LauQ6LWTkxM Tennis https://www.youtube.com/watch?v=G1j2LLY5RIQ&t=2 7