1. Image Processing and Reconstructions Tools
Ramesh Raskar
Mitsubishi Electric Research Labs
Cambridge, MA

2. Image Tools Gradient domain operations, Graph cuts,
Tone mapping, fusion and matting
Graph cuts,
Segmentation and mosaicing
Bilateral and Trilateral filters,
Denoising, image enhancement

3. Intensity Gradient in 1D
105
105 1 1
I(x)
G(x)
Gradient at x, G(x) = I(x+1)- I(x) Forward Difference

4. Reconstruction from Gradients
Intensity
105
105 ? ? 1 1
G(x)
I(x)
For n intensity values, about n gradients

5. Reconstruction from Gradients
Intensity
105
105 ? 1 1
G(x)
I(x)
1D Integration
I(x) = I(x-1) + G(x)
Cumulative sum

6. Intensity Gradient in 2D
Gradient at x,y as Forward Differences Gx(x,y) = I(x+1 , y)- I(x,y) Gy(x,y) = I(x , y+1)- I(x,y)
G(x,y) = (Gx , Gy)
Grad X
Grad Y

7. Intensity Gradient Vectors in Images

8. Image Intensity Gradients in 2D
Sanity Check: Recovering Original Image
Grad X
2D Integration
Solve Poisson Equation, 2D linear system
Grad Y

9. Intensity Gradient Manipulation
A Common Pipeline
Grad X
New Grad X
2D Integration
Gradient Processing
Grad Y
New Grad Y
Modify Gradients

10. Graph and Images
Credits: Jianbo Shi

11. Agrawala et al, Digital Photomontage, Siggraph 2004

13. Source images
Brush strokes
Computed labeling
Composite

14. Segmentation = Graph partition
Wij i j
Wij
V: graph node
E: edges connection nodes
Wij: Edge weight
Image pixel
Link to neighboring pixels
Pixel similarity

15. Minimum Cost Cuts in a graph
Cut: Set of edges whose removal makes a graph disconnected
Si,j : Similarity between pixel i and pixel j
Cost of a cut, A A

16. Graph Cuts for Segmentation and Mosaicing
Cut ~ String on a height field
Brush strokes
Computed labeling

17. Bilateral Filtering Input [Ben Weiss, Siggraph 2006]
Gaussian Smoothing
Log(Intensity) Bilateral Smoothing
[Ben Weiss, Siggraph 2006]

18. Start with Gaussian filtering
Here, input is a step function + noise
output
input

19. Start with Gaussian filtering
Spatial Gaussian f
output
input

20. Start with Gaussian filtering
Output is blurred
output
input

21. Gaussian filter as weighted average
Weight of x depends on distance to x
output
input

22. The problem of edges Here, “pollutes” our estimate J(x)
It is too different
output
input

23. Principle of Bilateral filtering
[Tomasi and Manduchi 1998]
Penalty g on the intensity difference
output
input

24. Bilateral filtering Spatial Gaussian f output input
[Tomasi and Manduchi 1998]
Spatial Gaussian f
output
input

25. Bilateral filtering Spatial Gaussian f
[Tomasi and Manduchi 1998]
Spatial Gaussian f
Gaussian g on the intensity difference
output
input

26. The weights are different for each output pixel
[Tomasi and Manduchi 1998]
The weights are different for each output pixel
output
input

27. Bilateral filtering is non-linear
[Tomasi and Manduchi 1998]
The weights are different for each output pixel
output
input

28. Bilateral Filtering Unilateral filtering Bilateral filtering
Smoothing using filtering
Bilateral filtering
Edge-preserving smoothing
Input
Gaussian Smoothing
Log(Intensity) Bilateral Smoothing
[Ben Weiss, Siggraph 2006]