1. Lecture 5: Resampling, Compositing, and Filtering Li Zhang Spring 2008
CS559: Computer Graphics
Lecture 5: Resampling, Compositing, and Filtering
Li Zhang
Spring 2008

2. Today Image re-sampling (wrap-up) More on pre-multiplied alpha
Image filtering
Reading
Chapter 4, Shirley
Alvy Ray Smith, "Image Compositing Fundamentals"
Page 1-6, required, others optional
Tom Porter and Tom Duff, "Compositing Digital Images (optional)

3. Practical upsampling
When resampling a function (e.g., when resizing an image), you do not need to reconstruct the complete continuous function.
For zooming in on a function, you need only use a reconstruction filter and evaluate as needed for each new sample.
Here’s an example using a cubic filter:
Go back to show sinc function.
Put cubic functions at sampled locatoins
Summation it up
Try to evaluate this summation at x=1.5
This function has many terms.
Does the leftmost contribute the summation?
We only need to sum up the these curves How many terms will contribute to x?
What are those?

4. Practical upsampling This can also be viewed as:
putting the reconstruction filter at the desired location
evaluating at the original sample positions
taking products with the sample values themselves
summing it up

5. 2D Fourier transform Spatial domain Frequency domain
Brighter means bigger values
Most energy is concetrated near zero
High frequency coefficients are small.

6. Reconstruction filters in 2D
We can also perform reconstruction in 2D…
This is zoom.
Show how box and hedge kernel work,

7. Reconstruction filters in 2D
1D example:
g[n+1]
g[n]
n-1 n x
n+1
n+2
2D example:
g[n,m]
g[n+1,m]
g[n,m+1]
g[n+1,m+1]
(x,y)

8. Reconstruction filters in 2D
We’ve been looking at separable filters:
How might you use this fact for efficient resampling in 2D?

9. Image Downsampling
1/8
1/4
There is a application – resizing
Also this is an assignment.
This is a high resolution image, with many details.
Let’s say we want to resize the image to be something smaller.
Throw away every other row and column to create a 1/2 size image
- called image sub-sampling

10. Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom)
New structure is created
Subsampling assumes that the original singal does not have higher frequency that is larger that half the sampling fequency.
That is, if we do FT transform of this image, coef for frequeny that are larger that half of the sampling freq is zero.
1/2
1/4 (2x zoom)
1/8 (4x zoom)
Why does this look so crufty?
Minimum Sampling requirement is not satisfied – resulting in Aliasing effect

11. Practical downsampling
Downsampling is similar to upsampling, but filter has larger support and smaller amplitude.
Operationally:
given the downsampling rate, d, ratio of new sampling rate to old sampling rate
Choose reconstruction filter
Stretch the filter by 1/d and scale it down by d
Follow upsampling procedure (previous slides) to compute new values

12. Downsampling
Gaussian 1/2
G 1/4
G 1/8

13. Compare with subsampling
1/2
1/4 (2x zoom)
1/8 (4x zoom)

14. Explanation using Fourier Transform
Convolution
Show wrippling effect using the brown demo
We can use a large number of samples.
But suppose we can’t afford a large number of samples, what can we do?
We sample a smoother version. Otherwise, we’ll create artifacts.

15. Explanation using Fourier Transform
|F(sx,sy)|
f(x,y) *
h(x,y)
|H(sx,sy)|
Convolve and image with a Gaussian makes it smooth.
For each location, we computed a average of its neighbors.
How smooth it needs to be depends on the downsampling ratio
Now let’s look at how convolution works
g(x,y)
|G(sx,sy)|

16. Convolution Definition

17. Convolution Example Result h Filter g Function f
Show all the examples in brown web, explain the shift, product, and final integration results
© University of Wisconsin, CS559 Spring 2004
9/23/04

18. Convolution theorems
Convolution theorem: Convolution in the spatial domain is equivalent to multiplication in the frequency domain.

19. Explanation using Fourier Transform
Convolution
Show wrippling effect using the brown demo
We can use a large number of samples.
But suppose we can’t afford a large number of samples, what can we do?
We sample a smoother version. Otherwise, we’ll create artifacts.

20. Convolution theorems
Convolution theorem: Convolution in the spatial domain is equivalent to multiplication in the frequency domain.
Symmetric theorem: Convolution in the frequency domain is equivalent to multiplication in the spatial domain.

21. Why do we need pre-multiplied alpha?
“Over” Operator

22. Why do we need pre-multiplied alpha?
“Over” Operator

23. Why do we need pre-multiplied alpha?
“Over” Operator

24. Why do we need pre-multiplied alpha?
“Over” Operator

25. Why do we need pre-multiplied alpha?
“Over” Operator

26. Why do we need pre-multiplied alpha?
Simple reason. Simple equation. Reduce computation
Closed under computation. – use alpha_f = 0 as a special case to convice people
Given two colors without pre-multiplied alpha, we will end up with a new color with pre-multiplied color. Inconvenient.
Important when you want to composite the result image with a third image.
“Over” Operator

27. Image as a discreet function
Represented by a matrix

28. Convolution
0.5
0.5 I

29. What is image filtering
Modify the pixels in an image based on some function of a local neighborhood of the pixels. 5 1 4 7 3 10
Local image data
Modified image data
Some function

30. Linear filtering as convolution
Simplest: linear filtering.
Replace each pixel by a linear combination of its neighbors.
The prescription for the linear combination is the “convolution kernel”. 5 1 4 7 3 10
0.5
Local image data
kernel
Modified image data

31. Linear filtering (warm-up slide)? 1
kernel
original

32. Linear filtering (warm-up slide)1
kernel
original
Filtered
(no change)

33. Linear filtering ? 1
kernel
original

34. shift 1
kernel
original
shifted

35. Linear filtering ? 1
kernel
original

36. Blurring 1
kernel
original
Blurred

37. Blur Examples Pixel offset coefficient original 8 2.4 impulse 0.3
Pixel offset
coefficient
original 8
2.4
impulse
0.3
filtered

38. Blur Examples Pixel offset coefficient original 8 2.67 impulse 0.33
Pixel offset
coefficient
original 8
2.67
impulse
0.33
filtered
Pixel offset
coefficient 8 8
edge 4 4
0.33
original
filtered

39. Linear filtering (warm-up slide)? 2 1
kernel
kernel
original

40. Linear Filtering (no change)- 2 1
kernel
kernel
Filtered
(no change)
original

41. Linear Filtering - ? 2 1
kernel
kernel
original

42. (remember blurring) 1
kernel
original
Blurred.

43. Sharpening - 2 1
Sharpened original
original ) = 1 +( 1 - 1

44. Sharpening example 1.66 10.64 8 8 coefficient -2.64 -0.33 original
Sharpened
(differences are
accentuated; constant
areas are left untouched).

45. Sharpening
before
after