1. Image
2016, Fall

2. Image Processing Quantization Pixel Operation Filtering Warping
Uniform Quantization
Halftoning & Dithering
Pixel Operation
Add random noise
Add luminance
Add contrast
Add saturation
Filtering
Blur
Detected edges
Warping
Scale
Rotate
Warp
Combining
Composite
Morph
Ref] Thomas Funkhouser,
Princeton Univ.

3. What is an Image? An image is a 2D rectilinear array of samples
A pixel is a sample, not a little square

4. Image Resolution Intensity resolution Spatial resolution
Each pixel has only “Depth” bits for colors/intensities
Spatial resolution
Image has only “Width” x “Height” pixels
Temporal resolution
Monitor refreshes images at only “Rate” Hz
Width
Width x Height
Depth
Rate
NTSC
640x480 8 30
Workstation
1280x1024 24 75
Film
3000 x 2000 12
Laser Printer
6600x5100 1 -
Depth
Height
Typical Resolutions

5. Source of Error Intensity quantization Spatial aliasing
Not enough intensity resolution
Spatial aliasing
Not enough spatial resolution
Temporal aliasing
Not enough temporal resolution
Real value
Sample value

6. Quantization Artifact due to limited intensity resolution
Frame buffers have limited number of bits per pixel
Physical devices have limited dynamic range

7. Uniform Quantization
반올림(Round)

8. Uniform Quantization
Image with decreasing bits per pixel

9. Reducing Effects of Quantization
Our eyes tend to integrate or average the fine detail into an overall intensity
Halftoning
A technique used in newspaper printing
The process of converting a continuous tone image into an image that can be printed with one color ink (grayscale) or four color inks (color)
Dithering
Dithering is used to create a wide variety of patterns for use as backgrounds, fills and shadings, as well as for creating halftones, and for correcting aliasing.
Halftones are created through a process called dithering
Dithering  Methods(방법론) , Halfton  result(결과물)

10. Classical Halftoning Use dots of varying size to represent intensities
Area of dots proportional to intensity in image

11. Halftoning A technique used in newspaper printing
Only two intensities are possible, blob of ink and no blob of ink
But, the size of the blob can be varied
Also, the dither patterns of small dots can be used

12. Classical Halftoning
Original Image
Halfton Image

13. Halfton patterns Use cluster of pixels to represent intensity
Trade spatial resolution for intensity resolution
2 by 2 pixel grid patterns
mask
3 by 3 pixel grid patterns

14. Halfton patterns

15. Dithering Distribute errors among pixels
Exploit spatial integration in our eye
Display greater range of perceptible intensities

16. Halftoning – Moire Patterns
Moire[more-ray] patterns
Repeated use of same dot pattern for particular shade results in repeated pattern (Perceived as a moire pattern)
result from overlapping repetitive elements
Instead, randomize halftone pattern

17. Halftoning – Moire Patterns

19. Signal Processing (1/2) Signal Image Real World : Continuous Signal
Digital World: Discrete Signal
Image
spatial domain
not temporal domain
intensity variation
over space

20. Signal Processing (2/2) Sampling and reconstruction process Sampling
The process of selecting a finite set of values from a signal
Samples (the selected values)
Reconstruction
recreate the original continuous signal from the samples

21. Sampling and Reconstruction

22. Aliasing In general Specifically, in graphics
Artifact due to under-sampling or poor reconstruction
Specifically, in graphics
Spatial aliasing
Temporal aliasing
Sampling below the Nyquist rate

23. Nyquist rate and aliasing
lower bound on the sampling rate
Sampling at the Nyquist rate
sampling rate > 2* max frequency
주파수(frequency): 초당 사이클의 횟수(Hz)

24. Spatial Aliasing
Artifact due to limited spatial resolution

25. Spatial Aliasing
Artifact due to limited spatial resolution

26. Temporal Aliasing Artifact due to limited temporal resolution
Spinning Backward
The wheel is spinning at 5 revolution per second
3rd row  appear to be spinning backward at revolution per second

27. Temporal Aliasing Artifact due to limited temporal resolution
Flickering

28. Anti-Aliasing Sample at higher rate
Not always possible
Doesn’t always solve problem
Pre-filter to form bandlimited signal
Form bandlimited function(low-pass filter)
Trades aliasing for blurring

29. Pixel Operations Grayscale Compute luminance L
L=0.30R+0.59G+0.11B (old)
 R G B (CRT, HDTV)
Original Image
Grayscale Image

30. Pixel Operations Negative Image Simply inverse pixel components
Ex) RGB(5, 157, 178)  RGB(250, 98, 77)
Original Image
Negative Image
(R, G, B)  (255-R, 255-G, 255-B)

31. Pixel Operations Adjusting Brightness Simply scale pixel components
Must clamp to range (e.g., 0 to 255)
(R, G, B)  (R, G , B) * scale

32. Pixel Operations Adjusting Contrast
Compute mean luminance L for all pixels
Luminance = 0.30* r *g *b
Scale deviation from L for each pixel component
Must clamp to range (e.g., 0 to 255)
R  L + (R-L)*scale
G  L + (G-L)*scale
B  L + (B-L)*scale

33. Pixel Operations
Changing Brightness
Changing Contrast

34. Pixel Operations Adjusting Saturation Color Convert RGB  HSV
Scale from S for each pixel component
Color Convert HSV  RGB
Must clamp to range (e.g., 0 to 255)

35. Filtering Convolution Spatial domain
Output pixel is weighted sum of pixels in neighborhood of input image
Patten of weights is the “filter”

36. Filtering
Convolution

37. Filtering Adjust Blurriness
Convolve with a filter whose entries sum to one
Each pixel becomes a weighted average of its neighbors
Filter Sum = 1

38. Filtering Edge Detection
Convolve with a filter that finds differences between neighbor pixels

39. Filtering
Sharpening
Convolve with a Sharpening filter

40. Filtering Embossing Convert the image into grayscale
Convolve with a Embossing filter
Plus 0.5 (128) for each pixel
Must clamp to range (e.g., 0 to 255)

41. Summary Image representation Halftoning and dithering
A pixel is a sample, not a little square
Images have limited resolution
Halftoning and dithering
Reduce visual artifacts due to quantization
Distribute errors among pixels
Exploit spatial integration in our eye
Sampling and reconstruction
Reduce visual artifacts due to aliasing
Filter to avoid undersampling
Blurring is better than aliasing

42. Image Processing Quantization Pixel Operation Filtering Warping
Uniform Quantization
Halftoning & Dithering
Pixel Operation
Add random noise
Add luminance
Add contrast
Add saturation
Filtering
Blur
Detected edges
Warping
Scale
Rotate
Warp
Combining
Composite
Morph
Ref] Thomas Funkhouser,
Princeton Univ.

43. Image Warping Move pixels of image Mapping Resampling Forward Reverse
Point sampling
Triangle Filter
Gaussian filter

44. Mapping Define transformation
Describe the destination (x,y) for every location (u,v) in the source (or vice-verse, if invertible)
Source (u, v)
Destination (x, y)

45. Example Mappings Scale by factor x = factor * u y = factor * v
Destination (x, y)
Source (u, v)

46. Example Mappings Rotate by q degrees x = ucosq - vsinq
y = usinq + vcosq
Source (u, v)
Destination (x, y)

47. Example Mappings Shear in X by factor Shear in Y by factor
x = u + factor * v
y = v
Shear in Y by factor
x = u
y = v + factor * u

48. Other Mappings
Any function of u and v
X = fx (u, v)
Y = fy (u, v)

49. Image Warping Implementation I
Forward mapping
for (int u =0; u < umax; u++) {
for (int v = 0; v < vmax; v++) {
float x = fx (u, v);
float y = fy (u, v);
dst (x, y) = src(u, v) ; }

50. Forward Mapping Iterate over source image
Many source pixels can map to same destination pixel
Some destination pixels may not be covered

51. Image Warping Implementation II
Reverse mapping
for (int x =0; x < xmax; x++) {
for (int y = 0; y < ymax; y++) {
float u = fx-1 (x, y);
float v = fy-1 (x, y);
dst (x, y) = src(u, v); }

52. Reverse Mapping Iterate over destination image Must resample source
May oversample, but much simpler

53. Resampling Evaluate source image at arbitrary (u,v)
(u, v) does not usually have integer coordinates

54. Resampling
Resampling
Point sampling
Triangle filter
Gaussain filter

55. Point Sampling Take value at closest pixel
int iu = trunc(u + 0.5)
int iv = trunc(v + 0.5)
dst(x, y) = src(iu, iv)
This method is simple, but it cause aliasing

56. Filtering Compute weighted sum of pixel neighborhood
Weights are normalized values of kernel function
Equivalent to convolution at samples

57. Triangle Filtering
Kernel is triangle function

58. Triangle Filtering (with width = 1)
Bilinearly interpolate four closest pixels
a = linear interpolation of src(u1, v2) and src(u2, v2)
b = linear interpolation of src(u1, v1) and src(u2, v1)
dst = linear interpolation of “a” and “b”

59. Gaussian Filtering
Kernel is Gaussian function
Filter width =2

60. Filtering Methods Comparison
Trade-offs
Aliasing versus blurring
Computation speed
Point
Bilinear
Gaussian

61. Image Warping Implementation
Reverse mapping
for (int x =0; x < xmax; x++) {
for (int y = 0; y < ymax; y++) {
float u = fx-1 (x, y);
float v = fy-1 (x, y);
dst (x, y) = resample_src(u, v, w); }

62. Example: Scale scale (src, dst, sx, sy) :
Float w = max (1.0/sx, 1.0/sy)
for (int x =0; x < xmax; x++) {
for (int y = 0; y < ymax; y++) {
float u = x / sx;
float v = y / sy;
dst (x, y) = resample_src(u, v, w); }

63. Example: Rotate Rotate (src, dst, theta) :
for (int x =0; x < xmax; x++) {
for (int y = 0; y < ymax; y++) {
float u = x*cos(-theta) - y*sin(-theta);
float v = x*sin(-theta) + y*cos(-theta);
dst (x, y) = resample_src(u, v, w); }

64. Example: Fun Swirl (src, dst, theta) :
for (int x =0; x < xmax; x++) {
for (int y = 0; y < ymax; y++) {
float u = rot(dist(x, xcenter)*theta);
float v = rot(dist(y, yecnter)*theta);
dst (x, y) = resample_src(u, v, w); }

65. Combining images Image compositing Image morphing Blue-screen mattes
Alpha channel
Image morphing
Specifying correspondence
Warping
Blending

66. Image Compositing Separate an image into “elements” Application
Render independently
Composite together
Application
Cel animation
Chroma-keying
Blue-screen matting

67. Blue Screen Matting Composite foreground and background images
Create background image
Create foreground image with blue background
Insert non-blue foreground pixels into background
Problem : no partial coverage

68. Alpha Channel Encodes pixel coverage informations Example: a = 0.3
a = 0  no coverage (or transparent)
a = 1  full coverage (or opaque)
0 < a < 1  partial coverage (or semi-transparent)
Example: a = 0.3

69. Composite with Alpha Composite with Alpha
Control the linear interpolation of foreground and background pixels when elements are composited

70. Pixels with Alpha Alpha channel convention
(r, g, b, a) represents a pixel that is a covered by
the color C = (r/a, g/a, b/a)
Color components are premultiplied by a
Can display (r, g, b) value directly
Closure in composition algebra

71. Semi-Transparent Objects
Suppose we put A over B background G
How much of B is blocked by A ? aA
How much of B show through A ?
(1-aA)
How much of G shows through both A and B ?
(1-aA) (1-aB)
αAA+ (1-αA )αBB+ (1-αA )(1-αB )G = A over[B over G]

72. Opaque Objects How do we combine 2 partially covered pixels?
3 possible colors (0, A, B)
4 regions (0, A, B, AB)

73. Composition Algebra
12 reasonable combinations

74. Image Morphing
Animate transition between two images

75. Cross-Dissolving Blend images with “over” operator
alpha of bottom image is 1.0
alpha of top image varies from 0.0 to 1.0
blend(i, j) = (1-t) src(i, j) + t dst(i, j) (0 <= t <= 1)

76. Image Morphing
Combines warping and cross-disolving

77. Feature Based Morphing
Beier & Neeley use pairs of lines to specify warp
Given p in dst image, where is p’ in source image ?

78. Feature Based Morphing
Feature Matching
Image Warping
Image Warping
Morphed Image

79. Feature Based Morphing
Warping
Black Or White-Michael Jackson

80. Image Processing Quantization Pixel Operation Filtering Warping
Uniform Quantization
Halftoning & Dithering
Pixel Operation
Add random noise
Add luminance
Add contrast
Add saturation
Filtering
Blur
Detected edges
Warping
Scale
Rotate
Warp
Combining
Composite
Morph
Ref] Thomas Funkhouser,
Princeton Univ.