1. Image Processing - in short
Image in  image out
image = 2D array of pixels (pixel = img[x,y])
color : 3 values / pixel
gray : 1 value / pixel
value usually 0.0 to 1.0 (float) or 0 to 255 (byte)
No 3D scene
no sub-pixel sampling
must work with given pixels
(no extra info)

2. 3 Kinds of Processes Local Neighborhood Global
out[x,y] depends on in[x,y] only
Neighborhood
out[x,y] depends on in[x,y] and pixels near in[x,y]
Global
out[x,y] depends on ALL pixels in
out

3. Local Processes out[x,y] = f ( in[x,y] ) adjust colors, brightness
correct for acquisition bias
eg, camera sensor with non-linear response
no spatial filtering
no sharpening, blurring

4. Brightness adjustment
out[x,y] = in[x,y] + 
(clamp to 0-1 range) in
K > 0
K < 0

5. Quantization out[x,y] = f( in[x,y] ) fixed threshold ordered dither 1
if in[x,y]>thresh[x,y]
then out=1
else out=0
out in 1

6. Contrast stretch out[x,y] = f( in[x,y] ) more contrast less contrast 1
shallow slope
steep slope in 1 in 1

7. Gamma Correction out[x,y] = in[x,y] in >1 <1 1 1 out out in 1in 1 in 1

8. Histogram Equalization
Frequent pixels  higher contrast
original
(low contrast)
high contrast
(detail lost)
histogram equalized
(more details)

9. Histogram Equalization
1 : get color frequencies
p(v) = (#pixels with color v) / (#total pixels)
2 : compute cummulative probability
C(v) = fraction of colors  v
= p(x) , for x v
3 : apply equalization
out = C(in)

10. Histogram Equalization
more frequent in
Pixel frequency
less frequent
less steep
Cummulative
pixel
probability
out
more steep

11. Luminance (colorgrey)
Y-component of CIE X-Y-Z color
in: 3 values/pixel
(R,G,B)
out: 1 value/pixel
Y  0.3 R G B

12. Hue Rotation 1: in(R,G,B) (H,S,V) 2: H  H + 
3: (H,S,V)  out(R,G,B)

13. Adjust Saturation 1: in(R,G,B) (H,S,V) 2: S  S + S
3: (H,S,V)  out(R,G,B)
S < 0 in
S > 0

14. Neighborhood Processes
out[x,y] = f (pixels near in[x,y])
linear
f( ) = linear combination of pixel colors
non-linear
f( ) = other function
non-linear combination
procedure

15. Linear filtering Equivalent to convolution Integral form:
out(x) = integral{ in(x - t) × g(t) × dt }
out = in * g
Discrete form:
out[x,y] = i,j in[x + i, y + j] × g[i, j]
out[x,y] = weighted sum of pixels
g( ) = “filter kernel”

16. Filter Kernels Matrix of weights example: box filter
1D: g(x) = 1/a if (0  x  a)
2D: g(x,y) = 1/a2 if (0  x  a) and (0  y  a)
discrete box filter (3 × 3), a=3, a2=9
g[x,y] =     ÷  
1/a a

17. Discrete 3 × 3 Box Filter in out ×1/9 × 1/9 × 1/9 × 1/9 × 1/9 × 1/9
out[x,y]
× 1/9
× 1/9
× 1/9

18. Discrete Convolution (2n+1) × (2n+1)
out[x,y] = in[x,y] * g[x,y]
s = 0; for (i=-n; i<=n; i++) for (j=-n; j<=n; j++) s += in[x+i, y+j] * g[i,j]; out[x,y] = s;
3 × 3 : n=1

19. Convolution examples Smoothing (box, gaussian filters)
Edge detection (derivative, Sobel, laplacian)
Edge enhancement
Gradient

20. Remapping Convolution may change range Example : edge detection
in : originally 0 to 1
out : should be 0 to 1 too
Example : edge detection
in pixel: to 1
filtered pixel: -1 to +1
out = (filtered + 1) / 2

21. Smoothing sharp edge brighter darker 10 11 10 9 10 5 6 5 4 5
scanline in:
1 × 3 box filter
scanline out:
(details lost) 10 10 10 10 8 7 5 5 5 5
softened edge

22. 5 × 5 Box Filter in
out
Boundaries blurred,
detail lost

23. Gaussian Filter 1D: g(x)= 1/(2) × e-x2/22
2D: g(x,y)= 1/(2) × e- (x2+y2) /22 4
Discrete 5x5 gaussian:
(=1.4)

24. Gaussian Filter in
5x5 box filter
5x5 gaussian
(edges sharper)

25. Edge Detection edge f(x) df(x)  f(x+h) - f(x) dx h scanline
out[x,y] = in[x+1,y] - in[x,y]
g[x,y] =      
df(x) dx
(must remap result to range 0-1)

26. Derivative sharp edges brighter darker brighter 10 11 10 9 10 5 6 5 10
scanline in:
values 0-10 -1 1
g[x]=
= {in[x] - in[x-1]}/2 + 5
scanline out: 5 5 4 4 5 2 5 4 7 5
dark pixel
bright pixel

27. Better Edge Detection Use neighboring scanlines
avoid single-pixel edges
g[x,y] =     (vertical edges)  
g[x,y] =     (horizontal edges)  

28. Sobel Edge Detection Neighboring scanlines have less weight
g[x,y] =     (vertical edges)  
g[x,y] =     (horizontal edges)  
endpoints of edges less blurred

29. Horizontal Derivativein
simple df dx
derivative
amplifies noise!
(why?)
Sobel df dx

30. Gradient P = [P/x, P/y] P/x = horizontal derivative
P is height of terrain
P is “uphill” vector
|P| is terrain slope
P/x = horizontal derivative
P/y = vertical derivative
|P| = gradient strength
ignores orientation
uphill
P/y
P/x

31. Oriented Edges in |P| , Sobel (all edges) simple df dx
(vertical edges only)

32. Gradient + Blur in |P| , Sobel features lost gaussian blur,
then |P| Sobel
less noise

33. Laplacian Non-oriented edges 2P = 2P/x2 + 2P/y2 (2nd derivatives)
scalar, not a vector
Estimate for 2nd derivative
g[x,y] =    |  
where did this come from?

34. Estimated 2nd Derivative
df = f '  f(x+h) - f(x) dx h
d2f = f ''  f '(x+h) - f '(x) dx h
 f(x+2h) - 2f(x+h) + f(x) h2
 f(x+h) - 2f(x) + f(x-h) (shifted left) h2
d2f  f(y+h) - 2f(y) + f(y-h) (in Y direction) dy h2

35. 2nd Derivative Edges original: two edges gaussian blur
inflection points: f '' = 0
first derivative
second derivative: zero crossing at edges

36. 2nd Derivative Edges More robust to noise Smoothing : removes noise
but blurs edge
2nd derivative: curvature
edge at inflection point
zero crossing

37. Edge Enhancement AKA “Sharpening” out[x,y] = out[x,y] - α × edge[x,y]
Find edges, amplify them, add them back
out[x,y] = out[x,y] - α × edge[x,y]
g[x,y] =  0 -α 0 -α 1+4α -α | (laplacian)  0 -α 0

38. Edge Enhancement in
out

39. Non-linear Processes
de-speckle
median
bilateral filter

40. De-speckle get in[x,y] and neighbours compute standard deviation 
if ( < threshold) out[x,y] = mean value (ie, apply blur)
removes single-pixel noise
dust spots
noise removal

41. Despeckle in
out

42. Median Filter get in[x,y] and neighbours out[x,y] = median value
also removes noise
preserves edges (does not bring in new colors)
better than blurring

43. Median Filter
median, 3x3 neighbourhood in
median, 5x5 neighbourhood

44. Fourier Filtering Fourier Transform modify in[], apply inverse F.T.
in[x]  F.T.  in[]
convert to frequency image
in[x] = color of xth pixel
in[] = strength of  frequency
modify in[], apply inverse F.T.
uses WHOLE image per pixel
global process

45. Frequency filtering sinusoid in(x,y) = 1+sin(1/x) filter: low-pass
high-pass
band-pass

46. Fourier Transform F.T. in two copies of frequency zero frequency
at center
repeating bands
higher
freqs.
on edges

47. Fourier Low-pass filter
F.T. in
high freqs
suppressed
out
inverse F.T.
blurred,
ringing

48. Fourier High-pass filter
F.T. in
low freqs
suppressed
out
inverse F.T.
edges
detected

49. Remove Periodic Artifacts
frequency
peaks in
F.T.
inverse F.T.
out
peaks
zeroed
out

50. Arithmetic operations
(Binary images) a b
NOT b
a OR b
a AND b
a XOR b
a - b