1. Smoothing Techniques in Image Processing
Prof. PhD. Vasile Gui
Polytechnic University of Timisoara

2. Content Introduction Brief review of linear operators
Linear image smoothing techniques
Nonlinear image smoothing techniques

3. Introduction Why do we need image smoothing?
What is “image” and what is “noise”?
Frequency spectrum
Statistical properties

4. Brief Review of Linear Operators [Pratt 1991]
Generalized 2D linear operator
Separable linear operator:
Space invariant operator:
Convolution sum

5. Brief Review of Linear Operators
h00
h01
h02
h10
h20
h21
h22
h11
h12
N+L-1 N
N-L+1
L-1 2 H G F
Geometrical interpretation of 2D convolution

6. Brief Review of Linear Operators
Matrix formulation
Unitary transform
Separable transform
Transform is invertible

7. Brief Review of Linear Operators
Basis vectors are orthogonal
Inner product and energy are conserved
Unitary transform: a rotation in N-dimensional vector space

8. Brief Review of Linear Operatorsx1 x2 1 2
2D vector space interpretation
of a unitary transform

9. Brief Review of Linear Operators
DFT unitary matrix

10. Brief Review of Linear Operators
1D DFT
2D DFT

11. Brief Review of Linear Operators
Circular convolution theorem
Periodicity of f,h and g with N are assumed

12. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator

13. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Separable
Can be computed recursively, resulting in roughly 4 operations per pixel
L pixels + 
m,n
m,n+1

14. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Optimality properties
Signal and additive white noise
Noise variance is reduced N times

15. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Unknown constant signal plus noise
Minimize MSE of the estimation g:

16. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
i.i.d. Gaussian signal with unknown mean.
Given the observed samples, maximize
Optimal solution: arithmetic mean

17. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Frequency response:
Non-monotonically decreasing with frequency

18. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
An example

19. Linear Image smoothing techniques Box filters
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Image smoothed with 33, 55, 99
and 11 11 box filters

20. Lena image filtered with
Linear Image smoothing techniques Box filters. Arithmetic mean LL operator
Lena image filtered with
5x5 box filter
Original Lena image

21. Linear Image smoothing techniques Binomial filters [Jahne 1995]
Computes a weighted average of pixels in the window
Less blurring, less noise cleaning for the same size
The family of binomial filters can be defined recursively
The coefficients can be found from (1+x)n

22. Linear Image smoothing techniques Binomial filters. 1D versions
As size increases, the shape of the filter is closer to a Gaussian one

23. Linear Image smoothing techniques Binomial filters. 2D versions

24. Linear Image smoothing techniques Binomial filters. Frequency response
Monotonically decreasing
with frequency

25. Linear Image smoothing techniques Binomial filters. Example
Original Lena image
Lena image filtered
with binomial 5x5 kernel
Lena image filtered
with box filter 5x5

26. Linear Image smoothing techniques Binomial and box filters
Linear Image smoothing techniques Binomial and box filters. Edge blurring comparison
Linear filters have to compromise smoothing with edge blurring
Step edge
Result of size L
box filter L
Size L binomial

27. Nonlinear image smoothing The median filter [Pratt 1991]
Block diagram f1 f2 fN .
Order samples
f(1)
f(2)
f(N)
select
f(m)
median
N is odd

28. Nonlinear image smoothing The median filter
Numerical example 6 7 3 8 2 4
2, 3, 3, 4, 6, 7, 7, 7, 8

29. Nonlinear image smoothing The median filter
Nonlinearity
median{ f1 + f2}  median{ f1} + median{ f2}.
However:
median{ c f } = c median{ f },
median{ c + f } = c + median{ f }.
The filter selects a sample from the window, does not average
Edges are better preserved than with liner filters
Best suited for “salt and pepper” noise

30. Nonlinear image smoothing The median filter
Noisy image
5x5 median filtered
5x5 box filter

31. Nonlinear image smoothing The median filter
Optimality
Grey level plateau plus noise. Minimize sum of absolute differences:
Result:
If 51% of samples are correct and 49% outliers, the median still finds the right level!

32. Nonlinear image smoothing The median filter
Caution: points, thin lines and corners are erased by the median filter
Test images
Results of 33 pixel median filter

33. Nonlinear image smoothing The median filter
Cross shaped window can correct some of the problems
Results of the 9 pixel cross shaped window median filter

34. Nonlinear image smoothing The median filter
Implementing the median filter
Sorting needs O(N2) comparisons
Bubble sort
Quick sort
Huang algorithm (based on histogram)
VLSI median

35. Nonlinear image smoothing The median filter
VLSI median block diagram a b
Min(a,b)
Max(a,b)
x(1)
x(2)
x(3)
x(4)
x(5)
x(6)
x(7) x1 x2 x3 x4 x5 x6 x7

36. Nonlinear image smoothing The median filter
Color median filter
There is no natural ordering in 3D (RGB) color space
Separate filtering on R,G and B components does not guarantee that the median selects a true sample from the input window
Vector median filter, defined as the sample minimizing the sum of absolute deviations from all the samples
Computing the vector median is very time consuming, although several fast algorithms exist

37. Nonlinear image smoothing The median filter
Example of color median filtering
5x5 pixels window
Up: original image
Down: filtered image

38. Nonlinear image smoothing The weighted median filter
The basic idea is to give higher weight to some samples, according to their position with respect to the center of the window
Each sample is given a weight according to its spatial position in the window.
Weights are defined by a weighting mask
Weighted samples are ordered as usually
The weighted median is the sample in the ordered array such that neither all smaller samples nor all higher samples can cumulate more than 50% of weights.
If weights are integers, they specify how many times a sample is replicated in the ordered array

39. Nonlinear image smoothing The weighted median filter
Numerical example for the weighted median filter 6 7 3 8 2 4
2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8 1

40. Nonlinear image smoothing The multi-stage median filter
Better detail preservation
mi = median( Ri ), i =1,2,3,4.
Result = median( m1,m2,m3,m4,m5 ) R1 R2 R3 R4 m5 

41. Nonlinear image smoothing Rank-order filters (L filters)
Block diagram
 sum y
f(1) f1 f2 fN .
ordering
f(2)
f(N)
a1
a2
aN
weights
y = a1f(1) + a2 f(2)+...+ aN f(N)

42. Nonlinear image smoothing Rank-order filters (L filters)
Some examples
a1=1
Min
aN=1
Max
am=1
Median
a1=.5
aN=.5
Mid range
Percentile
filters (rank selection):
0%,
50%
100%
Particular cases of grey level morphological filters

43. Nonlinear image smoothing Rank-order filters (L filters)
Optimality
Minkovski distance
Case r = 1: best estimator is median.
Case r = 2, best estimator is arithmetic mean.
Case r  , best estimator is mid-range.

44. Nonlinear image smoothing Rank-order filters (L filters)
Alpha-trimmed mean filter
 = (2Q + 1) / N, defines de degree of averaging
= 1 corresponds to arithmetic mean
 = 1 / N corresponds to the median filter
Properties: in between mean and median

45. Nonlinear image smoothing Rank-order filters (L filters)
Median of absolute differences trimmed mean
Better smoothing than the median filter and good edge preservation
M2 is a robust estimator of the variance

46. Nonlinear image smoothing Rank-order filters (L filters)
k nearest neighbour (kNN) median filter
Median of the k grey values nearest by rank to the central pixel
Aim: same as above

47. Nonlinear image smoothing Selected area filtering [Nagao 1980]
Kuwahara type filtering
Form regions Ri
Compute mean, mi and variance, Si for each region.
Result = mi : mi < mj for all j different from i, i,e. the mean of the most homogeneous region
Matsujama & Nagao
reg 1
reg 2
reg 3
reg 4
reg 5
reg 6
reg 7
reg 8

48. Nonlinear image smoothing Conditional mean
Pixels in a neighbourhood are averaged only if they differ from the central pixel by less than a given threshold:
L is a space scale parameter and th is a range scale parameter

49. Nonlinear image smoothing Conditional mean
Example with L=3, th=32

50. Nonlinear image smoothing Bilateral filter [Tomasi 1998]
Space and range are treated in a similar way
Space and range similarity is required for the averaged pixels
Tomasi and Manduchi [1998] introduced soft weights to penalize the space and range dissimilarity.
s() and r() are space and range similarity functions (Gaussian functions of the Euclidian distance between their arguments).

51. Nonlinear image smoothing Bilateral filter
The filter can be seen as weighted averaging in the joint space-range space (3D for monochromatic images and 5D – x,y,R,G,B - for colour images)
The vector components are supposed to be properly normalized (divide by variance for example)
The weights are given by:

52. Nonlinear image smoothing Bilateral filter
Example of Bilateral filtering
Low contrast texture has been removed
Yet edges are well preserved

53. Nonlinear image smoothing Mean shift filtering [Comaniciu 1999, 2002]
Mean shift filtering replaces each pixel’s value with the most probable local value, found by a nonparametric probability density estimation method.
The multivariate kernel density estimate obtained in the point x with the kernel K(x) and window radius r is:
For the Epanechnikov kernel, the estimated normalized density gradient is proportional to the mean shift:
S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere

54. Nonlinear image smoothing Mean shift filtering
The mean shift procedure is a gradient ascent method to find local modes (maxima) of the probability density and is guaranteed to converge.
Step1: computation of the mean shift vector Mr(x).
Step2: translation of the window Sr(x) by Mr(x).
Iterations start for each pixel (5D point) and tipically converge in 2-3 steps.

55. Nonlinear image smoothing Mean shift filtering
Example1.

56. Nonlinear image smoothing Mean shift filtering
Detail of a 24x40
window from the
cameraman image
a) Original data
b) Mean shift paths for some points
c) Filtered data
d) Segmented data

57. Nonlinear image smoothing Mean shift filtering
Example 2

58. Nonlinear image smoothing Mean shift filtering
Comparison to bilateral filtering
Both methods based on simultaneous processing of both the spatial and range domains
While the bilateral filtering uses a static window, the mean shift window is dynamic, moving in the direction of the maximum increase of the density gradient.

59. REFERENCES
D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th International Conference on Computer Vision, Kerkyra, Greece, Sept. 1999,
D. Comaniciu, P. Meer: Mean shift: a robust approach toward feature space analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002,
M. Elad: On the origin of bilateral filter and ways to improve it. IEEE trans. on Image Processing Vol. 11, No. 10, October 2002,
B. Jahne: Digital image processing. Springer Verlag, Berlin 1995.
M. Nagao, T. Matsuiama: A structural analysis of complex aerial photographs. Plenum Press, New York, 1980.
W.K. Pratt: Digital image processing. John Wiley and sons, New York 1991
C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images. Proc. Sixth Int’l. Conf. Computer Vision , Bombay,