Image Enhancement in the Frequency Domain Spring 2005, Jen-Chang Liu

Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

Background 1807, French math. Fourier Any function that periodically repeats itself can be expressed as the sum of of sines and/or cosines of different frequencies, each multiplied by a different coefficient (Fourier series)

Periodic function f(t) = f(t+T), T: period (sec.) 1/T: frequency (cycles/sec.)

Periodic function f Frequency Weight f 1 w 1 f 2 w 2 f 3 w 3 f 4 w 4

How to measure weights? Assume f 1, f 2, f 3, f 4 are known How to measure w 1, w 2, w 3, w 4 ? min Minimize squared error

Minimize MSE calculation min

Orthogonal condition f 1 and f 2 are orthogonal if f 1, f 2, f 3, f 4 are orthogonal to each other 正交

Minimization calculation To satisfy min We have => Recall in linear algebra: projection

Weight = Projection magnitude Represent input f(x) with another basis functions v Vector space (1,0) projection Functional space f f1f1

Summary 1 A function f can be written as sum of f 1, f 2, f 3, … If f1, f2, f3, … are orthogonal to each other Weight (magnitude)

Summary 1: sine, cosine bases Let f 1, f 2, f 3, … carry frequency information Let them be sines and cosines n, k:integers => They all satisfy orthogonal conditions

Summary 1: orthogonal

Fourier series For (Assume periodic outside) DC 頻率 =1 頻率 =2 頻率 =3

Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

Correlation with different phase Weight calculation 相關係數 f1f1 f 相位

Correlation with different phase (cont.) Weight calculation 相關係數 ? f1f1 f

Deal with phase: method 1 For example, expand f(t) over the cos(wt) basis function Consider different phases  0 22  Corr(  Problem: weight(w,  )

Deal with phase: method 2 Complex exponential as basis j 1 real With frequency w: Advantage: Derive magnitude and phase  simultaneously

Deal with phase 2: example Input magnitude phase

Fourier series with phase For (Assume periodic outside) DC 頻率 k=1 k=2 k=3 Complex weight

Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

Fourier transform Functions that are not periodic can be expressed as the integral of sines and/or cosines multiplied by a weighting functions Frequency up to infinity Perfect reconstruction Functions  --  Fourier transform Operation in frequency domain without loss of information

1-D Fourier Transform Fourier transform F(u) of a continuous function f(x) is: Inverse transform: Forward Fourier transform:

2-D Fourier Transform Fourier transform F(u,v) of a continuous function f(x,y) is: Inverse transform: x y u v F

Future development 1950, fast Fourier transform (FFT) Revolution in the signal processing Discrete Fourier transform (DFT) For digital computation

1-D Discrete Fourier Transform f(x), x=0,1, …,M-1. discrete function F(u), u=0,1, …,M-1. DFT of f(x) Inverse transform: Forward discrete Fourier transform:

0123 … M-1 f(x) x Assume periodic outside 0123M-1 x 頻率 = M-1 x 頻率 =1/M ( 基頻 ) … … 0123 M-1 x 頻率 =2/M … … 0123 M-1 x 頻率 =(M-1)/M …

Frequency Domain 頻率域 Where is the frequency domain? j 1 Euler ’ s formula: frequency u F(u)

Fourier transform

Physical analogy Mathematical frequency splitting Fourier transform Physical device Galss prism 三稜鏡 Split light into frequency components

F(u) Complex quantity? Polar coordinate real imaginary m magnitude phase Power spectrum

Some notes about sampling in time and frequency axis Time index Frequency index Also follow reciprocal property

Extend to 2-D DFT from 1-D 2-D: x-axis then y-axis

Complex Quantities to Real Quantities Useful representation magnitude phase Power spectrum

Some notes about 2-D Fourier transform Frequency axis x y u v u v Fshift 0

DFT: example log(F)

Properties in the frequency domain Fourier transform works globally No direct relationship between a specific components in an image and frequencies Intuition about frequency Frequency content Rate of change of gray levels in an image

+45,-45 degree artifacts

Outline Introduction to the Fourier Transform and Frequency Domain Magnitude of frequencies Phase of frequencies Fourier transform and DFT Filtering in the frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation of Fourier transform

Filtering in the frequency domain Filter (mask) in the spatial domain Apply a filter transfer function in the frequency domain Output(u,v)=Input(u,v) H(u,v) Suppress certain frequencies while leaving others unchanged 乘法

Filtering in the frequency domain even dim. gray-level scaling … complex real real part

Basic filters Notch filter Make a hole(notch) in the frequency domain Ex. DC

Notch filter: result

Basic filters (cont.) Low-pass filter Attenuates high frequencies while passing low frequencies smoothing High-pass filter Attenuates low frequencies while passing high frequencies sharpening

Low-pass High-pass

Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

Periodic function f(t) = f(t+T), T: period 1/T: frequency How to measure?

Auto-correlation function Recall correlation function 相關係數 f1f1 f

Auto-correlation function (cont.) 自相關係數  Corr(  )   T 

Example: audio file phone = wavread('phone.wav'); wavplay(phone);

Example: autocorrelation corr=autocorr(phone);

Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

Function scaling and frequency P=2  P=  P=1 sin  t)

P=1 P=8 P=4 P=8/3? P=3

P=2 P=8/5 ? If t is continuous in time, then has period =8/5 However, now t ={0,1,2, …,} is discrete, sampling period 1>4/5 P=8/3? P=3

Half of the DFT frequencies is redundant x=0,1, …,M-1, u=0,1, …,M-1 u<2/M u |F(u)| 01M2M2 … M-1 … M2M2 …

For real signal Forward DFT: That is,Magnitude: |F(M-u)| = |F(u)|, u<2/M Phase: F(M-u) = - F(u)

DFT example for real signal f(x) |F(u)| angle(F(u))

DFT implementation m = 0:1:M-1; F = 1/M*f*exp(-j*2*pi*1/M*m'*m); Matlab code:

Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

2-D Translation

2-D Translation (cont.) M N M/2 N/2

Periodicity and conjugate symmetry Periodicity F(u,v)=F(u+M,v)=F(u,v+N)=F(u+M,v+N) Conjugate symmetry

M N M/2 N/2 0

Separability of 2-D DFT

Distribution and Scaling Distribution over addition Scaling linear Reciprocal property

Rotation invariant f(x,y) -> f(r,  )

Inverse Fourier transform using Forward Fourier transform forward inverse F(u) F * (u) Inverse transform (1/M)f * (x) xM f * (x) f(x)

Periodicity and zero padding What is the equivalent spatial-domain convolution to the frequency-domain multiplication? ?

correct conv. erroneous conv. 400

800 Implicit periodicity of DFT zero padding Correct convolution after zero-padding

Ideal low-pass filter with zero-padding Spatial convolution

Outline Review concept of frequencies How to measure the period(or frequency)? Adjust frequency by scaling Properties of DFT Convolution theorem – relation between spatial domain filtering(masking) and frequency domain operation

Connection between spatial and frequency filters Convolution theorem

Convolution - 1-D case convolution h(m) m m h(-m) 轉 m h(x-m) x m f(m) 乘 x m f(m)h(x-m) x 積

Convolution – 2-D case 2d convolution Mask operation

mask coefficients underlying neighborhood X (product) output

Convolution theorem f Fourier transform F h Fourier transform H Time domain Frequency domain convolution multiplication

Gaussian filters (low pass) mean=0 variance Frequency domain Time domain Low-pass

Difference of Gaussian (high pass)

Why frequency domain operation? It ’ s more intuitive to design frequency filters Computation complexity Frequency domain: Fourier transform + multiplication Spatial domain: convolution Size of mask

Outline Introduction to the Fourier Transform and Frequency Domain Filtering in the Frequency domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

Practical applications 1028x732

Filtering in the frequency domain

Smoothing frequency-domain filters Design issue G(u,v)=F(u,v) H(u,v) Remove high freq. component (details, noise, …) Ideal low-pass filter Butterworth filter Gaussian filter More smooth in the edge of cut-off frequency

Ideal low-pass filter Sharp cut-off frequency where D(u,v) is the distance to the center freq.

Ideal low-pass filter (cont.) Cut-off freq.

Ideal low-pass filter (con.t) ILPF can not be realized in electronic components, but can be implemented in a computer Decision of cut-off freq.? Measure the percentage of image power within the low freq. Total image power

ILPF: example originalFreq

original  =92 D 0 =5  =94.6 D 0 =15  =96.4 D 0 =30  =98 D 0 =80  =99.5 D 0 =230 Ideal low-pass filtering

Ringing effect

Effects of ideal low-pass filtering Blurring and ringing ILPF: Freq. F -1 blurring ringing ILPF: spatial

Effects of ideal low-pass filtering (cont.) spatial impulse ILPF spatial

Butterworth low-pass filters H=0.5 when D(u,v)=D 0

Order of butterworth filters n=1n=2n=5n=20

Butterworth filters Order = 2 original D 0 =5 D 0 =15D 0 =30 D 0 =80D 0 =230

Gaussian low-pass filters Variance or cut-off freq. D(u,v)=D 0 H = 0.607

Gaussian smoothing original D 0 =5 D 0 =15D 0 =30 D 0 =80D 0 =230

Practical applications: 1 444x508 GLPF, D 0 =80

Practical applications: 2 GLPF, D 0 =100 GLPF, D 0 = x732

Practical applications: 3 588x600 GLPF, D 0 =30 GLPF, D 0 =10 Scan line

Outline Introduction to the Fourier Transform and Frequency Domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

Sharpening frequency-domain filters Image details corresponds to high- frequency Sharpening: high-pass filters H hp (u,v)=1-H lp (u,v) Ideal high-pass filters Butterworth high-pass filters Gaussian high-pass filters Difference filters

Ideal HPF Butterworth HPF Gaussian HPF

Spatial-domain HPF ideal Butterworth Gaussian negative

Ideal high-pass filters D 0 =15D 0 =30D 0 =80 ringing original

Butterworth high-pass filters n=2,D 0 =15D 0 =30D 0 =80

Gaussian high-pass filters D 0 =15D 0 =30D 0 =80

Laplacian frequency-domain filters Spatial-domain Laplacian Fourier transform

Laplacian frequency-domain filters Input f(x,y) Laplacian F(u,v) F F -(u 2 +v 2 )F(u,v) ? -(u 2 +v 2 ) The Laplacian filter in the frequency domain is H(u,v) = -(u 2 +v 2 )

0 frequency spatial

original Laplacian Scaled Laplacian original+ Laplacian

High-boost filtering and High- frequency emphasis Ideal: tune the ratio of original image and the high-passed image High-boost filter High-frequency emphasis H hb (u,v)= (A-1) + H hp (u,v), A >= 1 Equivalent to (A-1)f(x,y) in the spatial-domain H hfe (u,v)= a + b H hp (u,v), a>=0, b>a

High-boost filtering original Laplacian A=2 A=2.7

High-frequency emphasis original Butterworth a=0.5 b=2.0 Histogram Equalization

Outline Introduction to the Fourier Transform and Frequency Domain Smoothing Frequency Domain Filters Sharpening Frequency Domain Filters Homomorphic Filtering Implementation

Homomorphic filtering Homomorphism: Image formation model f(x,y)=i(x,y) r(x,y) illumination: reflectance: Slow spatial variations vary abruptly, particularly at the junctions of dissimilar objects

Image Formation Model Illumination source scene reflection

Homomorphic filtering Product term Log of product z(x,y)=ln f(x,y)=ln i(x,y)+ ln r(x,y) Separation of signal source:

Homomorphic filtering approach ln i(x,y) ln r(x,y)

How to identify the illumination and reflection Illumination -> low frequency Reflection -> high frequency Radius from the origin sharpening

Homomophic filtering: example originalHomomorphic filtering