1. Introduction ENGI 4559 Signal Processing for Software Engineers
Dr. Richard Khoury
Fall 2009

2. Introduction: Overview
ENGI 4559 © Dr. Richard Khoury, 2009
Introduction: Overview
Signals
Examples of signals
Properties of signals
Analog and digital signals
Digital Images
Digital Image Processing
Matlab

3. A Simple Question What is a signal?
ENGI 4559 © Dr. Richard Khoury, 2009
A Simple Question
What is a signal?

4. ENGI 4559 © Dr. Richard Khoury, 2009
Examples of Signals
These mathematical functions are signals that vary with the independent variable x

5. Examples of Signals Sound is a signal that varies over time
ENGI 4559 © Dr. Richard Khoury, 2009
Examples of Signals
Sound is a signal that varies over time
s4(t) = “Windows makes it easy for several users to share a computer”
s5(t) = The Microsoft “tada” sound

6. ENGI 4559 © Dr. Richard Khoury, 2009
Examples of Signals
Signals can vary with several independent variables

7. ENGI 4559 © Dr. Richard Khoury, 2009
Examples of Signals
Altitude (s7) is a signal that varies over longitude and latitude

8. ENGI 4559 © Dr. Richard Khoury, 2009
Properties of Signals
Some signals can be represented by mathematical formulae
s1, s2, s3, and s6 are functions of independent variables that follow known formulae
Deterministic signals
Sometimes the mathematical function is too complex, unknown, or does not exist
s4, s5 and s7 are functions of independent variables, but there is no exact mathematical formulae modelling them
Random signals

9. ENGI 4559 © Dr. Richard Khoury, 2009
Properties of Signals
Signals can vary over any number of independent variables
s1 to s5 varied over a single variable, s6 and s7 varied over two variables
Signals can vary over any independent variable(s)
The x of s1, s2, s3 and the (x,y) of s6 could be anything
Most commonly, they vary over time, as was the case for the sound signals s4 and s5
Signals can also vary over spatial position, such as s7

10. Properties of Signals Signals can represent any information
ENGI 4559 © Dr. Richard Khoury, 2009
Properties of Signals
Signals can represent any information
s4 and s5 represent sound
s7 represents geographical altitude
s1, s2, s3, and s6 represent the value of a function – we don’t know if it has any physical meaning

11. Properties of Signals All the signals we’ve seen previously are analog
ENGI 4559 © Dr. Richard Khoury, 2009
Properties of Signals
All the signals we’ve seen previously are analog
They are function of continuous variables: x, y, time, physical position
Can make processing difficult
Alternative is a digital signal
Function of discrete variables
A variable that takes its value from a finite or countably infinite set

12. Analog and Digital Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Analog and Digital Signals

13. Analog and Digital Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Analog and Digital Signals
Two ways of seeing digital signals
Special case of a continuous signal
Sample of a continuous signal
where Ts is the sampling period and n is an integer
In this example, Ts = 0.1 and n goes from 0 to 10

14. Analog to Digital Conversion
ENGI 4559 © Dr. Richard Khoury, 2009
Analog to Digital Conversion
Analog-to-digital converter
Analog signal
Digital signal
Input analog signal
Analog-to-digital converter
Input digital signal
Digital signal processor
Output digital signal
Digital-to-analog converter
Output analog signal
Live show
Digital video camera
Digital copy
Edit, upload, download
Modified copy
Computer screen and speakers
Play-back

15. Analog to Digital Conversion
ENGI 4559 © Dr. Richard Khoury, 2009
Analog to Digital Conversion
What to do in-between samples?
We’ve already seen one option:
Digital function doesn’t look much like analog one
We can do better

16. Analog to Digital Conversion
ENGI 4559 © Dr. Richard Khoury, 2009
Analog to Digital Conversion
Zero-Order Hold
Simplest option
Hold the current value for one sample period Ts
Good enough approximation for many applications
Better ones exist
Interpolate based on past few samples

17. Digital to Analog Conversion
ENGI 4559 © Dr. Richard Khoury, 2009
Digital to Analog Conversion
The opposite of sampling: reconstructing
Multiple options
Interpolate between (subset of) samples
Example: straight line between pairs of samples
Curve-fitting if the original signal followed a mathematical function

18. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
Was easy to do with s2
What about s3?
What went wrong?
Reconstructed s3

19. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
We did not take enough samples
Without sufficient samples, we could not accurately reconstruct the signal
How to know how many samples we need?

20. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
Any periodic analog signal can be represented as a sum of sinusoids of different amplitudes A, frequencies Ω and phases θ
In our example, Ω = Hz
Speech signal, Ωmax = 3000 Hz
Audio signal, Ωmax = 20 kHz
TV signal, Ωmax = 5 MHz

21. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
Sampling Theorem
If the maximum frequency contained in an analog signal is Ωmax = B, then it can be perfectly reconstructed from samples taken at the sampling frequency Ωs = 2B.
Ωmax is called the Nyquist frequency
Ωs is called the Nyquist rate
Typically we sample at a rate a little above Ωs for safety

22. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
Ωmax = Hz
Ωs = Hz
Previous example was sampled at 10 Hz, too low

23. Sampling and Reconstruction
ENGI 4559 © Dr. Richard Khoury, 2009
Sampling and Reconstruction
Ωmax = Hz
Ωs = Hz
Sampling at 14 Hz
Bad image because of straight-line interpolation, but we have the right general shape

24. Digital Images as Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Digital Images as Signals
We’ve seen that a signal carries information as function of independent variables
A greyscale digital image is a digital signal
Information (intensity) at a given sample (pixel) is a function of two independent variables (x,y coordinates)

25. Digital Images as Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Digital Images as Signals
(x,y) are two independent discrete variables
For an XY image, (x,y) are integers between 0 and the maximum position (X-1, Y-1)
(0,0) is the top-left corner
Two independent sampling periods, Tsx and Tsy
Information in image is the intensity
Integer between 0 (black) and a maximum value (white)
Number of bits used to encode intensity: b
b-bit image with 2b possible shades of gray
0 to 2b-1
Examples:
b = 1  binary (black and white) image
b = 8  256 shades, standard 8-bit grayscale image

26. Digital Images as Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Digital Images as Signals
Images, like other signals, might or might not be represented by mathematical functions
I355,168 = 198

27. Digital Image Processing
ENGI 4559 © Dr. Richard Khoury, 2009
Digital Image Processing
Three main families of operations
Image restoration
Image enhancement
Image compression
These are signal-processing operations applied to images
We can do them on any signal with any dimensions

28. Image Restoration An original image im1 has become corrupted
ENGI 4559 © Dr. Richard Khoury, 2009
Image Restoration
An original image im1 has become corrupted
Undesirable differences (noise) has been added
Corrupted image is im2
We want to recover the original image
Restored image im3 has to be as similar to im1 as possible

29. ENGI 4559 © Dr. Richard Khoury, 2009
Image Restoration
We will see several ways of measuring restoration quality
A simple one: signal to noise ratio improvement
In decibels (db)
Assumes im1 is available for comparison
Requires us to compute the variance of the image
Recall: variance is the average difference with the average value of a sequence:

30. Image Restoration Signal to noise ratio:
ENGI 4559 © Dr. Richard Khoury, 2009
Image Restoration
Signal to noise ratio:
Consider the corrupted image im2
The signal is the original image im1
The noise is the difference between the corrupted image and the original: im2 – im1

31. Image Restoration For the restored image im3
ENGI 4559 © Dr. Richard Khoury, 2009
Image Restoration
For the restored image im3
The signal is the original image im1
The noise is the difference between the restored image and the original: im3 – im1
The restoration performance is the improvement in SNR
Positive if log of a ratio > 1, if less noise in im3 than in im2

32. Image Enhancement Accentuating a desired feature in the image
ENGI 4559 © Dr. Richard Khoury, 2009
Image Enhancement
Accentuating a desired feature in the image
Contrast improvement
Modify the brightness of the image
Edge detection
Find the exact limits of objects in the image
Image sharpening
Reduce blurring in the image
Image segmentation
Split the image in regions (segments) following some criteria (objects, textures, colours, etc.)

33. ENGI 4559 © Dr. Richard Khoury, 2009
Image Enhancement

34. Image Compression Images contain a lot of data 512512 grayscale image
ENGI 4559 © Dr. Richard Khoury, 2009
Image Compression
Images contain a lot of data
512512 grayscale image
8 bits/pixel  262,144 pixels
2,097,152 bits of data
2-hour movie in standard definition television (SDTV)
720  480 pixels per image
24 bits (3 bytes) per pixel
30 images per second
3B/pix  (720  480) pix/im  30 im/s  3600 s/h  2h  224 GB  48 single-layer 4.7GB DVD disks
That’s not counting sound!

35. ENGI 4559 © Dr. Richard Khoury, 2009
Image Compression
Impractical for fast transmission and efficient storage
Also unnecessary
Coding redundancy: some information can be recomputed
Spatial/temporal redundancy: some information is repeated
Irrelevance: Some information is invisible to human eyes
Compression: reducing the amount of data needed to represent the same amount of information

36. Image Compression Examples of compression
ENGI 4559 © Dr. Richard Khoury, 2009
Image Compression
Examples of compression
Lenna: 512512 grayscale image = 262,144 bytes of data
BMP: 263,222 bytes
TIFF: 262,598 bytes
JPEG: 162,762 bytes
PNG: 151,447 bytes
GIF: 38,572 bytes

37. Matlab We’ll use “fields.png” as an example
ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
We’ll use “fields.png” as an example
720x576 8-bit grayscale image

38. fields = imread('fields.png');
ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
Loading an image into a matrix
fields = imread('fields.png');
Matlab supports a number of image formats, including JPEG, GIF, PNG, BMP, and TIFF
Different number of bits for each, but 8-bit is standard for all

39. double(someVariable);
ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
Matlab saves the image in a variable in the workspace
fields <576x720 uint8>
Notice the uint8 format
Matlab’s default format is double
For some operations, you’ll need to convert
double(someVariable);
uint8(someVariable);
Notice also the dimensions
Our 720x576 image is stored as a 576x720 matrix
Coordinates are in (Y,X) order!
Recall also that a Matlab MxN matrix goes from (1,1) to (M,N), not (0,0) to (M-1,N-1)

40. title('Original Image')
ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
Displaying an image
colormap(gray)
imagesc(imagereal)
Colormap specifies the palette of colours to use
imagesc maps the matrix values to the full range of the colormap
Standard image functions apply
title('Original Image')

41. imwrite(fields, 'fields.tiff', 'tiff')
ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
Saving an image
imwrite(fields, 'fields.tiff', 'tiff')
Saved to your current work directory
Automatically overwrites an existing image with the same name
Note!
Needs the image to be in uint8 format
Does not map the values like imagesc does – so your saved image will look different from the displayed one

42. ENGI 4559 © Dr. Richard Khoury, 2009
Matlab
Working with an image is just like working with any other matrix
For example: adding 10% random noise pixels
noise = rand(size(field));
noise(find(noise > 0.9)) = 255;
noise(find(noise < 255)) = 0;
fieldnoise = ...
max(field, uint8(noise));
colormap(gray)
imagesc(fieldnoise)
title('Noise Image')

43. Summary Definition and Properties of Signals Sampling Signals
ENGI 4559 © Dr. Richard Khoury, 2009
Summary
Definition and Properties of Signals
Deterministic vs. random signals
Number of dimensions
Representation of information
Analog vs. digital signals
Sampling Signals
Sampling period
Zero-Order Hold
Reconstruction
Sampling Theorem

44. Summary Digital Image Processing Image restoration Image enhancement
ENGI 4559 © Dr. Richard Khoury, 2009
Summary
Digital Image Processing
Image restoration
Signal to noise ratio improvement
Image enhancement
Contrast improvement
Edge detection
Image sharpening
Image segmentation
Image compression
Coding redundancy
Spatial/temporal redundancy
Irrelevant information