2. Introduction to MatLab: Image Processing
Most of the programming operations have as input or output a matrix or a vector.
Images are often represented a matrices.
- MatLab is a powerful tool to manipulate graphics and images.

3. Robot Vision Development in Practice
Build your robot (you are done)
Install a camera on it. Learn software that comes with camera, understand formats. JPG
Install Matlab on your laptop – student version, much software for free on WWW – Octave
Be able to display image from camera in one window so that you will see it and Matlab code or processing in other windows.
Guide your robot through stage or labyrinth and see with your own eyes what robot sees.
Think what is a good processing method for the image that you see.
Protytype this in Matlab using existing functions.
If necessary, use parallel Matlab on CUDA
If necessary, rewrite to C++ or Python or Java.

4. This introduction will give
MATLAB
This introduction will give
a brief overview, it’s not a MATLAB tutorial !
Some basic ideas
Main advantages and drawbacks compared to other languages

5. MATLAB high-performance language for technical computing
computation, visualization, and programming in an easy-to-use environment
Typical uses include:
Math and computation
Algorithm development
Modelling, simulation, and prototyping
Data analysis, exploration, and visualization
Scientific and engineering graphics
Application development, including Graphical User Interface building

6. Why MATLAB A good choice for vision program development because:
Easy to do very rapid prototyping
Quick to learn, and good documentation
A good library of image processing functions
Excellent display capabilities
Widely used for teaching and research in universities and industry
Another language to impress your boss with !

7. Why not MATLAB Has some drawbacks: • Slow for some kinds of processes
• Not geared to the web
• Not designed for large-scale system development

8. MATLAB Components MATLAB consists of: The MATLAB language
a high-level matrix/array language with control flow statements, functions, data structures, input/output, and object-oriented programming features.
The MATLAB working environment
the set of tools and facilities that you work with as the MATLAB user or programmer, including tools for developing, managing, debugging, and profiling
Handle Graphics
the MATLAB graphics system. It includes high-level commands for two-dimensional and three-dimensional data visualization, image processing, animation, and presentation graphics.
…(cont’d)

9. MATLAB Components … The MATLAB function library.
a vast collection of computational algorithms ranging from elementary functions like sum, sine, cosine, and complex arithmetic, to more sophisticated functions like matrix inverse, matrix eigenvalues, Bessel functions, and fast Fourier transforms as well as special image processing related functions
The MATLAB Application Program Interface (API)
a library that allows you to write C and Fortran programs that interact with MATLAB. It include facilities for calling routines from MATLAB (dynamic linking), calling MATLAB as a computational engine, and for reading and writing MAT-files.

10. MATLAB Some facts for a first impression
Everything in MATLAB is a matrix !
MATLAB is an interpreted language, no compilation needed (but possible)
MATLAB does not need any variable declarations, no dimension statements, has no packaging, no storage allocation, no pointers
Programs can be run step by step, with full access to all variables, functions etc.

11. What does Matlab code look like?
A simple example:
a = 1
while length(a) < 10
a = [0 a] + [a 0]
end
which prints out Pascal’s triangle: 1
1 1
1 2 1
(with “a=” before each line).

12. What does Matlab code look like?
Another simple example:
t = 0:pi/100:2*pi;
y = sin(t);
plot(t,y)

13. What does Matlab code look like?
Another simple example:
t = 0:pi/100:2*pi;
y = sin(t);
plot(t,y)
Remember:
EVERYTHING IN MATLAB IS A MATRIX !
creates 1 x 200 Matrix
Argument and result: 1 x 200 Matrix

14. jpg
bmp

15. Formats of Images in MATLAB
MATLAB can import/export several image formats
BMP (Microsoft Windows Bitmap)
GIF (Graphics Interchange Files)
HDF (Hierarchical Data Format)
JPEG (Joint Photographic Experts Group)
PCX (Paintbrush)
PNG (Portable Network Graphics)
TIFF (Tagged Image File Format)
XWD (X Window Dump)
MATLAB can also load raw-data or other types of image data
Data types in MATLAB
Double (64-bit double-precision floating point)
Single (32-bit single-precision floating point)
Int32 (32-bit signed integer)
Int16 (16-bit signed integer)
Int8 (8-bit signed integer)
Uint32 (32-bit unsigned integer)
Uint16 (16-bit unsigned integer)
Uint8 (8-bit unsigned integer)

16. Images in MATLAB
• Binary images : {0,1} • Intensity images : [0,1] or uint8, double etc. • RGB images : m-by-n-by-3 • Indexed images : m-by-3 color map • Multidimensional images m-by-n-by-p (p is the number of layers)

17. Image import and export
Read and write images in Matlab
>> I=imread('cells.jpg');
>> imshow(I)
>> size(I)
ans = (RGB image)
>> Igrey=rgb2gray(I);
>> imshow(Igrey)
>> imwrite(lgrey, 'cell_gray.tif', 'tiff')
Alternatives to imshow
>>imagesc(I)
>>imtool(I)
>>image(I)

18. Images and Matrices How to build a matrix (or image)?
>> B = zeros(3,3)
B =
>> C = ones(3,3)
C =
>>imshow(A) (imshow(A,[]) to get automatic pixel range)

19. Matrices

20. Rows and columns are always numbered starting at 1
Matrices
Rows and columns are always numbered starting at 1
Matlab matrices are of various types to hold different kinds of data (usually floats or integers)
A single number is really a 1 x 1 matrix in Matlab!
Matlab variables are not given a type, and do not need to be declared
Any matrix can be assigned to any variable

21. Building matrices with [ ]: A = [2 7 4] A = [2; 7; 4]
B = [ A A ] 2 7 4 2 7 4 2 7 4 3 8 9 ?

22. Building matrices with [ ]: A = [2 7 4] A = [2; 7; 4]
B = [ A A ] 2 7 4 2 7 4 2 7 4 3 8 9 2 7 4 2 7 4 3 8 9 3 8 9

23. Matrices

24. Some operators must be handled with care: A = [1 2 ; 4 5]
Matrices
Some operators must be handled with care:
A = [1 2 ; 4 5]
B = A * A prints
B = A .* A prints
Element by element multiplication

25. Submatrices
A matrix can be indexed using another matrix, to produce a subset of its elements:
a = [ ] b = [3 5 6]
c = a(b):

26. This works in 2-D as well, e.g. c(2:3, 1:2) produces a
Submatrices
To get a subsection of a matrix, we can produce the index matrix with the colon operator:
a(2:5)
prints
ans =
This works in 2-D as well, e.g. c(2:3, 1:2) produces a
2 x 2 submatrix.
The rows and columns of the submatrix are renumbered.

27. ‘for’ loops in MATLAB iterate over matrix elements: b = 0
for i = [ ]
b = b + i;
end
Result: 29
Note:
The MATLAB way to write that program would have been:
b = sum([ ]);
Avoid loops if possible !

28. The typical ‘for’ loop looks like: for i = 1:6 … end
loops
The typical ‘for’ loop looks like:
for i = 1:6 …
end
Which is the same as:
for i = [ ]

29. loops
Once again:
AVOID LOOPS

30. So why MATLAB and IMAGE PROCESSING ?
Images
So why MATLAB and IMAGE PROCESSING ?

31. Images can be treated as matrices !

32. Matrix >> A=[1 2 3;3 2 1] >>b=A(1,:) A = >> B=A' b =
>>b=A(1,:)
b =
>> B=A'
B =

33. Images and Matrices X Y A = 2 3 5 6 7 8 9
Accesing image elements (row, column)
>> A(2,1)
ans = 4
: can be used to extract a whole column or row
>> A(:,2)
ans = 2 5 8
or a part of a column or row
>> A(1:2,2)
A =

34. Image Arithmetic
Arithmetic operations such as addition, subtraction, multiplication and division can be applied to images in MATLAB
+, -, *, / performs matrix operations
>> A+A
ans =
>> A*A
ans =
To perform an elementwise operation use . (.*, ./, .*, .^ etc)
>> A.*A
ans =
A =

35. Logical Conditions
equal (==) , less than and greater than (< and >), not equal (~=) and not (~)
find(‘condition’) - Returns indexes of A’s elements that satisfies the condition.
>> [row col]=find(A==7)
row = 3
col = 1
>> [row col]=find(A>7) 3
col = 2
>> Indx=find(A<5)
Indx = 1 2 4 7
A =

36. Flow Control Flow control in MATLAB - if, else and elseif statements
(row=1,2,3 col=1,2,3)
if row==col
A(row, col)=1;
elseif abs(row-col)==1
A(row, col)=2;
else
A(row, col)=0;
end
A =

37. Flow Control Flow control in MATLAB - for loops for row=1:3
for col=1:3
if row==col
A(row, col)=1;
elseif abs(row-col)==1
A(row, col)=2;
else
A(row, col)=0;
end
A =

38. Flow Control while, expression, statements, end A = 2 3 Indx=1; 5 6
while A(Indx)<6
A(Indx)=0;
Indx=Indx+1;
end
A =
A =

39. Working with M-Files
M-files can be scripts that simply execute a series of MATLAB statements, or they can be functions that also accept input arguments and produce output.
MATLAB functions:
Are useful for extending the MATLAB language for your application.
Can accept input arguments and return output arguments.
Store variables in a workspace internal to the function.

40. Working with M-Files Create a new empty m-file function B=test(I)
[row col]=size(I)
for r=1:row
for c=1:col
if r==c
A(r, c)=1;
elseif abs(r-c)==1
A(r, c)=2;
else
A(r, c)=0;
end
B=A;

42. Examples
Try these MATRIX AND VECTOR OPERATIONS
This is how we can define a vector
>> v=[1, 2, 3]
Matlab prints out the following
v =    
1     2     3
Similarly we can define a matrix
>> M= [ 1 2 3; 4 5 6; 7 8 9]
The result is:
M =    
1     2     3    
4     5     6    
7     8     9
If you want to suppress the MatLab output then you need to finish the line with semicolon as follows.
>>M= [ 1 2 3; 4 5 6; 7 8 9];

43. Projection
Say you want to extract some rows and columns of a matrix. This is called a projection. We simply give the subset of rows and columns as parameters, as follows
>> M11=M(2:3 , 2:3)
M11 =
     5     6
     8     9
To specify all elements in a given dimension one can use ':‘ So to get all rows but just columns 1 and 2, we type
>> A= M( :, 1:2)
A =
     1     2
     4     5
     7     8

44. WORKING WITH IMAGES in MatLab
Let’s talk about image files and their formats…..
Color vs GrayScale
Basic Image Processing functions:
Reading in an image:
>> img1=imread('Water lilies.jpg');
Displaying an image:
>> imshow(img1);
Finding out size of an image:
>> size(img1);
>> size(img1)
ans =
imread
imshow
size

45. variable imgsmall Cropping an image: 4. Display resulting image
WORKING WITH IMAGES in MatLab
Cropping an image:
>> imgsmall=img1(200:300,300:400,1:3);
>> imshow(imgsmall)
>> imgsmall=img1(150:250,350:450,1:3);
>> size(imgsmall)
ans =
Exercise: 1. Find 2 images online
2. Crop them to the same size
3. Add the two images together.
4. Display resulting image
Advanced Exercise:
Rescale images to same size then add them
See next slide to see HOWTO rescale
variable imgsmall

46. ReScaling linspace round
We can rescale by changing the number of rows and columns, yet preserve the information in the image
>> [rows, cols, colors]= size(img1)
rows = cols = 800 colors = 3
% Increase the number of rows
>> stretchfactor = 1.5
>> rowVec= linspace(1,rows,stretchfactor*rows);
>> newrows=round(rowVec);
>> newimag=img1(newrows,:,:)
>> imshow(newimg);
% Decrease number of columns
>> stretchfactor = 0.75;
>> colVec= linspace(1,cols,stretchfactor*cols);
>> newcols=round(colVec);
>> newimag=newimg(:,newcols,:)
>>imshow(newimg)
linspace
round

47. Example Program: Inverting an image
To invert or to add two images we need to convert to double and then rescale the result back so that it looks like an image
InvImg= 1 - double(IMG1)/255;
NewImg = uint8(round(InvImg*255)))
Imshow(NewImg);
uint8

48. input image(v) row= input(‘which row?’); red = v(row,:,1);
WORKING WITH IMAGES in MatLab
Color Masking
Sometimes we want to replace pixels of an image of one or more colors with pixels from another image. It is useful to use a “blue or green screen” in some instances.
Find an image with a big plot of one color. First we will replace that color. And then we will find another image for pixel replacement.
Let us plot the color values of one chosen row…This will tell us the pixel values of the color we want to replace.
v = imread(‘myimg.jpg’)
image(v)
row= input(‘which row?’);
red = v(row,:,1);
green = v(row,:,2);
blue = v(row,:,3);
plot(red,’r’);
hold on
plot(green,’g’);
plot(blue,’b’);
input

49. v= imread(‘myimg.jpg’); thresh= 160
WORKING WITH IMAGES in MatLab
Suppose we want to replace those values whose intensities exceed a threshold value of 160 in each color.
v= imread(‘myimg.jpg’);
thresh= 160
layer = (v(:,:,1) > thresh) & (v(:,:,2) > thresh) (v(:,:,2) > thresh)
mask(:,:,1) = layer;
mask(:,:,2) = layer;
mask(:,:,3) = layer;
If you want to only mask a portion of the image you can use something like…
>> mask(700:end,:,:)= false;
Which sets the mask so that we do not affect rows 700 and above
To reset the color to red
>>newv = v;
>>newv(mask)(1) = 255
Or to replace pixels from a different image w use… >> newv(mask) = w(mask);
Let us try to do this with a blue screen…..

50. Histograms

51. Images in MATLAB
imread
imshow

52. imhist

53. Histogram Equalization
histeq

55. Noise

56. Adding Noise
Filtering Noise
imnoise
medfilt2

57. Add Image to noise

58. Median Filtering removes noise

59. Median Filtering removes Gaussian noise
imnoise
fspecial
imfliter

60. FSPECIAL creates special 2D filters

61. Imfilter – multidimensional filtering

62. Thresholding

64. IM2BW – converting to binary by thresholding

65. Repeated thresholding

66. Separating background by thresholding

67. Extracting features:
Corner
Center point

68. Edge Detection

69. Example of kernel-based filtering - Sobel
edge
Example of kernel-based filtering - Sobel

70. Edge, Sobel and Prewitt

71. Roberts, Laplacian, Zero-cross, Canny edge detection methods

72. Use of threshold in Sobel affects what is found

73. “Laplacian of Gaussian” filtering is often better

74. Canny Filter is even better – this is a major invention with many applications in robotics.
Can one invent a better one?

75. Many variants of combining thresholding and edge detection exist

76. Hough Transform

77. Hough Transform has links to Fourier, Radon and Neural Nets.
A deep concept

78. Camera Calibration

79. Calibration of Cameras Matrix/vector multiplication

80. Sequences of operations

81. Noise, filtering and histogramming

82. Matlab versus other languages

83. Help
Help command
Help svd

84. for i = 1:2:N for J = 1:N A(I,J) =(I+J-1); end C++ vs. Matlab int j; .
for (j=1;j<23;j=j+2) {
A[4][j]=3*j; }
for i = 1:2:N
for J = 1:N
A(I,J) =(I+J-1);
end

85. int j; while (j<28) { …….; } while N> 0, E = E + F; F = A*F/N;
C++ vs. Matlab (cont.)
int j;
while (j<28) {
…….; }
while N> 0,
E = E + F;
F = A*F/N;
N = N + 1;
end

86. If (i==j) { A[i][j]=2; } else if (abs(i-j)!=1) { …….; if i == j
C++ vs. Matlab (cont.)
If (i==j)
{ A[i][j]=2; }
else if (abs(i-j)!=1) {
…….;
if i == j
A(i,j) = 2;
elseif abs(i-j) ~= 1
A(i,j) = -1;
else
A(i,j) = 0;
end

87. Index to an array can be zero. double, float , int…
C++ vs. Matlab (cont.)
Index to an array can be zero.
double, float , int…
“;” is very important
Index into matrix can’t be negative or zero.
Don’t need to worry about the data type
“;” not so important

88. function mean = avg(x,n) mean = sum(x)/n;
Functions
function [mean,stdev] = stat(x)
n = length(x);
mean = avg(x,n);
stdev =…
function mean = avg(x,n)
mean = sum(x)/n;
double* stat(double *x) {
…….;
return stdev; }

89. void Matrix2Vector( ) { ……; } function Matrix2Vector Av=A(1,:);
Functions(cont.)
void Matrix2Vector( ) {
……; }
function Matrix2Vector
Av=A(1,:);
for i=2:x
Av=[Av A(i,:)];
end
Av=Av';

90. void AddF(int i); int main() { …… Functions(cont.) addF(i); }
File name testFunR.m
function testFun
i=2;
AddF(i); i
function AddF(i)
i=i+1;
A function declaration cannot appear within a script M-file

91. To make a graph of y = sin(t) on the interval t = 0 to t = 10
Graphing
To make a graph of y = sin(t) on the interval t = 0 to t = 10
In file PlotTest.m
t = 0:.3:10;
y = sin(t);
plot(t,y,’r’) ;

92. graphing the fuction z(x,y) = x exp( - x^2 - y^2)
Graphing(cont.)
graphing the fuction z(x,y) = x exp( - x^2 - y^2)
In file MeshTest.m
[x,y] = meshgrid (-2:.2:2, -2:.2:2);
z = x .* exp(-x.^2 - y.^2);
mesh(z)

93. Multiple windows in one figure
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m
t = 0:.3:10;
y = sin(t);
z= cos(t);
subplot(2,1,1);
plot(t,y) ;
subplot(2,1,2);
plot(t,z);

94. Multiple windows in one figure
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m
t = 0:.3:10;
y = sin(t);
z= cos(t);
subplot(1,2,1);
plot(t,y) ;
subplot(1,2,2);
plot(t,z);

95. Singular value decomposition---[U,S,V]=svd(A)
Matrix Manipulation
Singular value decomposition---[U,S,V]=svd(A)
Eigenvalues and eigenvectors---[V,D] = eig(A)
Orthogonal-triangular decomposition- [Q,R]=qr(A)
LU factorization [L,U] = lu(A)
Matrix rank a=rank(A)
Condition number a=cond(A)

96. Image Processing Toolbox
Read an image I=imread(filename)
Display an image ---- imshow(I)
Write an image imwrite(I, filename,FMT)

97. Image Processing Toolbox(cont.)
ImageRWD.m
function ImageTest
Itif=imread('image1.tif');
imwrite(Itif,'image1.bmp','bmp');
Ibmp=imread('image1.bmp');
subplot(1,2,1);
imshow(Itif);
subplot(1,2,2);
imshow(Ibmp);

98. Image Processing Toolbox(cont.)
EdgeTest.m
function EdgeTest
Itif=imread('image1.tif');
B=edge(Itif,'canny');
subplot (1,2,1);
imshow(Itif);
subplot(1,2,2);
imshow(B);

99. Image Processing Toolbox(cont.)
Pixel values and statistics --corr2,imhist…
Image enhancement – histeq, medfilt2…
Linear filter conv2, filter2…
Image transform dct2, fft…
Binary image Op dilate, erode…

100. Neural Network Feed-forward backpropagatoin Hopfield Networks
Self-organizing maps
Radial Basis networks
………………

101. Neural Networks(cont.)
Create feed-forward NN
Net=newff([-1 2;0 5],[31],{‘tansig’,’purelin’},’traingd’);
Neural Model – tansig, logsig, purelin.
Training method –traingd, traingdm.
Training
[net,tr]=train(net,p,t)
net.trainParam.????; “show”, “lr”, “epochs”, “goal”
Simulation
A=sim(net,q);

102. Menu function MenuDemo cell_list = {}; fig_number = 1;
title_figure = 'Menu Demo';
cell_list{1,1} = {'Plot','PlotTest;'};
cell_list{1,2} = {'Mesh','MeshTest;'};
cell_list{1,3} = {'Image','ImageRWD;'};
cell_list{1,4} = {'Image Edge','EdgeTest;'};
cell_list{2,1} = {'????','PlotTest;'};
cell_list{2,2} = {'????','PlotTest;'};
cell_list{2,3} = {'????','PlotTest;'};
cell_list{2,4} = {'Exit',['disp(''Bye. To run again, type "menu".'');
close(' num2str(fig_number) ');']};
show_window(cell_list,fig_number,title_figure,120,20,3);

103. cell_list{1,1} = {‘name',‘function;'};
Menu (cont.)
cell_list{1,1} = {‘name',‘function;'};
show_window(cell_list,fig_number,title_figure,x,y,z);

104. EDGE DETECTION IN MATLAB

105. Fourier Transform in MATLAB

106. By the way…
MATLAB can also handle
Movies
3D objects …

107. MATLAB is a mighty tool to manipulate matrices
Conclusion
MATLAB is a mighty tool to manipulate matrices
Images can be treated as matrices
MATLAB is a mighty tool to manipulate images

108. MATLAB should be used to code software prototypes
In my opinion…
MATLAB should be used to code software prototypes
Research is mostly about prototypes, not runtime-optimized software
MATLAB should be used in research

109. Algorithm development time is drastically shorter in MATLAB
In my opinion…
MATLAB prototypes must be re-coded (e.g. in C++) if there’s need for speed
Algorithm development time is drastically shorter in MATLAB

110. end

111. Introduction to MATLAB and image processing
Amin Allalou
Centre for Image Analysis
Uppsala University
Computer Assisted Image Analysis
April

112. Prepared by Fred Annexstein
University of Cincinnati
Some Rights Reserved

113. Introduction to MATLAB
CIS 350 – 1
Introduction to MATLAB
Dr. Rolf Lakaemper