Upload
anne-marsh
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
1
2
Introduction to MatLab: Image Processing
-Most of the programming operations have as input or output a matrix or a vector.
-Images are often represented a matrices.
- MatLab is a powerful tool to manipulate graphics and images.
Robot Vision Development in Practice
1. Build your robot (you are done)
2. Install a camera on it. Learn software that comes with camera, understand formats. JPG
3. Install Matlab on your laptop – student version, much software for free on WWW – Octave
4. Be able to display image from camera in one window so that you will see it and Matlab code or processing in other windows.
5. Guide your robot through stage or labyrinth and see with your own eyes what robot sees.
6. Think what is a good processing method for the image that you see.
7. Protytype this in Matlab using existing functions.
8. If necessary, use parallel Matlab on CUDA
9. If necessary, rewrite to C++ or Python or Java.3
MATLAB
This introduction will give
• a brief overview, it’s not a MATLAB tutorial !
• Some basic ideas
• Main advantages and drawbacks compared to other languages
MATLAB
• high-performance language for technical computing• computation, visualization, and programming in an easy-
to-use environment
Typical uses include:
• Math and computation• Algorithm development• Modelling, simulation, and prototyping• Data analysis, exploration, and visualization• Scientific and engineering graphics• Application development, including Graphical User
Interface building
Why MATLAB
A good choice for vision program development because:
• Easy to do very rapid prototyping• Quick to learn, and good documentation• A good library of image processing functions• Excellent display capabilities• Widely used for teaching and research in
universities and industry• Another language to impress your boss with !
Why not MATLAB
Has some drawbacks:
• Slow for some kinds of processes• Not geared to the web• Not designed for large-scale system
development
MATLAB Components
MATLAB consists of:
• The MATLAB language• a high-level matrix/array language with control flow statements, functions,
data structures, input/output, and object-oriented programming features.
• The MATLAB working environment• the set of tools and facilities that you work with as the MATLAB user or
programmer, including tools for developing, managing, debugging, and profiling
• Handle Graphics• the MATLAB graphics system. It includes high-level commands for two-
dimensional and three-dimensional data visualization, image processing, animation, and presentation graphics.
• …(cont’d)
MATLAB Components…
• The MATLAB function library. • a vast collection of computational algorithms ranging from elementary
functions like sum, sine, cosine, and complex arithmetic, to more sophisticated functions like matrix inverse, matrix eigenvalues, Bessel functions, and fast Fourier transforms as well as special image processing related functions
• The MATLAB Application Program Interface (API)• a library that allows you to write C and Fortran programs that interact with
MATLAB. It include facilities for calling routines from MATLAB (dynamic linking), calling MATLAB as a computational engine, and for reading and writing MAT-files.
MATLAB
Some facts for a first impression
• Everything in MATLAB is a matrix !
• MATLAB is an interpreted language, no compilation needed (but possible)
• MATLAB does not need any variable declarations, no dimension statements, has no packaging, no storage allocation, no pointers
• Programs can be run step by step, with full access to all variables, functions etc.
What does Matlab code look like?A simple example:
a = 1while length(a) < 10a = [0 a] + [a 0]end which prints out Pascal’s triangle:
11 11 2 11 3 3 11 4 6 4 11 5 10 10 5 11 6 15 20 15 6 11 7 21 35 35 21 7 11 8 28 56 70 56 28 8 11 9 36 84 126 126 84 36 9 1
(with “a=” before each line).
What does Matlab code look like?What does Matlab code look like?
Another simple example:
t = 0:pi/100:2*pi;y = sin(t);plot(t,y)
What does Matlab code look like?What does Matlab code look like?Another simple example:
t = 0:pi/100:2*pi;y = sin(t);plot(t,y)
Remember:
EVERYTHING IN MATLAB IS A MATRIX !
creates 1 x 200 Matrix
Argument and result: 1 x 200 Matrix
14
jpgjpg
bmpbmp
Formats of Images in MATLABFormats of Images in MATLAB
• MATLAB can import/export several image formats
– BMP BMP (Microsoft Windows Bitmap)– GIFGIF (Graphics Interchange Files)– HDF (Hierarchical Data Format)– JPEGJPEG (Joint Photographic Experts
Group)– PCX (Paintbrush)– PNG (Portable Network Graphics)– TIFF (Tagged Image File Format)– XWD (X Window Dump)– MATLAB can also load raw-dataraw-data or
other types of image data
• Data types in MATLAB– Double (64-bit double-precision
floating point)– Single (32-bit single-precision
floating point)– Int32 (32-bit signed integer)– Int16 (16-bit signed integer)– Int8 (8-bit signed integer)– Uint32 (32-bit unsigned integer)– Uint16 (16-bit unsigned integer)– Uint8Uint8 (8-bit unsigned integer)
Images in MATLAB
• Binary images : {0,1}
• Intensity images : [0,1] or uint8, double etc.
• RGB images : m-by-n-by-3
• Indexed images : m-by-3 color map
• Multidimensional images m-by-n-by-p (p is the number of layers)
Image import and exportImage import and export• Read and write images in Matlab
>> I=imread('cells.jpg');
>> imshow(I)
>> size(I)
ans = 479 600 3 (RGB image)
>> Igrey=rgb2gray(I);
>> imshow(Igrey)
>> imwrite(lgrey, 'cell_gray.tif', 'tiff')
Alternatives to imshow>>imagesc(I)
>>imtool(I)
>>image(I)
Images and MatricesImages and Matrices• How to build a matrix (or image)?
>> A = [ 1 2 3; 4 5 6; 7 8 9 ];
A = 1 2 3
4 5 6
7 8 9
>> B = zeros(3,3)
B = 0 0 0
0 0 0
0 0 0
>> C = ones(3,3)
C = 1 1 1
1 1 1
1 1 1
>>imshow(A) (imshow(A,[]) to get automatic pixel range)
MatricesMatrices
Matrices
•Rows and columns are always numbered starting at 1
•Matlab matrices are of various types to hold different kinds of data (usually floats or integers)
• A single number is really a 1 x 1 matrix in Matlab!
• Matlab variables are not given a type, and do not need to be declared
• Any matrix can be assigned to any variable
Matrices
Building matrices with [ ]:
A = [2 7 4]
A = [2; 7; 4]
A = [2 7 4; 3 8 9]
B = [ A A ]
2 7 4
2
7
4
2 7 4
3 8 9
?
Matrices
Building matrices with [ ]:
A = [2 7 4]
A = [2; 7; 4]
A = [2 7 4; 3 8 9]
B = [ A A ]
2 7 4
2
7
4
2 7 4
3 8 9
2 7 4
3 8 9
2 7 4
3 8 9
Matrices
Matrices
Some operators must be handled with care:
A = [1 2 ; 4 5]
B = A * A prints 9 12 24 33
B = A .* A prints 1 4 16 25
Element by element multiplication
Submatrices
A matrix can be indexed using another matrix, to produce a subset of its elements:
a = [100 200 300 400 500 600 700] b = [3 5 6]
c = a(b):
300 500 600
Submatrices
To get a subsection of a matrix, we can produce the index matrix with the colon operator:
a(2:5)printsans = 200 300 400 500
•This works in 2-D as well, e.g. c(2:3, 1:2) produces a 2 x 2 submatrix.
•The rows and columns of the submatrix are renumbered.
loops
‘for’ loops in MATLAB iterate over matrix elements:
b = 0for i = [ 3 9 17]
b = b + i;end
Result: 29
Note:The MATLAB way to write that program would have been:b = sum([ 3 9 17]);
Avoid loops if possible !
loops
The typical ‘for’ loop looks like:
for i = 1:6…
end
Which is the same as:
for i = [1 2 3 4 5 6]…
end
loops
Once again:
AVOID LOOPS
Images
So why MATLAB and IMAGE PROCESSING ?
Images
Images can be treated as matrices !
Matrix
>> A=[1 2 3;3 2 1]
A =
1 2 3
3 2 1
>>b=A(1,:)
b =
1 2 3
>> B=A'
B =
1 3
2 2
3 1
Images and Matrices
• Accesing image elements (row, column)>> A(2,1)ans = 4
• : can be used to extract a whole column or row>> A(:,2)ans =2
5 8• or a part of a column or row
>> A(1:2,2)ans =2
5
X
Y
A = 12 345 67 8 9
Image ArithmeticImage Arithmetic• Arithmetic operations such as addition, subtraction, multiplication and
division can be applied to images in MATLAB– +, -, *, / performs matrix operations
>> A+Aans = 2 4 6
8 10 12 14 16 18
>> A*Aans = 30 36 42
66 81 96 102 126 150
• To perform an elementwise operation use . (.*, ./, .*, .^ etc)>> A.*Aans = 1 4 9
16 25 36 49 64 81
A = 12 345 67 8 9
Logical ConditionsLogical Conditions• equal (==) , less than and greater than (< and >), not equal (~=) and not (~)
• find(‘condition’) - Returns indexes of A’s elements that satisfies the condition.
>> [row col]=find(A==7)
row = 3
col = 1
>> [row col]=find(A>7)
row = 3
3
col = 2
3
>> Indx=find(A<5)
Indx = 1
2
4
7
A = 12 345 67 8 9
Flow ControlFlow Control• Flow control in MATLAB
- if, else and elseif statements
(row=1,2,3 col=1,2,3)
if row==col
A(row, col)=1;
elseif abs(row-col)==1
A(row, col)=2;
else
A(row, col)=0;
endA =
1 2 0 2 1 2 0 2 1
Flow Control
• Flow control in MATLAB
- for loops
for row=1:3
for col=1:3
if row==col
A(row, col)=1;
elseif abs(row-col)==1
A(row, col)=2;
else
A(row, col)=0;
end
end
end
A =
1 2 0 2 1 2 0 2 1
Flow Control
• while, expression, statements, end
Indx=1;
while A(Indx)<6
A(Indx)=0;
Indx=Indx+1;
end
A = 12 345 67 8 9
A =
0 2 3 0 5 6 7 8 9
Working with M-FilesWorking with M-Files• M-files can be scripts that simply execute a series of MATLAB statements, or
they can be functions that also accept input arguments and produce output.
• MATLAB functions:– Are useful for extending the MATLAB language for your application.– Can accept input arguments and return output arguments.– Store variables in a workspace internal to the function.
Working with M-Files
• Create a new empty m-file
function B=test(I)[row col]=size(I)for r=1:row
for c=1:col if r==c
A(r, c)=1;elseif abs(r-c)==1
A(r, c)=2;else
A(r, c)=0;end
endend
B=A;
41
42
Examples
Try these MATRIX AND VECTOR OPERATIONS
This is how we can define a vector>> v=[1, 2, 3]
Matlab prints out the followingv =
1 2 3
Similarly we can define a matrix>> M= [ 1 2 3; 4 5 6; 7 8 9]The result is:M =
1 2 3 4 5 6 7 8 9
If you want to suppress the MatLab output then you need to finish the line with semicolon as follows.>>M= [ 1 2 3; 4 5 6; 7 8 9];
43
ProjectionProjectionSay you want to extract some rows and columns of a matrix. This is called a
projection. We simply give the subset of rows and columns as parameters, as follows
>> M11=M(2:3 , 2:3)
M11 = 5 6 8 9
To specify all elements in a given dimension one can use ':‘ So to get all rows but just columns 1 and 2, we type
>> A= M( :, 1:2)A = 1 2 4 5 7 8
44
Let’s talk about image files and their formats…..Color vs GrayScale
Basic Image Processing functions:
Reading in an image:>> img1=imread('Water lilies.jpg');
Displaying an image:
>> imshow(img1);
Finding out size of an image:>> size(img1);>> size(img1)
ans =
600 800 3
WORKING WITH IMAGES in MatLabWORKING WITH IMAGES in MatLab
imread
imshow
size
45
Cropping an image:
>> imgsmall=img1(200:300,300:400,1:3);>> imshow(imgsmall)>> imgsmall=img1(150:250,350:450,1:3);>> imshow(imgsmall)>> size(imgsmall)
ans =
101 101 3
Exercise: 1. Find 2 images online 2. Crop them to the same size 3. Add the two images together.
4. Display resulting image
Advanced Exercise:1. Rescale images to same size then add them2. See next slide to see HOWTO rescale
WORKING WITH IMAGES in MatLab
variable imgsmall
46
ReScalingReScalingWe can rescale by changing the number of rows and columns, yet preserve the
information in the image
>> [rows, cols, colors]= size(img1)
rows = 600 cols = 800 colors = 3
% Increase the number of rows>> stretchfactor = 1.5>> rowVec= linspace(1,rows,stretchfactor*rows);>> newrows=round(rowVec);>> newimag=img1(newrows,:,:)>> imshow(newimg);
% Decrease number of columns>> stretchfactor = 0.75;>> colVec= linspace(1,cols,stretchfactor*cols);>> newcols=round(colVec);>> newimag=newimg(:,newcols,:)>>imshow(newimg)
linspace
round
47
Example Program: Inverting an imageExample Program: Inverting an imageTo invert invert or to add two images we need to convert to double and then
rescale the result back so that it looks like an image
InvImg= 1 - double(IMG1)/255;
NewImg = uint8(round(InvImg*255)))
Imshow(NewImg);uint8
48
Color Masking
Sometimes we want to replace pixels of an image of one or more colors with pixels from another image. It is useful to use a “blue or green screen” in some instances.
Find an image with a big plot of one color. First we will replace that color. And then we will find another image for pixel replacement.
Let us plot the color values of one chosen row…This will tell us the pixel values of the color we want to replace.
1. v = imread(‘myimg.jpg’)2. image(v)3. row= input(‘which row?’);4. red = v(row,:,1);5. green = v(row,:,2);6. blue = v(row,:,3);7. plot(red,’r’);8. hold on9. plot(green,’g’);10. plot(blue,’b’);
WORKING WITH IMAGES in MatLab
input
49
Suppose we want to replace those values whose intensities exceed a threshold value of 160 in each color.
• v= imread(‘myimg.jpg’);• thresh= 160• layer = (v(:,:,1) > thresh) & (v(:,:,2) > thresh) (v(:,:,2) > thresh) • mask(:,:,1) = layer;• mask(:,:,2) = layer;• mask(:,:,3) = layer;
If you want to only mask a portion of the image you can use something like…>> mask(700:end,:,:)= false;Which sets the mask so that we do not affect rows 700 and above
To reset the color to red>>newv = v;>>newv(mask)(1) = 255
Or to replace pixels from a different image w use…>> newv(mask) = w(mask);
Let us try to do this with a blue screen…..
WORKING WITH IMAGES in MatLab
50
HistogramsHistograms
51
imread
imshow
Images in MATLABImages in MATLAB
52
imhist
53
histeq
HistogramHistogram EqualizationEqualization
54
55
NoiseNoise
56
imnoise
medfilt2
1.1.Adding NoiseAdding Noise2.2.Filtering NoiseFiltering Noise
57
Add Image to noiseAdd Image to noise
58
Median Filtering removes noiseMedian Filtering removes noise
59
imnoise fspecial
imfliter
Median Filtering removes Gaussian noiseMedian Filtering removes Gaussian noise
60
FSPECIAL creates FSPECIAL creates special 2D filtersspecial 2D filters
61
Imfilter – multidimensional filteringImfilter – multidimensional filtering
62
ThresholdingThresholding
63
64
IM2BW – converting to binary by IM2BW – converting to binary by thresholdingthresholding
65
Repeated thresholdingRepeated thresholding
66
Separating background by Separating background by thresholdingthresholding
67
Extracting features:Extracting features:1.1.CornerCorner2.2.Center pointCenter point
68
Edge Edge DetectionDetection
69
edge
Example of kernel-based Example of kernel-based filtering - Sobelfiltering - Sobel
70
Edge, Sobel and PrewittEdge, Sobel and Prewitt
71
Roberts, Laplacian, Zero-cross, Roberts, Laplacian, Zero-cross, Canny edge detection methodsCanny edge detection methods
72
Use of threshold in Use of threshold in Sobel affects what is Sobel affects what is
foundfound
73
““Laplacian of Gaussian” Laplacian of Gaussian” filtering is often betterfiltering is often better
74
1.1. Canny Filter is even better Canny Filter is even better – this is a major invention – this is a major invention with many applications in with many applications in robotics. robotics.
2.2. Can one invent a better Can one invent a better one?one?
75
Many variants of Many variants of combining combining
thresholding and thresholding and edge detection existedge detection exist
76
Hough Hough TransformTransform
77
1.1. Hough Transform has links to Fourier, Hough Transform has links to Fourier, Radon and Neural Nets. Radon and Neural Nets.
2.2. A deep conceptA deep concept
78
Camera Camera CalibrationCalibration
79
Matrix/vector multiplicationMatrix/vector multiplication
Calibration of CamerasCalibration of Cameras
80
Sequences of operationsSequences of operations
81
Noise, filtering and histogrammingNoise, filtering and histogramming
Matlab versus other languages
Help
Help command
Help svd
C++ vs. Matlab
int j;
.
for (j=1;j<23;j=j+2)
{
A[4][j]=3*j;
}
for i = 1:2:N
for J = 1:N
A(I,J) =(I+J-1);
end
end
C++ vs. Matlab (cont.)
int j;
while (j<28)
{
…….;
}
while N> 0,
E = E + F;
F = A*F/N;
N = N + 1;
end
C++ vs. Matlab (cont.)
If (i==j)
{ A[i][j]=2;
}
else if (abs(i-j)!=1)
{
…….;
}
if i == j
A(i,j) = 2;
elseif abs(i-j) ~= 1
A(i,j) = -1;
else
A(i,j) = 0;
end
C++ vs. Matlab (cont.)
• Index to an array can be zero.
• double, float , int…
• “;” is very important
• Index into matrix can’t be negative or zero.
• Don’t need to worry about the data type
• “;” not so important
Functions
double* stat(double *x)
{
…….;
return stdev;
}
function [mean,stdev] = stat(x)
n = length(x);
mean = avg(x,n);
stdev =…
function mean = avg(x,n)
mean = sum(x)/n;
Functions(cont.)
function Matrix2VectorAv=A(1,:);for i=2:x Av=[Av A(i,:)];endAv=Av';
void Matrix2Vector( )
{
……;
……;
}
Functions(cont.)
A function declaration cannot appear within a script M-file
void AddF(int i);
int main()
{ ……addF(i);
}
void AddF(int i)
{
i=i+1;
}
File name testFunR.m
function testFun
i=2;
AddF(i);
i
function AddF(i)
i=i+1;
Graphing
In file PlotTest.m
t = 0:.3:10;
y = sin(t);
plot(t,y,’r’) ;
To make a graph of y = sin(t) on the interval t = 0 to t = 10
Graphing(cont.)
graphing the fuction z(x,y) = x exp( - x^2 - y^2)
In file MeshTest.m
[x,y] = meshgrid (-2:.2:2, -2:.2:2);
z = x .* exp(-x.^2 - y.^2);
mesh(z)
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m
t = 0:.3:10;
y = sin(t);
z= cos(t);
subplot(2,1,1);
plot(t,y) ;
subplot(2,1,2);
plot(t,z);
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m
t = 0:.3:10;
y = sin(t);
z= cos(t);
subplot(1,2,1);
plot(t,y) ;
subplot(1,2,2);
plot(t,z);
Matrix Manipulation
•Singular value decomposition---[U,S,V]=svd(A)
•Eigenvalues and eigenvectors---[V,D] = eig(A)
• Orthogonal-triangular decomposition- [Q,R]=qr(A)
•LU factorization --[L,U] = lu(A)
•Matrix rank -- a=rank(A)
•Condition number -- a=cond(A)
Image Processing Toolbox
•Read an image ---- I=imread(filename)
•Display an image ---- imshow(I)
•Write an image ---- imwrite(I, filename,FMT)
Image Processing Toolbox(cont.)
ImageRWD.m
function ImageTest
Itif=imread('image1.tif');
imwrite(Itif,'image1.bmp','bmp');
Ibmp=imread('image1.bmp');
subplot(1,2,1);
imshow(Itif);
subplot(1,2,2);
imshow(Ibmp);
Image Processing Toolbox(cont.)
EdgeTest.m
function EdgeTest
Itif=imread('image1.tif');
B=edge(Itif,'canny');
subplot (1,2,1);
imshow(Itif);
subplot(1,2,2);
imshow(B);
Image Processing Toolbox(cont.)
•Pixel values and statistics --corr2,imhist…
•Image enhancement – histeq, medfilt2…
•Linear filter -- conv2, filter2…
•Image transform -- dct2, fft…
•Binary image Op. --- dilate, erode…
Neural NetworkNeural Network
•Feed-forward backpropagatoin
•Hopfield Networks
•Self-organizing maps
•Radial Basis networks
•………………
Neural Networks(cont.)•Create feed-forward NN
•Net=newff([-1 2;0 5],[31],{‘tansig’,’purelin’},’traingd’);
•Training •[net,tr]=train(net,p,t)
•net.trainParam.????; “show”, “lr”, “epochs”, “goal”
•Simulation
•A=sim(net,q);
•Neural Model – tansig, logsig, purelin.•Training method –traingd, traingdm.
MenuMenufunction MenuDemo
cell_list = {};
fig_number = 1;
title_figure = 'Menu Demo';
cell_list{1,1} = {'Plot','PlotTest;'};
cell_list{1,2} = {'Mesh','MeshTest;'};
cell_list{1,3} = {'Image','ImageRWD;'};
cell_list{1,4} = {'Image Edge','EdgeTest;'};
cell_list{2,1} = {'????','PlotTest;'};
cell_list{2,2} = {'????','PlotTest;'};
cell_list{2,3} = {'????','PlotTest;'};
cell_list{2,4} = {'Exit',['disp(''Bye. To run again, type "menu".'');
close(' num2str(fig_number) ');']};
show_window(cell_list,fig_number,title_figure,120,20,3);
Menu (cont.)
•cell_list{1,1} = {‘name',‘function;'};
•show_window(cell_list,fig_number,title_figure,x,y,z);
EDGE DETECTION IN MATLABEDGE DETECTION IN MATLAB
Fourier Transform in MATLABFourier Transform in MATLAB
By the way…
MATLAB can also handle
• Movies
• 3D objects
• …
Conclusion
MATLAB is a mighty tool to manipulate matrices
Images can be treated as matrices
MATLAB is a mighty tool to manipulate images
In my opinion…
MATLAB should be used to code software prototypes
Research is mostly about prototypes, not runtime-optimized software
MATLAB should be used in research
In my opinion…
•MATLAB prototypes must be re-coded (e.g. in C++) if there’s need for speed
•Algorithm development time is drastically shorter in MATLAB
110
end
Introduction to MATLAB and image processing
Amin Allalou
Centre for Image Analysis
Uppsala University
Computer Assisted Image Analysis
April 4 2008
112
Prepared by Fred AnnexsteinUniversity of CincinnatiSome Rights Reserved
CIS 350 – 1
Introduction to MATLAB
Dr. Rolf Lakaemper