Upload
others
View
23
Download
4
Embed Size (px)
Citation preview
Numerical Methods for Civil EngineersNumerical Methods for Civil Engineers
Welcome to
Lecture 1 – MATLAB 1
������ ����..����..���� � ��������� ��������S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
TextbookTextbookApplied Numerical MethodsApplied Numerical MethodsWith MATLAB for EngineersWith MATLAB for Engineersand Scientistsand Scientists
STEVEN C. CHAPRASTEVEN C. CHAPRA
McGraw-Hill International Edition
ReferencesNumerical Methods For Engineers with Personal Computer Applications,
(Third Edition) by Chapra, S.C. and R.P. Canale, McGraw-Hill, 1998
Numerical Methods with MATLAB : Implementations and Applications,
Gerald W. Recktenwald, Prentice-Hall, 2000
The Matlab 7 Handbook , Mathwork Inc.
KEEP THESE BOOKS! They are excellent career references
(at least for a while)
An Introduction to Numerical Methods : A MATLAB Approach,
Abdelwahab Kharab and Ronald B. Guenther, Chapman & Hall/CRC, 2002
Numerical Methods using MATLAB,
John H. Mathews and Kurtis D. Fink, Prentice-Hall, 2004
Topics Covered
• Introduction to Matlab
• Approximations and Errors
• Roots of Equations
• Linear Systems
• Curve Fitting
• Interpolation
• Numerical Integration
• Ordinary Differential Equations
• Optimization
Conduct of Course
Assignments 20 %
Midterm Exam 40 %
Final Exam 40 %
Grading Policy
100 - 90
Final Score Grade
A89 - 85 B+84 - 80 B79 - 75 C+74 - 70 C69 - 65 D+64 - 60 D59 - 0 F
���������ก�������
- ������������������ ก��! ���"����! #��" �$ %"�&
- "'��ก'���� ����!��������!(���)
- "'���#��&� ก��! � *+��!� �+, � I ���� F
MATLABThe Language of Technical Computing
The latest version Matlab 7.3 R2006b
www.mathworks.com
ATLAB �)-��)�.ก��������ก��/���0��
0 ,��",�1 ก��."��� .�+2�3������)�+" �$ 2�4
�4���"���"2��+.������������#� '�"/�#��&ก��/���0
����� ����"(�)5,�ก���.&&�)6� �/��#��)-�ก��'��
����+��� MATLAB ��ก��"/��������7� "������ก�� $8� #���
"�����������/���0,������7����,��ก��
M
MATLAB = MAT rix LAB oratory
- Math and computation
- Algorithm development
- Modeling, Simulation, and Prototyping
- Many toolboxes for solving problems:
Control System Toolbox
Signal Processing Toolbox
System Identification Toolbox
Neural Network Toolbox
Statistics Toolbox
Optimization Toolbox
Partial Diff. Equation Toolbox
Symbolic Math Toolbox
- Data analysis, exploration, and visualization
- Scientific and Engineering Graphics
http://www.math.utah.edu/lab/ms/matlab/matlab.html
http://www.mathworks.com/
http://www.math.mtu.edu/~msgocken/intro/intro.html
MATLAB Educational Sites
http://www.eece.maine.edu/mm/matweb.html
� The MATLAB System
� Basic Arithmetic
� Built-in Functions
� Built-in Variables
� Matrices & Magic Squares
MATLAB 1
The MATLAB System
The MATLAB system consists of five main parts:
Development Environment: set of tools and facilities that help you use MATLAB functions and files. Many of these tools are graphical user interfaces. It includes the MATLAB desktop and Command Window, a command history, an editor and dedugger, and browsers for viewing help, the workspace, files, and the search path.
The MALAB Mathematical Function Library: a vast collection of computational algorithms.
The MATLAB Language: This is a high-level matrix/array language with control flow statements, functions, data structures, input/output, and objected-oriented programming features.
Graphics: MATLAB has extensive facilities 2-D and 3-D data visualization, animation, and presentation graphics.
The MATLAB Application Program Interface (API): allows you to write C and Fortran programs that interact with MATLAB.
Getting Started
MATLAB Desktop
Getting Started
Command Window
Command prompt >> DEMO
Getting Started
Editor/Debugger
Basic ArithmeticBasic Arithmetic
Last-line editing
Calculator functions work as you'd expect:
>>(1+4)*3
ans =
15
+ and - are addition, / is division, * is multiplication, ^ is an exponent.
>> 5*5*5
>> 5^(-2.5)
>> 3*(23+14.7-4/6)/3.5
Up Arrow
>> 2 + 6 - 4
ans =
4
The value in ans can be recalled:
>> ans/2
ans = 2
Assign value to a variable:
>> a = 5
>> b = 6
>> c = b/a
>> a=2;
>> A=3;
>> 2*a
>> 2*A
Upper & Lower Case
Several commands in the same line:
>>x=2;y=6+x,x=y+7
… For too long command:
>> Num_Apples = 10;
>> Num_Orange = 25;
>> Num_Pears = 12;
>> Num_Fruit=Num_Apples+Num_Orange...+ Num_Pears
sin(x), cos(x), tan(x),
sqrt(x), log(x), log10(x),
asin(x), acos(x), atan(x)
Built-in Functions
>> sin(pi/4)
ans =
0.7071
>> pi
ans =
3.1416
The output of each command can be suppress by using semicolon ;
>> x = 5;
>> y = sqrt(59);
>> z = log(y) + x^0.25
z =
3.5341
The commas allow more than one command on a line:
>> a = 5; b = sin(a), c = sinh(a)
b =
-0.9589
c =
74.2099
MATLAB Variables is created whenever it appears on the left-hand of “ = “
>> t = 5;
>> t = t + 2
t =
7
Format
>> pi
>> format long
>> pi
>> format short
>> format bank
>> format short e
>> format long e
>> format compact
>> format loose
>> format
Any variable appearing on the right-hand side of“ = “ must already be defined.
>> x = 2*z
??? Undefined function or variable ‘z’
Use long variable names is better to remember
and understandable for others
>> radius = 5.2;
>> area = pi*radius^2;
>> who
>> whos
>> clear
>> help log
>> lookfor cosineOn-line Help:
Built-in VariablesUse by MATLAB, Should not be assigned to other values
Variable Meaning
ans value of an expression when not assigned to variable
eps floating-point precision
i, j unit imaginary numbers, i = j =
pi π = 3.14159265 . . .
realmax largest positive floating-point number
realmin smallest positive floating-point number
Inf ∞, a number larger than realmax , result of 1/0
NaN not a number (0/0)
1−
>> x = 0; >> 5/x>> x/x
Matrices and Magic SquaresMatrices and Magic Squares
MATLAB allows you to work with entire matrices quickly and easily.
In MATLAB, a matrix is a rectangular array of numbers.
scalars = 1-by-1 matrices
vectors = one row or column matrices
Renaissance engraving Melencolia I by the German artist and amateurmathematician Albrecht Dürer.
The matrices operations in MATLAB are designed to be as natural as possible.
it is usually best to think of everything as a matrix.
"�����,� ก�1
��������ก��� �������ก�������ก� � ���������� �������:• ����ก��� �!� "ก�#�������$ �� ���� %�& � ��� ,
• ��$�'����� � ; ����$�� ������� �
• �(�%���$�����%���$������)#�*ก�%����� [ ]
ก���$������ก'+�� MATLAB �,-�$%����.�:• �� �������ก��$-��������
• �,��$�����ก'+!ก-/�+�$ �"�0�� ก
• ��$������ก'+�����$/1�ก+����0���
• ��$������ก'+�����$ก����������ก���� M-file
� �2��2+�$ ����$������%�$����,���� Command Window
>> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]
MATLAB !����������ก'+������� �����$-�:
A =
16 3 2 13
5 10 11 8
9 6 7 12
4 15 14 1
�����ก'+!� "ก#���3ก��%�������!,� �����ก������ก�� MATLAB workspace
����� $� �������ก'+��� ���������& A
ก��"���� workspace ���$ �"� �-� �"�#$�!���$�,���� who %�& whos :
>> whos
Name Size Bytes Class Attributes
A 4x4 128 double
Workspace BrowserWorkspace Browser
The MATLAB workspace consists of the set of variables (named arrays) built up during a MATLAB session and stored in memory.
You add variables to the workspace by using functions, running M-files, and loading saved workspaces.
To view the workspace and information about each variable,
use the Workspace browser, or use the functions who and whos.
sum, transpose, and sum, transpose, and diagdiag
>> sum(A)
ans = 34 34 34 34
A =
16 3 2 13
5 10 11 8
9 6 7 12
4 15 14 1
34 34 34 34
>> A’
ans = 16 5 9 4 3 10 6 15 2 11 7 14 13 8 12 1
>> sum(A’)’
>> diag(A)
>> sum(diag(A))
SubscriptsSubscripts
����ก��� �� � i ���� ����� j � ������ก A !� "กก,%����� A(i,j)
��� ������� A(4,2) �& ��������� �� ���� 4 ���� �������� 2
>> A(4,2)
ans =
15
A =
16 3 2 135 10 11 89 6 7 124 15 14 1
������� $�$ �ก��,��45����� �� �������� 4 � � A:
>> A(1,4)+ A(2,4)+ A(3,4)+ A(4,4)
ans =
34
The Colon OperatorThe Colon Operator
��� � : �& ���� �� ���� �+����,��6�ก���%�3���� MATLAB
>> 100:-7:50
>> 1:10
ans =
1 2 3 4 5 6 7 8 9 10
�����,���� !���$���ก�� �+� �� �� �!,�����)�!ก 1 3� 10
$�$ �ก�����%�� &�����-�����ก�#%�3�� �%$ก,%��������%��
ans =
100 93 86 79 72 65 58 51
SubmatrixSubmatrix
>> A(1,:)
>> A(:,2)
>> A(3:4,1:2)
����� ��$ : �2&� ก,%��#������ ������ก'+ ���� A(1:k,j)
�& ����ก������ 1 3� k � �� �������� j � ������ก'+ A
>> sum(A(1:4,4)
>> sum(A(:,end)
$�$ �ก��,��45����� �� �������� 4 �,���������$�&
%�& !��$��.�������ก�������$�,�� end %�� 3�� �� �%�& � ������7��$�
>> [A B]
>> size(ans)
>> [A ; B]
JuxtapositionJuxtaposition
>> B = [9 8 2 5 ; 4 5 6 7 ; 2 1 3 4]
Row vector >> A = [ 2 3 5 7 11]
Column vector >> A = [ 2; 3; 5; 7; 11]
Transposition >> At = A’
Row and Column VectorsRow and Column Vectors
>> A / B
>> A ./ B
>> A .^ 2
>> odd = 1:2:11
>> even = 2:2:12
>> natural = 1:6
>> angle = 0:pi/10:pi;
>> sin(angle)
>> A = [ 3 5 7 9 11 ]
>> A(3)
>> length(A)
>> clear(A)
>> B = [ 2 4 6 8 10 ]
>> A + B
>> A - B
>> A * B
>> A .* B
Arrays OperationsArrays Operations
Generate matrices using builtGenerate matrices using built --in functionsin functions
>> A = zeros(4) >> A = zeros(3,4)
>> A = ones(4)
>> A = eye(4)
>> A = magic(4)
>> S = A + B
>> D = A - B
>> A*B
>> C = [ 10 11; 12 13; 14 15];
>> A*C
>> A^2
>> L = log10(A)
>> [m , n] = size(A)
>> det(A)
>> inv(A)
>> v = [1 2 3];
>> A = diag(v)
>> B = diag([1 2 1 2])
>> w = diag(B)
Elementary Matrix OperationsElementary Matrix Operations
linspace function creates row vectors with equally spaced elements.
>> u = linspace(0.0,0.25,5)
>> v = linspace(0,9,4)’
>> x = linspace(0,pi/6,6*pi);
>> s = sin(x);
>> c = cos(x);
>> t = tan(x);
>> [x’ s’ c’ t’]
LinspaceLinspace
Example 2.1 Transportation route analysisExample 2.1 Transportation route analysis
The following table gives data for the distance travel along five truck routes and thecorresponding time required to traveled each route. Use the data to compute the average speed required to drive each route. Find the route that has the highest average speed.
7.510.19.18.210.3Time (hrs)
370530490440560Distance (miles)
54321
Solution:>>d = [560, 440, 490, 530, 370]
>>t = [10.3, 8.2, 9.1, 10.1, 7.5]
>>speed = d./t
speed =54.3689 53.6585 53.8462 52.4752 49.3333
>>[highest_speed, route] = max(speed)
highest_speed =54.3689
route =1
Lecture 2 - MATLAB 2
Numerical Methods for Civil EngineersNumerical Methods for Civil Engineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
�� Save & Load from External FilesSave & Load from External Files
�� GRAPHICSGRAPHICS
- Creating a plot
- Plotting Tools
- Plot Graph from Data
Save Data to External FileSave Data to External File
>> clear
>> x = linspace(0,2*pi); y = cos(x); z = sin(x);
>> save xyz
Load Data from External FileLoad Data from External File
>> whos
>> load xyz
>> whos
���������ก�ก� ������ xyz.matxyz.matxyz.matxyz.mat �������� binary filebinary filebinary filebinary file ������ Current DirectoryCurrent DirectoryCurrent DirectoryCurrent Directory
>> clear
>> XY = load(‘xyvals.txt’)
>> x = XY(:,1)
>> y = XY(:,2)
>> clear
>> x = 0:5; y=5*x;
>> XY = [x’ y’];
>> save xyvals.txt XY -ascii
Save Data to Plain Text FileSave Data to Plain Text File
Load Data from Plain Text FileLoad Data from Plain Text File
Command Description
Input/Output CommandsInput/Output Commands
User
Input
MATLABMATLAB
Output - Screen
- File
disp (A) Displays the contents, but not the name, of theArray A.
disp (‘text’) Displays the text string enclosed within single quotes.
fprintf Control the screen’s output display format.
x = input(‘text’) Displays the text in quotes, waits for user input fromthe keyboard, and stores the value in x.
x = input(‘text’,’s’) Displays the text in quotes, waits for user input fromthe keyboard, and stores the input as a string in x.
INPUT AND OUTPUTINPUT AND OUTPUT
Prompting for User Input
>> x = input(‘Enter a value for x’);
By default, the input function returns a numerical value.
To obtain a string input, a second parameter, ‘s’ , must be provided.
>> yourname = input(‘Enter your name ‘,‘s’);
function s = inputAbuse% inputAbuse Use input messages to compute sum of 3 variables
x = input(‘Enter the first variable to be added ‘);y = input(‘Enter the second variable to be added ‘);z = input(‘Enter the third variable to be added ‘);s = x+y+z;
Text OutputText Output
>> disp(‘My favorite color is red’)
Since disp requires only one argument, the message and variable
must be combined into a single string.
>> yourName = input(‘enter your name ‘,’s’);
>> disp([‘Your name is ‘,yourName])
The disp Function
>> Speed = 63;
>> disp(‘The vehicle’s predicted speed is:’)
>> disp(Speed)
G R A P H I C SG R A P H I C S
Creating a PlotCreating a Plot
ก�����ก����� MATLAB ����� ������� plot � ����� y ������ก����� plot(y)
�����ก���������� y ����� � !�� y
������ก�"�������ก������������� plot(x,y) MATLAB �����ก������� �ก����������ก����� x �#��ก��$��������ก����� y
>> x = 0:pi/100:2*pi;
>> y = sin(x);
>> plot(x,y)
��������� ������������ก�����ก���!�� sine(x) ���� ��� x ��%���� 0 �&� 2ππππ �"�'$�'�������!���#���#���"�������������
Graph ComponentsGraph Components
MATLAB ������ก�����"������� Figure 1 ������(�!���#���
>> xlabel('x = 0 to 2\pi')
>> ylabel('Sine of x')
>> title('Plot of the Sine Function','FontSize',12)
����������)�����#��'$���������ก�)� �� !���������ก�)
ก�"�� ���ก������*# �#� ����ก�$� :
>> axis([0 2*pi -1.2 1.2])
>> grid on
Plotting ToolsPlotting Tools
���,����,� ������ก�����"������� Figure �� �����ก��ก�"���(�#�ก-.��"�����/���� ��������ก��
ก���� �ก� ����,����,� �����"������������� :
>> plottools
ก���������,����,� ������ก�����ก���( View "�,��#$ก�01� Show Plot Tools
)�(#)�����"������� Figure
����ก���2�ก3�����#$ก�01� Hide Plot Tools ���(��$�ก��
� �#$ก�#,�ก����ก��� ����ก��� ��� Property Editor ����#��� ���"�ก�"��#�ก-.� !������ก���
� �#$ก�#,�ก����"������ก����#�� ��/����� �����ก��
� �#$ก��',%� ����� ก�"������ก�$������ x �#� y
���ก�������� ���������!��ก"����ก�������� ���������!��ก"�
>> legend('sin(x)','sin(x-.25)','sin(x-.5)')
>> clear
>> x = 0:pi/100:2*pi;
>> y = sin(x);
>> y2 = sin(x-.25);
>> y3 = sin(x-.5);
>> plot(x,y,x,y2,x,y3)
� ������� legend �����)0����ก��� :
>> axis([min(x) max(x) min(y) max(y)]
Controlling the AxesControlling the Axes
>> hold on
>> plot([min(x) max(x)],[0,0])
>> hold off
>> grid
Plot Graph from DataPlot Graph from Data
��������������ก�����ก!���(# �� ��(� ����������ก����� x �#� y �� !������ก��
>> x = [0 1 2 3 4 5]
>> y = [0 6 7 5 8 9]
>> plot(x,y)
"�,����!��!���(#��ก/�#�5����ก:
>> clear
>> load(‘xy.data’)
>> xy
>> x = xy(:,1)
>> y = xy(:,2)
>> plot(x,y)
marker
Import from EXCELImport from EXCEL
� �����/�#�!���(# xls ���� � EXCEL
>> clear
>> xlsread(‘xy.xls’)
>> x = xy(:,1)
>> y = xy(:,2)
>> plot(x,y)
Lecture 3 - MATLAB 3
Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Programming with MATLABProgramming with MATLAB
�� Working with MWorking with M --FilesFiles
�� Script MScript M --FilesFiles
�� Function MFunction M --FilesFiles
�� Flow ControlFlow Control
Working with MWorking with M --FilesFiles
OverviewOverview
�� MATLAB ���������� ก�����������ก����������������� �� ������������!"�#�����$����%$&�$�'���������#��'����'
�������(���������ก��$��� text �%$&)����� ����!�����ก"$ .m ����ก'�� M-file
�!��(���%$&! �� filename.m #�'�� filename ก�*+��,�#�������ก������ก�!�����ก�� �����-������+'��������-�! ��.-�#������������/��$�'�� MATLAB
�����!�0�# � scripts .1����,�!"�#�����)�����
�$+ functions .1���� ������ก�!�*+�����#�� input
�$+�� #�� output
Types of MTypes of M --FilesFiles
Open a BuiltOpen a Built --in Editorin Editor
���'��*+����� M-file .1����,��%$& text ����!� editor �!�� Notepad � � Wordpad
ก���� ���(���!� built-in editor �����ก� MATLAB *+�+�'กก'����ก
ก������ก�!� built-in editor ���������$ �ก���/ File > New > Blank M-file
� �#$0ก�"2� New M-File
Script MScript M --FileFile
�� M-file ����!��#� ������ % ��0����� ��������*+��������#'���)0 ��ก���!����
$������� M-file ��-�! ��'�� prod_abc.m ������/�����$���
A Script to Plot FunctionsA Script to Plot Functions
trigplot.m
t = linspace(0,2*pi);
y1 = sin(t);
y2 = cos(t);
y3 = y1.*y2;
plot(t,y1,’-’,t,y2,’.’,t,y3,’—’);
>> trigplot
Replace the plot statement with
plot(t,y1,’-’,t,y2,’:’,t,y3,’—’);
axis([0 2*pi -1.5 1.5])
legend(‘sin(t)’,’cos(t)’,’sin(t)*cos(t)’);
>> close all; trigplot
Use TEX notation to display “ θθθθ”
legend(‘sin(\theta)’,’cos(\theta)’,
’sin(\theta)*cos(\theta)’);
xlabel(‘\theta (radius)’,’FontName’,’Times’,
’FontSize’,14)
>> close all; trigplot
>> x = linspace(0,2*pi);>> y1 = sin(x);>> y2 = cos(x);>> y3 = y1.*y2;>> plot(x,y1,'-',x,y2,':',x,y3,'--');>> axis([0 2*pi -1.5 1.5])>> legend('sin(\theta)','cos(\theta)','sin(\theta)*cos (\theta)')>> xlabel('\theta (radius)','FontName','Times','Fon tSize',14)>> title('Plot of simple trigonometric functions',.. .
'FontName','Times','FontSize',12)
FunctionFunction MM--FilesFiles
Command Window>>
input parameters
function
output parameters
function [outputParameterList] = functionName (inputPramaterList)
��ก*�ก Built-in function �������ก� MATLAB �$�' ��������(�����%3�ก&!����1-����!������� .1��*+�����(�� #�������0����&������#��'4�$�'���ก$� 5$$��)&������
Function Definition LineFunction Definition Line
�������ก�� M-file �� �� �ก MATLAB '����,�%3�ก&!��� ��0�������'�#�'�� function
��� functionName *+������,�! ������'ก� M-file �� ������# � functionName.m
pyt.m
function h = pyt(a, b)
h = sqrt(a.^2 + b.^2);
>> a = 7.5
>> b = 3.342
>> c = pyt(a,b)
c =
8.2109
Input argument
function name
output argument
Creating a Function MCreating a Function M --FileFile
1 ����� M-file ����!� text editor :
2 ����ก�!� M-file *�ก���������#����� � �*�ก M-file � ��
Creating PCreating P--Code Code FilesFiles
You can convert average.m into a pseudocode called P- code file.
>> pcode average
Text file:can be viewedby any editor
Pseudocode:can’t be viewedby any editor
- Faster for large program
- Use to hide algorithm
FLOW CONTROLFLOW CONTROL
MATLAB has several flow control constructs:
•• ifif
•• switch & caseswitch & case
•• forfor
•• whilewhile
•• continuecontinue
•• breakbreak
if Conditional Control
if condition
expression
elseif condition
expression
else
expression
end
Condition:
Equal A == B
Not equal A ~= B
Greater A > B
Smaller A < B
Greater or equal A >= B
Smaller or equal A <= B
AND &
OR |
The if statement evaluates a logical expression and executes a group of statements when the expression is true. The optional elseif and else keywords provide for the execution of alternate groups of statements. An end keyword, which matches the if, terminates the last group of statements.
if a < 0disp(‘a is negative’);
end
Example 1:
if a < 0, disp(‘a is negative’); end
Example 2:
if x >= yc = x^2 - y;
elseif y/x > 2.0c = log(y/x);
elsec = x + y;
end
Example 3:
switch expression
case value1
block of statements
case value2
block of statements
.
.
.otherwise
block of statements
end
switch sign(x)case -1
disp(‘x is negative’);case 0
disp(‘x is exactly zero’);case 1
disp(‘x is positive’);otherwise
disp(‘sign test fail’);end
switch & caseThe switch statement executes groups of statements based on the value of a variable or expression. The keywords case and otherwise delineate the groups. Only the first matching case is executed. There must always be an end to match the switch.
>> P = zeros(5, 5);
>> for k = 1:5for l = 1:5
P(k, l) = pyt(k, l);end
end
>> P
forThe for loop repeats a group of statements a fixed,
predetermined number of times.
A matching end delineates the statements.
while condition
expressions
end
Break Loops break
>> x=1;>> while 1+x > 1
x = x/2;end
>> x
Return Loops return
return
break
LOOP
whileThe while loop repeats a group of statements an indefinite number of times under control of a logical condition.
A matching end delineates the statements.
Here is a complete program, illustrating while, if, else, and end, that uses interval bisection to find a zero of a polynomial.
a = 0; fa = -Inf;b = 3; fb = Inf;while b-a > eps*b
x = (a+b)/2;fx = x^3-2*x-5;if sign(fx) == sign(fa)
a = x; fa = fx;else
b = x; fb = fx;end
endx
The result is a root of the polynomial x3 - 2 x - 5 , namely
x =2.09455148154233
a
fa
fb
bx
Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Lecture 4 : Approximations & Errors
� Significant Figures
� Error Definitions
� Round-off Errors
� Truncation Errors
� Taylor Series
• For many engineering problems, we cannot obtain
analytical solutions.
• Numerical methods yield approximate results that are close to the exact analytical solution.
• How confident we are in our approximate result?
• The question is “how much error is present in our calculation and is it tolerable?”
Approximation & Approximation & ErrorError
• Modeling Error (������������� �����������)
• Round-off Error (���������������กก��������)
• Truncation Error (���������������กก�������)
• Data Error (�������������ก �����)
• Human Error (�������������ก�� �!")
Sources of Sources of ErrorError
Composite surfaces andterrain modellingMixed approximation methods
Significant FiguresSignificant Figures
Car speed = ?
49 km/h48.8 km/h48.9 km/h48.86 km/h
2 digits3 digits3 digits4 digits
ZERO is NO significant: 184500, 0.1845, 0.01845, 0.00001845 are 4 digits
Significant FiguresSignificant Figures
� Number of significant figures indicates precision.
� Significant digits of a number are those that can be used with confidence, e.g., the number of certain digits plus one estimated digit.
53,800 How many significant figures?
5.38 x 104 3
5.380 x 104 4
5.3800 x 104 5
� Zeros are sometimes used to locate the decimal point not significant figures.
0.00001753 4
0.0001753 4
0.001753 4
Use of Significant Figures in Numerical MethodsUse of Significant Figures in Numerical Methods
1) Numerical method yield approximation results. So we have
to specify how confident in our approximated results.
2) Specific quantities such as π , e , or cannot be expressed
exactly by a limited number of digits.
7
π = 3.141592653589793238462643 . . .
But computers can contain only . . .
>> format short ; pi = 3.1416 (5 digits)
>> format long ; pi = 3.14159265358979 (15 digits)
Try also exp(1), sqrt(7)
* Omission of the remaining significant figures is called round-off error.
Accuracy and PrecisionAccuracy and Precision
Increasing accuracy
Incr
easi
ng p
reci
sion
Computes valueclose to true value
Computed value close to each other
A C C U R A C Y
P R E C I S I O N
B I
A S
UNCERTAINTY
• Accuracy. How close is a computed or
measured value to the true value
• Precision (or reproducibility). How close is a
computed or measured value to previously
computed or measured values.
• Inaccuracy (or bias). A systematic deviation
from the actual value.
• Imprecision (or uncertainty). Magnitude of
scatter.
Error DefinitionsError Definitions
True value = Approximation + Error
True Error, Et = True value – Approximation
�������������� �������������� ����� ���
True fractional relative error = True Error
True value
×××× 100%True Error
True valueTrue percent relative error, εεεεt =
������ ������� ���� ��ก������������ �������!"�ก#$%�&��'�()��
εa =current approximation - previous approximation
current approximation100%
Approximation Approximation ErrorError
��ก���$�������� ����%ก&����� ������� (������ก+&��)��������� ���������(��������)
�%��%-��.�)��� ���������������������ก������� ���&�� ��
×××× 100%Approximate Error
ApproximationPercent Approximate error, εεεεa =
��ก���ก��"/��'�(ก���� ���0- ����(1��%-� ������&���������2.-����(1
εs = (0.5×102-n)%
�������3��������ก ����2.-��� (��%4&�� Prespecified tolerance, εs
��� |εa| < εs �������� �)4�%-���ก)���.��5��6��(��� n
EXAMPLE 4.1 Error Estimates for Iterative Methods
= + + + + +⋯2 3
12! 3! !
nx x x x
e xn
Approximate e0.5 by using the series
true value: >> format long ; exp(0.5)
>> ans =
1.648721 . . .
1st estimate: ex = 1
2nd estimate: ex = 1 + x e0.5 = 1 + 0.5 = 1.5
true error: ε −= × =
1.648721 1.5100% 9.02%
1.648721t
approximate error: ε −= × =
1.5 1100% 33.3%
1.5a
Terms Results εεεεt (%) εεεεa (%)
123456
11.51.6251.6458333331.6484375001.648697917
39.39.021.440.1750.01720.00142
33.37.691.270.1580.0158
After six terms are included, the approximate error falls below εa = 0.05%,and the computation is terminated.
>> Term = 1:6
>> et = [39.3 9.02 1.44 0.175 0.0172 0.00142]
>> ea = [33.3 7.69 1.27 0.158 0.0158]
>> plot(Term,et,'o-‘,Term(2:6),ea,'+-')
ROUND-OFF
ROUNDROUND--OFFOFF ERRORSERRORS
>> format long e
>> 2.6 + 0.2
>> ans + 0.2
>> ans + 0.2
Creation of erroneous digit
� Computers retain only a fixed number of significant figures
� All computations in MATLAB are done in double precision
FORMAT LONG E : Floating point format with 15 digits.
True value π = 3.141592653589793238462643 . . .
MATLAB pi = 3.141592653589793e+000
% display all of the significant digits
Computer Representation of NumbersComputer Representation of Numbers
Numbers on the computers are represented with a binary (base 2) system.
1 0 1 0 1 1 0 1
27 26 25 24 23 22 21 20
1 × 1 = 1
173
0 × 2 = 01 × 4 = 41 × 8 = 80 × 16 = 01 × 32 = 320 × 64 = 01 × 128 = 128
We use the decimal (base 10) system
Bits Bits && BytesBytes
BIT = Binary digit (0 or 1)
= 0 = 1
BYTE = Group of 8 bits
= 0001 1011
= 27
MAX BYTE NUMBER = 1111 1111 = 27+26+25+24+23+22+21+20 = 255
Integer RepresentationInteger Representation
Computer uses 2 bytes = 16 bits & 1st bit for sign
1 0 0 0 0 0 1 0 1 1 0 1 0 0 1 1
Sign Number
Integer range: [ -32768 to 32767 ]
Upper limit = 214 + 213 + . . . + 22 + 21 + 20
= 215 - 1 = 32,767
Zero = 0000 0000 0000 0000
- Zero = 1000 0000 0000 0000
Redundant
Mantissa
Signed exponent
37.5 = 0.3750 × 102
-812.5 = - 0.8125 × 103
0.005781 = 0.5781 × 10-2
FloatingFloating --point Numberspoint Numbers
Stored in binary equivalent of scientific notation
123.456 = 123.456 × 100 = 1.23456 × 102 = 123456 × 10-3
MATLAB uses the IEEE floating-point standard
Sign
Signed exponent Mantissa
How computer stored floating-point number
Sign
Single Precision 32-bitSign1 bit
Signed exponent8 bit
Mantissa23 bit
Range: +1.18 × 10-38 to +3.40 × 1038
−3.40 × 1038 to −1.18 × 10-38
Double Precision 64-bit
Sign1 bit
Signed exponent11 bit
Mantissa53 bit
Range: +2.23 × 10-308 to +1.80 × 10308
−1.80 × 10308 to −2.23 × 10-308
Single & Double PrecisionSingle & Double Precision
MATLAB Discrete ApproximationMATLAB Discrete Approximation
underflow
underflow usable rangeusable range overflowoverflow
-10+308 -10-308 10-308 103080
denormal
- realmax - realmin realmin realmax
>> format long e
>> 10*realmax
>> realmin/10
>> realmin/1e16
denormal (fewer significant digits)
Effect of Order of OperationsEffect of Order of Operations
0.99 + 0.0044 + 0.0042 = 0.9986
3-Digits: (0.99 + 0.0044) + 0.0042 = 0.998
0.99 + (0.0044 + 0.0042) = 0.999
x approximate x* to t significant digits when
*
5 10 tx x
x−−
< ×
Example: 4 30.998 0.99866.012 10 5 10
0.998− −−
= × < ×
= 3 significant digits
TRUNCATION ERRORSTRUNCATION ERRORS
Example: Derivative of velocity 1
1
( ) ( )i i
i i
v t v tdv v
dt t t t+
+
−∆≅ =
∆ −
dv
dt
v
t
∆∆
Time
Vel
ocity Continuous
ti ti+1
Discrete
Truncation error results from using an approximation in place ofan exact mathematical procedure.
Bungee Jumper ProblemBungee Jumper Problem
rate of change of velocity with respect to time,
2vm
cg
dt
dv d−=
where v = vertical velocity (m/s), t = time (s),
g = gravity acceleration ( ≅ 9.81 m/s2)
cd = drag coefficient (kg/m)
m = jumper’s mass (kg)
Analytical solution by solving differential equation,
= t
m
gc
c
gmtv d
d
tanh)(xx
xx
ee
eex −
−
+−
=)tanh(
Example: Compute velocity of a free fall bungee jumper with a mass of 70 kg.Use a drag coefficient of 0.25 kg/m.
)1872.0tanh(41.5270
)25.0(81.9tanh
25.0
)70(81.9)( tttv =
=
>> t=0:2:12;>> v=52.41*tanh(0.1872*t);>> [t’ v’]
0 02.0000 18.75414.0000 33.25066.0000 42.38298.0000 47.4160
10.0000 49.987412.0000 51.2501
ans =
>> plot(t,v)0 2 4 6 8 10 12
0
10
20
30
40
50
60
Time, s
Vel
ocity
, m/s
Analytical Solution for the bungee jumper problem
Terminal velocity
EulerEuler ’’s method:s method:
Numerical Solution to the differential equationNumerical Solution to the differential equation
tti+1ti
v(ti)
v(ti+1)
True slopedv/dt
Approx. slope∆v
∆t
ii
ii
tttvtv
tv
−−
=∆∆
+
+
1
1 )()(
ii
ii
tttvtv
tv
dtdv
−−
=∆∆
≅+
+
1
1 )()(
Rate of change of velocity can be approximated by
Substitute into dv/dt = g – (cd/m)v2 to give
2
1
1 )()()(
id
ii
ii tvmc
gtt
tvtv−=
−−
+
+
Rearrange equation to yield
)()()()( 12
1 iiid
ii tttvmc
gtvtv −
−+= ++
tdtdv
tvtv iii ∆+=+ )()( 1 New value = old value x step size
Example 2: Numerical Solution to the Bungee Jumper Problem
Perform the same computation as previous example but use
the Euler’s method. Employ a step size ∆t = 2 sec.
@ start ti = 0 s, ti+1 = 2 s, v(0) = 0 m/s:
)()()()( 12
1 iiid
ii tttvmc
gtvtv −
−+= ++
m/s 62.192)0(7025.0
81.90)2( 2 =×
−+=v
Next step ti = 2 s, ti+1 = 4 s, v(2) = 19.62 m/s:
m/s 49.362)62.19(7025.0
81.962.19)4( 2 =×
−+=v
>> 0 [Enter]>> ans+(9.81-(0.25/70)*ans^2)*2 [Enter][Up Arrow] [Enter] many time . . .
MATLAB:
t(s) v(m/s)0 02 19.624 36.496 46.608 50.7110 51.9612 52.30
>> hold on>> v2 = [0 19.62 36.49 46.6 50.71 51.96 52.3]>> plot(t,v2,'+-')
0 2 4 6 8 10 120
10
20
30
40
50
60
Time, s
Vel
ocity
, m/s
Analytical Solution for the bungee jumper problem
Terminal velocity
Analytical solution
Approx. solution
T A Y L O R S E R I E ST A Y L O R S E R I E S
• It is common practice to use a finite number of
terms of the series to approximate a function.
• Taylor series is a representation of a function as an
infinite sum of terms calculated from the values of its
derivatives at a single point.
• It is named after the English mathematician Brook Taylor .
• If the series is centered at zero, the series is also
called a Maclaurin series , named after the Scottish
mathematician Colin Maclaurin .
x
f(x)
T A Y L O R S E R I E ST A Y L O R S E R I E S
a
f(a)f(x)
x
The Taylor series of a function f(x)
that is indefinitely differentiable in a
neighbourhood of a, is the power
series
DefinitionDefinition
⋯+−′′′
+−′′
+−′+= 32 )ax(!3
)a(f)ax(
!2)a(f
)ax)(a(f)a(f)x(f
where n! = the factorial of n
f(n)(a) = the n-th derivative of f at point a
( 1)1( )
( 1)!
nn
n
fR h
n
ξ++=
+Remainder:
nin
iiiii h
nxf
hxf
hxf
hxfxfxf!
)(!3
)(!2
)()()()(
)(32
1 ++′′′
+′′
+′+≅+ ⋯
nni
nii
iii Rhn
xfh
xfh
xfhxfxfxf +++
′′′+
′′+′+=+ !
)(!3
)(!2
)()()()(
)(32
1 ⋯
A remainder term, Rn is included to account for all terms from n+1 to ∞.
TTayloraylor SSerieseries ((nnthth--order approximation)order approximation)
Let’s change a to x i and x to x i+1 and define the step size as h = (x i+1- x i),
the series becomes:
The complete Taylor series is given by
Note that, an equal sign replaces the approximate sign.
Where ξξξξ is not known but lies somewhere between xi and xi+1
xi ξξξξ xi+1
Second
order
f(xi+1) ≅ f(xi) + f’(xi)h + f’’(xi)h2/2!
h
f(xi+1) ≅ f(xi) + f’(xi)h
First order
Zero-order approx .f(xi+1) ≅ f(xi)
True
xi+1
f(xi+1): want to know
T A Y L O R S E R I E ST A Y L O R S E R I E S
Predict function value of one point in term of function value and its derivatives at other points
xi
f(xi)
Known
x
f(x)
Slope @ xi
Example: Approximation of function cos( x) by Taylor Series Expansion
Use Taylor series expansions with n = 0 to 6 to approximate f(x) = cos x at xi+1 = π /3
on the basis of the value of f(x) and its derivatives at xi = π /4.
Step size: h = π /3 - π /4 = π /12 True value: cos(π /3) = 0.5
Zero-order approximation: f(π /3) ≅ cos(π /4) = 0.707106781
which represents a percent relative error of
%4.41%1005.0
707106781.05.0=×
−=tε
First-order approximation: f’(x) = -sin(x)
f(π /3) ≅ cos(π /4) – sin(π /4)(π /12) = 0.521986659
which has εt = 4.40%
Solution:
Second-order approximation: f’’(x) = -cos(x)
which has εt = 0.449%
497754491.0122
)4/cos(124
sin4
cos3
2
=
−
−
≅
ππππππ
f
Order n
0
1
2
3
4
5
6
Approximate
0.7071067810.521986659
0.497754491
0.499869147
0.500007551
0.500000304
0.499999988
εεεεt(%)
41.44.40
0.449
2.62 x 10-2
1.51 x 10-3
6.08 x 10-5
2.44 x 10-6
To get more accurate approximation, we can use higher order (add more terms)
and/or reduce the step size h.
MaclaurinMaclaurin SSerieseries
��7� special case 2� Taylor Series �� ก ������� a = 0
Taylor Series :
⋯+−′′′
+−′′
+−′+= 32 )ax(!3
)a(f)ax(
!2)a(f
)ax)(a(f)a(f)x(f
Taylor Series with a = 0 :
⋯+′′′
+′′
++= 32 x!3
)a(fx
!2)a(f
x)0('f)0(f)x(f
Using this formula Taylor series of functions can often be rather
easily computed.
TTayloraylor Series of a Function: Series of a Function: exp(xexp(x ))
� Compute several derivatives of the given function.
� Evaluate these derivatives at x = 0.
,e)x(f x= ,e)x(f x=′ ,e)x(f x=′′ …, x)n( e)x(f =
,1e)0(f 0 == ,1)0(f =′ ,1)0(f =′′ …, 1)0(f )n( =
⋯+′′′
+′′
++= 32 x!3
)a(fx
!2)a(f
x)0('f)0(f)x(f
� Substitute into the series,
2 3
( ) 1 ...2! 3! !
nx x x x
f x e xn
= = + + + + +
MATLAB Example: MATLAB Example: TTayloraylor Series of Series of exp(xexp(x))
>> clear
>> x = -3:0.1:3;
>> y = exp(x);
>> plot(x,y)
>> y0 = ones(size(x));
>> plot(x,y,x,y0)
>> y1 = y0 + x;
>> plot(x,y,x,y1)
>> y2 = y1 + x.^2/2;
>> plot(x,y,x,y2)
>> y3 = y2 + x.^3/6;
>> plot(x,y,x,y3)
���� y4, y5, y6 ... Until then ?
MATLAB Example2: Taylor Series of exp(x) Symbolic Toolbox
>> clear
>> syms x;
>> y = taylor(exp(x),5,x,0)
% declare symbolic x
% expand ex about x = 0 to order 5.
y =
x^4/24 + x^3/6 + x^2/2 + x + 1
Compare at x = 2 with the exact value.
>> approx = subs(y,x,2);
>> exact = exp(2);>> et = (1-approx/exact)*100
et =
5.2653
Plot graph with ezplot :
>> ezplot(y)
>> hold on>> ezplot(exp(x))>> hold off
Which line is an approx.red or blue ?
TaylorTaylor Approximation ErrorApproximation Error
How accurate is the Taylor series approximation?
The n terms of the approximation are the first n terms of the exact expansion:
!2x
x1e2
x ++= ⋯++!3
x3
approximate truncation error
So the function f(x) can be written as the Taylor series approximation plus
an error (truncation) term:
f(x) =Pn(x) + Rn
( 1)1( )
( 1)!
nn
n
fR h
n
ξ++=
+Remainder:
3 5 2 1
sin( ) ( 1)3! 5! (2 1)!
nnx x x
x xn
−
= − + − − −−
⋯
2 4 2
cos( ) 1 ( 1)2! 4! (2 )!
nnx x x
xn
= − + − + −⋯
Taylor Series Expansions for Some Common FunctionsTaylor Series Expansions for Some Common Functions
for all x
for all x
2 3
12! 3! !
nx x x x
e xn
= + + + + +⋯ for all x
2 3 4
ln(1 ) ( 1)2! 3! 4! !
nnx x x x
x xn
+ = − + − − − −⋯ -1 1x≤ ≤
3 5 2
arctan( ) ( 1)3! 5! (2 1)!
nnx x x
x xn
= − + − − −−
⋯
2 3( 1) ( 1)( 2)(1 ) 1
2! 3!p p p p p p
x px x x− − −
+ = + + + +⋯ for 1x <
-1 1x≤ ≤
( 1)1 1( )
( )( 1)!
nn n
n
fR h O h
n
ξ++ += =
+
BigBig --O NotationO Notation
f(x) =Pn(x) + O(hn+1)
Big-O is used to describe the error term in an approximation.
For example:
)x(O!2
xx1e 3
2x +++= 0xas →
expresses the fact that the error, the difference
++−
!2x
x1e2
x
is smaller in absolute value than some constant time |x3|
when x is close enough to 0.
Example : Estimate the error of the approximation
From MATLAB example 2, let’s estimate the error in the interval [-1, 1]
[-1,1]
!4x
!3x
!2x
x1e432
x ++++≈
)x(O!4
x!3
x!2
xx1e 5
432x +++++=
The error done when approximating exp(x) by a Taylor series of degree
5 is bounded by the absolute value of the first term left out.
%8333.0100!5
1100
!5
x 5
t =×≤×≤ε for -1 ≤ x ≤ 1
Example: Taylor series approximation of function sin(x)
Plot the first n = 1 to 6 terms of the Taylor series of ⋯−+−=1206
)sin(53 xx
xx
>> x = -2*pi:pi/30:2*pi;>> y = sin(x);plot(x,y)>> axis([-2*pi 2*pi -1.5 1.5])>> hold on>> y1 = x; plot(x,y1)>> y2 = x-x.^3/6; plot(x,y2)>> y3 = x-x.^3/6+x.^5/120; plot(x,y3)>> legend('sin(x)','n = 1','n = 2','n = 3')
-6 -4 -2 0 2 4 6-1.5
-1
-0.5
0
0.5
1
1.5
sin(x)n = 1n = 2n = 3
f x =
Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Lecture 5 : Roots of Equations
� Graphical Methods
� Bracketing Mehods
� Open Methods
� MATLAB Methods
What Are Roots?What Are Roots?
Quadratic formula: f(x) = ax2 + bx + c = 0
aacbb
x2
42 −±−=
x
f(x)
0 x1 x2
Roots = Values of x that make f(x) = 0
Open Methods: require only a single starting value of x without bracketing the root.
� Fixed-point iteration
� Newton-Raphson method
� Secant method
Root Finding MethodsRoot Finding Methods
Two major classes of methods:
Bracketing methods: start with guesses that bracket, or contain, the root and then systematically reduce the width of the bracket
� Graphical method
� False-position method
� Bisection method
f(xf(x ) = 0) = 0
x = ?x = ?
GRAPHICAL METHODSGRAPHICAL METHODS
Plot the function and observe where it crosses the x axis.
x
f(x)
0
x* = ?
Graphical method : NOT precise
ZOOM
ZOOM
Double root?
BracketingBracketing MethodMethod
Coarse level search for roots over large interval
1) Subdividing large interval into smaller subinterval
2) Examine sign at the end of each subinterval
+ + + + + + + + +- - - - - - - - - -
+ + + +
+ + + + - - - + +f (x)
xx1 x2
Possible ways in an intervalPossible ways in an interval
Part (a) and (c) : f(xl) and f(xu) same sign
No roots or Even number of roots
Part (b) and (d) : f(xl) and f(xu) different signs
One roots or Odd number of roots
Exceptions:Exceptions:
(a) Multiple roots
(b) Discontinuous function
0.00
Example: Graphical Methods
Plot the function and observe where it crosses x axis
( )0.147667.38( ) 1 40xf x e
x−= − −
f(x)
x4 8 12
16
200
-10
20
40
x f(x)
48
121620
34.11517.653
6.067-2.269-8.401
Observe: x ≈ 15 → f (15) = -0.4133
Interpolation (False-position method)
Assume as linear function for short interval
12 16
6.607
2.269
Interpolation:6.607
12 (16 12) 14.9776.607 ( 2.269)
x = + − =− −
f (14.977) = -0.3691
f (15) = -0. 4133,f (14) = 1.582
2nd Interpolation: 1.58214 (15 14) 14.793
1.582 ( 0.4133)x = + − =
− −f (14.793) = -0.013
Repeatedly halve interval while bracketing root
f (x)
xxa
xb
1) Choose interval [ xa , xb ] which has sign-change
+
-
2) Compute midpoint of interval xm = ( xa + xb ) / 2
xm
+
3) Select subinterval which has sign-change and repeat 2)
-
Bisection Method
Example: Use bisection method find the root of equation
( )0.147667.38( ) 1 40xf x e
x−= − −
f(x)
x4 8 12
16
200
-10
20
40
Initial interval[ 12 , 16 ]
Estimate root at midpoint:
12 1614
2
14.7802 14100% 5.279%
14.7802
r
t
x
ε
+= =
−= × =
True value of the root:14.7863
f (14) = 1.582→ [ 14 , 16 ]+ -
Select interval with sign-change:
Next estimation: xr = (14+16)/2 = 1515 14
100% 6.667%14aε−
= × =
< εs = 0.5%
Termination Criteria and Error EstimatesStop computation when εa < εs , Ex. εs = 0.5%
Iteration xa xb xr εa(%) εt(%)
123456
12141414.514.7514.75
161615151514.875
141514.514.7514.87514.8125
6.6673.4481.6950.8400.422
5.2791.4871.8960.2040.6410.219
• Simple Fixed-Point Iteration
• Newton-Raphson Method
• Secant Methods
• MATLAB function: fzero
• Polynomials
Open MethodsOpen Methods
� How to construct the magic formulae g(x)?
� How can we ensure convergence?
� What makes a method converges quickly or diverge?
� How fast does a method converge?
What you should know about Open MethodsWhat you should know about Open Methods
Fixed-Point Iteration
xxgxf =⇔= )(0)(
� Also known as one-point iteration or successive substitution
� To find the root for f(x) = 0 , we reformulate f(x) = 0 so that there is an
x on one side of the equation.
� If we can solve g(x) = x , we solve f(x) = 0 .
x is known as the fixed point of g(x) .
given with)( 01 xxgx ii =+
� We solve g(x) = x by computing
until x i+1 converges to x.
Fixed-Point Iteration Procedure
3) Predict new xi+1 as a function of old xi
Iterative until satisfy
1
1
100%i ia
i
x x
xε +
+
−= ×
1) Rewrite original equation f(x) = 0 into another form x = g(x).
4) Use iteration x i+1 = g(x i) to find a value that reaches convergence.
xxxx
xxxx
+=⇒=
+=⇒=+−
sin 0sin2
3 032
22Example:
2) Select initial value x0
xi+1 = g (xi)
i xi εa (%)
0123456789
10
01
0.36790.69220.50050.60620.54540.57960.56010.57110.5649
100171.846.938.317.411.25.903.481.931.11
0 1 2 3 4 5 6 7 8 9 10 11 120
0.2
0.4
0.6
0.8
1
Number of Iteration
Ap
pro
xim
ate
d R
oo
t
True value = 0.5671
>> 0>> exp(-ans)
Example: Finding root of f(x) = x - e-x = 0 ans = 0.5671
x = e-x
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
x
Two-Curve Graphical Method
>> x = 0 : 0.1 : 1;
>> plot(x,exp(-x)-x)
>> x = 0 : 0.1 : 1;
>> plot(x,x,x,exp(-x))
( ) xf x e−=
( )f x x=
0 0.2 0.4 0.6 0.8 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
x
f(x)
= e
xp(-
x) -
x Root
( ) xf x e x−= −
Example: Beam Deflection
( )xLxLxEIL
wy 43250 2
120−+−=
Given:
w0 = 1.75 kN/cm
E = 50,000 kN/cm2;
I = 30,000 cm4; and
L = 450 cm;
Uniform beam subject to linearly increasing, distributed load.
Deflection, y, given by:w0
L Find the point of maximum deflection
( ) 065120
42240 =−+−= LxLxEIL
w
dx
dy
Max. deflection occurs at x where
i
iii
xL
xLxgx
2
44
16
5)(
+==+
Using fixed point iteration, we can rearrange this equation:
( )xLxLxEIL
wy 43250 2
120−+−=
Start at x0 = 100 cm and L = 450 cm
>> L = 450
>> 100
>> (L^4+5*ans^4)/(6*L^2*ans)
x1 = 341.6152
x2 = 262.8564
x3 = 203.1365
x4 = 201.4507
...
Enter
Converge @ x = 201.2461 cm
Max. deflection occurs at x = 201 cm
>> x = 201;
>> w0 = 1.75;
>> EI = 5e4*3e4;
>> y=w0/(120*EI*L)*(-x^5+
2*L^2*x^3-L^4*x)
y = -0.1141 cm Ans.
032)( 2 =−−= xxxf
There are infinite ways to construct g(x) from f(x).
(ans: x = 3 or -1)For example,
So which one is better ?
32)(
32
32
0322
2
+=⇒
+=⇒
+=⇒
=−−
xxg
xx
xx
xx
Case a:
2
3)(
2
3
03)2(
0322
−=⇒
−=⇒
=−−⇒
=−−
xxg
xx
xx
xx
Case b:
2
3)(
2
3
32
032
2
2
2
2
−=⇒
−=⇒
−=⇒
=−−
xxg
xx
xx
xx
Case c:
Case a:
>> 4
>> sqrt(2*ans+3)
3x2x i1i +=+
x1 = 3.3166
x2 = 3.1037
x3 = 3.0344
x4 = 3.0114
x5 = 3.0038
Enter
Converge!
Case b:
>> 4
>> 3/(ans-2)
x1 = 1.5000
x2 = -6
x3 = -0.3750
x4 = -1.2632
x5 = -0.9194
x6 = -1.0376
x7 = -0.9909
x8 = -1.0031
Enter
Converge, but slower
Case c:
>> 4
>> (ans^2-3)/2
x1 = 6.5000
x2 = 19.6250
x3 = 191.0703
Enter
Diverge!
)2x/(3x i1i −=+ 2/)3x(x 3i1i −=+
Convergence
00 0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
x
y
y1 = xy2 = g(x)
0.3679
0.6922
0.5005
0.6062
Divergence
00 0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
x
y
y1 = xy2 = g(x)
x0
(((( )))) (((( )))) (((( )))) (((( ))))(((( )))) (((( )))) ⋯++++−−−−
′′′′′′′′++++−−−−′′′′++++==== ++++++++++++
2i1ii1iii1i xx
!2f
xx xfxfxfξξξξ
• King of the root-finding methods
• Newton-Raphson method
• Based on Taylor series expansion
NewtonNewton ’’s Methods Method
Truncate the Taylor series to get
At the root, f(x i+1) = 0 , so
Newton-Raphson Method
)xx)(x(f)x(f)x(f i1iii1i −′+= ++
)xx)(x(f)x(f0 i1iii −′+= +
)x(f)x(f
xxi
ii1i ′−=+
Newton-Raphson Method
� Use the slope of f(x) to predict the location of the root.
f(xi)
xixi+1
slope = f’(xi)f(x)
Root
x
� xi+1 is the point where the tangent at xi intersects x-axis.
1ii
ixx
0)x(f
+−
−=
)x(f)x(f
xxi
ii1i
′−=+
Example: Finding root of f(x) = x - e-x = 0 ans = 0.5671
xe1)x(f −+=′)x(f)x(f
xxi
ii1i
′−=+
>> 0
>> ans-(ans-exp(-ans))/(1+exp(-ans))
x1 = 0.5000
x2 = 0.5663
x3 = 0.5671
Enter
Converge!
i
i
x
xi
ie1
exx
−
−
+
−−=
>> 0>> exp(-ans)
Fixed-Point Iteration:
x1 = 1
x2 = 0.3679
x3 = 0.6922
x4 = 0.5005
x5 = 0.6062
x6 = 0.5454
x7 = 0.5796
x8 = 0.5601
x9 = 0.5711
Enter
root
x*
04x3xxf 4 ====−−−−++++====)(
xixi+1
Newton’s method - tangent line
x1
f(x1)
Approximation Sequence of Newton’s Method
f(x)
xx3
x2
f(x2)
Failure of Newton’s Method
Case 1: Inflection point in vicinity of root
f(x)
x
x1x2 x3
Failure of Newton’s Method
Case 2: Oscillate around local maximum or minimum
f(x)
x
Failure of Newton’s Method
Case 3: Jump away for several roots
f(x)
x
Failure of Newton’s Method
Case 4: Disaster from zero slope
f(x)
x
• No general convergence criteria for Newton-Raphson
method.
• Convergence depends on function nature and accuracy of initial guess.
– A guess that's close to true root is always a better choice
– Good knowledge of the functions or graphical analysis can help
you make good guesses
• Good software should recognize slow convergence or divergence.
– At the end of computation, the final root estimate should alwaysbe substituted into the original function to verify the solution.
Overcoming the Failure of Newton’s Method
• Newton-Rahpson method converges quadratically (when it converges).
– Except when the root is a multiple roots
• When the initial guess is close to the root, Newton-Rahpson method usually converges.
– To improve the chance of convergence, we could use a
bracketing method to locate the initial value for the Newton-
Raphson method.
Other Facts
for function whose derivatives are difficult to evaluate
Derivative approximated by backward finite difference
( )( ) ( )1
1
i ii
i i
f x f xf x
x x−
−
−′ ≅
−1
( )
( )i
i ii
f xx x
f x+ = −′
( )11
1
( )
( ) ( )i i i
i ii i
f x x xx x
f x f x−
+
−
−= −
−
f(xi)
xixi-1x
f(xi-1)
Secant Method
Example: Use secant method to estimate root of e-x - x = 0
Initial estimate x-1 = 0 and x0 = 1.0
True root = 0.56714329. . .
First iteration:x-1 = 0 f (x-1) = 1.00000
x0 = 1 f (x0) = -0.63212
1
0.63212(0 1)1 0.61270 8.0%
1 ( 0.63212) tx ε− −
= − = =− −
Second iteration:x0 = 1 f (x0) = -0.63212
x1 = 0.61270 f (x1) = -0.07081
2
0.07081(1 0.61270)0.61270 0.56384 0.58%
0.63212 ( 0.07081) tx ε− −
= − = =− − −
Third iteration:x1 = 0.61270 f (x1) = -0.07081
x2 = 0.56384 f (x2) = 0.00518
3
0.00518(0.61270 0.56384)0.56384 0.56717
0.07081 ( 0.00518)
0.0048%t
x
ε
−= − =
− − −
=
Secant Method
Advantage of the secant method -
• It can converge even faster and it doesn’t
need to bracket the root
Disadvantage of the secant method -
• It is not guaranteed to converge!
• It may diverge (fail to yield an answer)
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-1 0 1 2 3 4 5 6 7
ln(x)
secant
Convergence not Guaranteed
no sign check,
may not bracket the root
y = ln xx1
x2
BisectionFalse position
Secant
Newton’s
Convergence criterion 10 -14
Bisection -- 47 iterationsFalse position -- 15 iterationsSecant -- 10 iterations
Newton’s -- 6 iterations
• Bracketing methods – reliable but slow
• Open methods – fast but possibly unreliable
• MATLAB fzero – fast and reliable
fzero: find real root of an equation (not suitable for double root!)
fzero( function, x0)
fzero( function, [ x0 x1])
MATLAB Function: fzero
Example: Finding root of f(x) = x - e-x = 0 ans = 0.5671
>> f = @(x)x-exp(-x);
Write an anonymous function f:
>> z = fzero(f,0)
Then find the zero near 0:
z =
0.5671
Open Methods:
- Fixed-Point Iteration
- Newton’s Method
- Secant Method
( )0.147667.38( ) 1 40xf x e
x−
= − −
f(x)
x4 8 12
16
200
-10
20 Initial interval[ 12 , 16 ]
Example: Finding root of a function ans = 14.7863
>> f = @(x)(667.38/x)*(1-exp(-0.147*x))-40;
Write an anonymous function f:
>> z = fzero(f,[12 16])
Then find the zero between 12-16:
z =
14.7863
Bracket Method:
- Bisection Method
20 1 2( ) n
n nf x a a x a x a x= + + + +⋯
n = order of polynomial
a’s = constant coefficients
where
Roots of polynomials:
(1) n th-order equation has n real or complex roots
(2) If n is odd, at least one root is real.
(3) Complex roots exist with conjugate pairs (λ + µi and λ − µi)
Roots of Polynomials
= 0
Bisection, False-position, Newton-Raphson, Secant methods cannot be
easily used to determine all roots of higher-order polynomials.
0 2 4 6 8 10-1
-0.5
0
0.5
1
1.5
2
2.5
3y
=f(
x)
x
distinctreal roots
repeatedreal roots
complexroots
Roots of Polynomials
MATLAB Function: roots
• Evaluation the root as an eigenvalue problem
• Zeros of a nth-order polynomial
>> x = roots(c) - roots
>> c = poly(r) - inverse function
012
21n
1nn
n cxcxcxcxc)x(p +++++= −− ⋯
]ccccc[c vector tCoefficien 0121nn ⋯−=
Example: Finding root of a function
>> roots([1 -3 2])
ans =
2
1
p(x) = x2 – 3x + 2 = 0
>> roots([1 -10 25])
ans =
5
5
p(x) = x2 – 10x + 25 = 0
>> roots([1 -17 72.5])
ans =
8.5000 + 0.5000i
8.5000 – 0.5000i
p(x) = x2 – 17x + 72.5 = 0
Lecture Lecture 66 Linear Algebraic EquationsLinear Algebraic Equations
� Basic Concepts
� Gaussian Elimination
� Backward Substitution
� LU Decomposition
Numerical Methods for Civil EngineersNumerical Methods for Civil Engineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Linear Algebraic EquationsLinear Algebraic Equations
In many problems of engineering interest, you will need to be
able to solve a set of linear algebraic equations.
The methods can be broadly classified as:
Direct methods Iterative methods
� Gauss-Elimination
� Gauss-Jordan
� LU Decomposition
� Jacobi
� Gauss-Seidel
Linear Algebraic EquationsLinear Algebraic Equations
a11x1 + a12x2 + . . . a1nxn = b1
a21x1 + a22x2 + . . . a2nxn = b2
an1x1 + an2x2 + . . . annxn = bn
.
.
....
n equations
n unknowns
A system of n linear equations in n unknowns,
where all the a’s and b’s are constants.
We need to find all the x’s such that all the above equations are satisfied.
Example:
9475
2382
753
321
321
321
=−+
−=+−
=−+
ccc
ccc
ccc
Matrix Form:
−=
−−
−
9
2
7
475
382
531
3
2
1
c
c
c
A x = bSymbolic Notation
Augment Form:
−−−
−
9475
2382
7531
System of linear algebraic equations A x = b
=
mmmnmm
n
n
b
b
b
x
x
x
aaa
aaa
aaa
⋮⋮
⋯
⋮⋱⋮⋮
⋯
⋯
2
1
2
1
21
22221
11211
where aij and b i are constants, i = 1,2,…,m, j = 1,2,…,n
m = number of rows
n = number of columns
Solution :Solution : A x = b
A-1A x = A-1b
x = A-1b
MATLAB :MATLAB :
>> x = inv(A)*b
- Use matrix inversion,
>> x = A\b
- Use backslash operator,
Requirements for a Solution , Am××××n
m rows and n columns
m = nSolving Ax = b for n unknowns from n equations
m > nOverdetermined systems(e.g., least-squares problems)
m < nUnderdetermined systems(e.g., optimization)
Example 1 – Unique solution
Consider the system
2x + 3y = 8
5x - 6y = 30
Geometrically this corresponds to
two lines crossing in one point.
>> x=-10:1:10;
>> y1 = (8-2*x)/3;
>> y2 = (5*x-30)/6;
>> plot(x,y1,x,y2)
Example 2 – No solution
Consider the system
2x + 3y = 8
4x + 6y = 30
Geometrically this corresponds to
two parallel lines never crossing.
>> x=-10:1:10;
>> y1 = (8-2*x)/3;
>> y2 = (30-4*x)/6;
>> plot(x,y1,x,y2)
Example 3 – Ill-conditioned
Consider the system
2x + 3y = 8
4x + 7y = 15
Geometrically this corresponds to two almost parallel lines barely
crossing. Sensitive to round-off error.
>> x=-10:1:10;
>> y1 = (8-2*x)/3;
>> y2 = (15-4*x)/7;
>> plot(x,y1,x,y2)
rank(A) = number of linearly independent columns in A
For matrix A with m rows and n columns, and Ax = b has a solution
- if rank(A) < n , → infinite number of solutions
- if rank(A) = n , → unique solution
For a system of n equations in n unknowns written in the form Ax = b,
the solution x exists and is unique for any b if and only if rank( A) = n.
Matrix RankMatrix Rank
>> A = [3 -4 1;6 10 2;9 -7 3];>> rank(A) ans = 2
ConsistencyConsistency TestTest
11 12 1 1
21 22 2 21 2
1 2
n
nn
m m mn m
a a a b
a a a bAx b x x x
a a a b
= ⇔ + + + =
⋯⋮ ⋮ ⋮ ⋮
Element of vector x are coefficients in the linear equation
Augmented matrix:
11 12 13 1 1
11 11 11 11 2
11 11 11 11 3
11 11 11 11
n
m
a a a a b
a a a a b
A a a a a b
a a a a b
=
⋯
⋯
ɶ ⋯
⋮ ⋮ ⋮ ⋱ ⋮ ⋮
⋯
Consistent when and have the same rankA Aɶ
= [Ab]
Practical Model: y = α x + β
Example: An Inconsistent System from Data Fitting
Experiment data:x
y
1
2
2
1
3
0.5
0 1 2 3 4-2
-1
0
1
2
3
1 1 2
2 1 1
3 1 0.5
αβ
=
Matrix Form:
>> A=[1 1; 2 1; 3 1]; b=[2; 1; 0.5];
>> rank(A)
>> rank([A b])
Consistency test :
5.03
12
2
=+=+=+
βαβαβα
Data → Model:
2nd Experiment:
=
0
1
2
13
12
11
βα
Consistency test:
>> A = [1 1; 2 1; 3 1]; b = [2; 1; 0];
>> rank(A)
>> rank([A b])
0 1 2 3 4-2
-1
0
1
2
3
Exact solution
=
+
−⇒
0
1
2
1
1
1
)3(
3
2
1
)1(
[ ] [ ]TT31−=βα
Nonsingular CaseNonsingular Case
A x = b x = A-1 b
� If the coefficient matrix A is nonsingular, then it is invertible and we can
solve Ax = b as follows:
� This solution is therefore unique. Also, if b = 0, it follows that the unique
solution to Ax = 0 is x = A-1
0 = 0.
� Thus if A is nonsingular, then the only solution to Ax = 0 is the trivial
solution x = 0.
Matrix
elimination
row operation
Upper triangular
backward
substitution
Solution
Naive Gauss EliminationNaive Gauss Elimination
The most basic systematic scheme for solving system of linear equations.
The procedure consisted of two steps:
1) Forward elimination of the linear equation matrix using row operationto obtain an upper triangular matrix.
2) Backward substitution to solve for the unknowns
Forward eliminationForward elimination
′′′′′′′⇒
333
22322
1131211
3333231
2232221
1131211
ba
baa
baaa
baaa
baaa
baaa
Backward substitutionBackward substitution
1121231311
2232322
3333
/)(
/)(
/
axaxabx
axabx
abx
−−=
′′−′=
′′′′=
SubstitutionsSubstitutions
Backward substitution:Backward substitution:
Forward substitution:Forward substitution:
Upper triangular matrix
nn
nn a
bx =
1,2,,2,1),(1
1
⋯−−=−= ∑+=
nnixaba
xn
ijjiji
iii
Lower triangular matrix
11
11 a
bx =
nixaba
xi
jjiji
iii ,,3,2,1),(
1 1
1
⋯=−= ∑−
=
Pivot row
Pivot element
EXAMPLE : Naive Gauss Elimination
6443
7766
123
321
321
321
−=+−−=+−−=−+−
xxx
xxx
xxx
−−−−−−−
⇒6443
7766
1123 R1
R2
R3
Forward elimination (Row operation):
R2 + 2×R1, R3+R1:
−−−−−−−
⇒7320
9520
1123
R3 - R2:
−−−−−−
⇒2200
9520
1123
Backward substitution:
22
952
123
2200
9520
1123
3
32
321
=−−=+−−=−+−
⇒
−−−−−−
x
xx
xxx
22
952
123
3
32
321
=−
−=+−
−=−+−
x
xx
xxx
12
23 −=
−=⇒ x
2)59(2
132 =−−
−=⇒ xx
2)21(3
1321 =+−−
−=⇒ xxx
>> A = [-3 2 -1;6 -6 7;3 -4 4]>> b = [-1;-7;-6]>> x=A\b
MATLAB:
EXAMPLE : Gauss-Jordan Elimination
Pivot row
Pivot element
6443
7766
123
321
321
321
−=+−−=+−−=−+−
xxx
xxx
xxx
−−−−−−−
⇒6443
7766
1123 R1
R2
R3
Forward elimination (Row operation):
R2 + 2×R1, R3+R1:
−−−−−−−
⇒7320
9520
1123
R3 - R2:
−−−−−−
⇒2200
9520
1123
No backward substitution !
R1/-3
R2/-2R3/-2
Divide diagonal to one:
−
−
−
1100
5.45.210
33.033.067.01
R1 + 0.67R2R2 + 2.5R3
Use “1” diagonal to eliminateto identity matrix:
−
−
1100
2010
33.333.101
R1 + 1.33R3
−1100
2010
2001 ⇒⇒⇒⇒ X1 = 2
⇒⇒⇒⇒ X2 = 2
⇒⇒⇒⇒ X3 = -1
next elimination fail
GaussianGaussian Elimination with PivotingElimination with Pivoting
To prevent failure from zero diagonal elements
Augmented matrix: [ ]
2 4 2 2 4
1 2 4 3 5
3 3 8 2 7
1 1 6 3 7
A A b
− − − − = = − − − − −
ɶ
R1
R2
R3
R4
2 4 2 2 4
0 0 5 2 7
0 3 5 5 1
0 3 5 4 5
− − − − − −
R2 – R1/2,R3 + 3R1/2,R4 + R1/2
First row operation:
P I V O T I N GP I V O T I N G
Exchange row to avoid zero pivot element
2 4 2 2 4
0 0 5 2 7
0 3 5 5 1
0 3 5 4 5
− − − − − −
2 4 2 2 4
0 3 5 4 5
0 3 5 5 1
0 0 5 2 7
− − − − − −
R3=R3-R2
2 4 2 2 4
0 3 5 4 5
0 0 0 1 4
0 0 5 2 7
− − − − − − −
2 4 2 2 4
0 3 5 4 5
0 0 5 2 7
0 0 0 1 4
− − − − − −
x = A \ b
MATLAB
Solving Systems with the Backslash OperatorSolving Systems with the Backslash Operator
A x = b x = A-1 b
A \ . . . means multiply on the left by the inverse of A
>> A = [2 4 -2 -2;1 2 4 -3;-3 -3 8 -2;-1 1 6 -3]
>> b = [-4;5;7;7]
>> x = A\b
x =
1.00002.00003.00004.0000
LULU DecompositionDecomposition
� When Gauss elimination is used to solve a linear system, the row operations are applied simultaneously
to the RHS vector. [ Ax = b 1 ]
� If the system is solved again for a different RHS vector,
the row operations must be repeated. [ Ax = b 2 ]
� The LU decomposition eliminates the need to repeat the row operators each time a new RHS vector is used.
*RHS = Right Hand Side = b
A LU
LULU DecompositionDecomposition
To solve Ax = b , we start by decompose A into L and U such
that A = LU where
=
=
33
2322
131211
33231
2221
11
00
00
00
u
uu
uuu
lll
ll
l
UL
lower triangularmatrix
upper triangularmatrix
−−−
−
243
131
152
=
−
45.33
05.01
002
−
−
100
110
5.05.21Example:
−−−
1020
610
321
LULU DecompositionDecomposition
Main idea is to record the steps used in Gaussian elimination.
Consider the matrix:
−−−
=1020
1252
321
A
Gaussian elimination:
R2 – 2××××R1 (2)
For recording
Recording Method:
A
Gaussian elimination (con’t):
−−−
2)2()0(
61)2(
321R3 – 0××××R1
R3 + 2××××R1
−=⇒
120
012
001
L
Lower triangular matrix:
Mysterious coincidence !!!
ALU =
−−−
=
−−
−=
1020
1252
321
200
610
321
120
012
001
>> L = [1 0 0;2 1 0;0 -2 1]
>> U = [1 -2 3;0 -1 6;0 0 2]
>> A = L*U
A =
1 -2 3
2 -5 12
0 2 -10
L
U
Gauss elimination as Gauss elimination as LULU decompositiondecomposition
Gauss elimination can be used to decompose A into L and U as illustrated for a three-equation system,
=
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa Forward
eliminationU=
′′′′
33
2322
131211
00
0
a
aa
aaa
Store row operation @ zeros position:
′′′′
333231
232221
131211
)()(
)(
aff
aaf
aaa
,11
2121 a
af =where ,
11
3131 a
af =
22
3232 and
aa
f′′
=
[A] → [L][U]
′′′′=
33
2322
131211
00
0
a
aa
aaa
U
=
1
01
001
3231
21
ff
fL
EXAMPLE : LU Decomposition with Gauss Elimination
−−
−−=
443
766
123
A
From previous example,
−−
−−=
200
520
123
UAfter forward elimination,
=
333231
232221
131211
aaa
aaa
aaa
′′′′=
33
2322
131211
00
0
a
aa
aaa
,23
6
11
2121 −=
−==
aa
f
,13
3
11
3131 −=
−==
aa
f
122
22
3232 =
−−
=′′
=aa
f
Lower triangular matrix,
−−=
=
111
012
001
1
01
001
3231
21
ff
fL
CheckCheck LULU == AA or not ?or not ?
After the second step of forward elimination
−−
−−⇒
320
520
123
′′′′=
3332
2322
131211
0
0
aa
aa
aaa
First step: R2+2R1
Second step: R3+R1
Third step: R3-R2
Solving Linear System by Solving Linear System by LULU DecompositionDecomposition
The linear system Ax = b can be solved in 3 steps:
(1) Find the LU decomposition of matrix A : A = LU
A x = b ⇒⇒⇒⇒ L U x = b (define Ux = y)
y
(2) Solve the system Ly = b :
=
3
2
1
3
2
1
333231
2221
11
b
b
b
y
y
y
LLL
0LL
00L
Forward Substitution
y1
y2
y3
(3) Solve the system Ux = y :
Backward Substitution
x1
x3
x2
=
3
2
1
3
2
1
33
2322
131211
y
y
y
x
x
x
U00
UU0
UUU
Solving Linear System by Solving Linear System by LULU DecompositionDecomposition
� LU Decomposition replaces the solving single Ax = b system
by solving two systems of Ly = b and Ux = y
� However, the latter systems are easy to solve since the
coefficient matrices are triangular.
Example:Example: Using Using LULU to solve equationsto solve equations
Ax = b → LUx = bLy = b
Ux = y
−−−
=
−−
−−
6
7
1
443
766
123
2
2
1
x
x
x
=
3
2
1
3
2
1
3231
21
1
01
001
b
b
b
y
y
y
ff
f
Step 2 : Ly = b : Forward Substitution
−−−
=
−−
6
7
1
111
012
001
3
2
1
y
y
y
11 by =
21122 fyby −=
32231133 fyfyby −−=
11 −=y
9)2)(1(72 −=−−−−=y
21)9()1)(1(63 =×−−−−−−=y
Step 1 : A = LU : Decomposition � done in previous example
Step 3 : Ux = y : Backward Substitution
=
′′′′
3
2
1
3
2
1
33
2322
131211
00
0
y
y
y
x
x
x
a
aa
aaa
−−
=
−−
−−
2
9
1
200
520
123
3
2
1
x
x
x
33
33 a
yx
′′=
22
23322 a
axyx
′′−
=
11
13312211 a
axaxyx
−−=
12
23 −=
−=x
22
)5)(1(92 =
−−−−
=x
23
)1)(1(2211 =
−−−−×−−
=x
MATLAB Function: MATLAB Function: lulu
>> [L, U] = lu(A)
−−−
=
−−
−−
6
7
1
443
766
123
2
2
1
x
x
x
>> A = [-3 2 -1;6 -6 7;3 -4 4];>> b = [-1; -7; -6];>> [L,U] = lu(A)
LU Decomposition:LU Decomposition:
>> y = L\b>> x = U\y
Solve equations:Solve equations:
2 3
2
3
For Gauss elimination with pivoting:
PA = LU P = identity matrix with same rows switched as A in pivoting.
For example PA : (switch R2 ↔ R3)
−−
−=
−−−
1252
1020
321
1020
1252
321
010
100
001
PAx = Pb ≡ b’ → LUx = b’Ly = b’
Ux = y
LULU Decomposition with pivotingDecomposition with pivoting
ALU =
−−−
=
−−
−=
1020
1252
321
200
610
321
120
012
001
>> A = [1 -2 3;2 -5 12;0 2 -10];
>> [L,U] = lu(A)
>> [L,U,P] = lu(A)
Example:Example: LULU decomposition with pivotingdecomposition with pivoting
From previous example:
P A = L U
001
100
010
−
−
−
1020
1252
321
=
−
−
−
321
1020
1252
R1�R3,
=
125.05.0
010
001
−
−
−
5.000
1020
1252
R2�R1, R3�R2
Next, we want to find the solution of the system: Ax = b
=
−
−
−
3
2
2
x
x
x
1020
1252
321
3
2
1
PAx = Pb ≡ b’ → LUx = b’Ly = b’
Ux = y
001
100
010
3
2
2
=
2
3
2 >> b_dash = P*b
>> y = L\b_dash
>> x = U\y
>> x = A\b
Compare with,
� Linear Fit
� Goodness of Fit
� Quadratic Fit
� Polynomial Fit
� MATLAB polyfit & polyval
Numerical Numerical Methods for Civil EngineersMethods for Civil Engineers
By Asst.Prof.Dr.Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Lecture Lecture 77 Curve FittingCurve Fitting
LINEAR REGRESSIONLINEAR REGRESSION
x
y
��ก��������� ���
y = αααα x + ββββ
- ��������ก������������������������������������� �!�����
��ก� �!����� [xi, yi] �����ก���������"#����������!�� �!�����
- �$��������#�%������������������ � ���� &$��� �!���������ก���������� �
Error Between Model and ObservationError Between Model and Observation
x
yData: [ xi yi ]
Error: ei = yi – α xi – β
Criteria for a “Best” Fit :
������������ �������� �%����$�&������������������� ΣΣΣΣei �������� �
Model: β+α= xy
y i
x i
y i
BESTBEST line with error minimized ?line with error minimized ?
������ก'���������������������� ei ���&�$��%��$��$������������( yi ��%�$������%��)��ก*����
yye ii ˆ−=
�����
βα += ixy �$�$��������������ก�$�%� ��������+�$�,�ก
��%�,"#��%��ก���ก�� �'����&(���ก����������(
�-������ก�����ก����ก���!����������� ±
�������(����������������������
�$�%�'�������./����ก���'���)����������������������� �
ERRORERROR DefinitionDefinition
yye ii ˆ−=
����+��#�(��������!�&����ก'��������
!�+�����ก����&����ก'������������� �����ก�$� least squareleast square
( )∑= 2ieS
LeastLeast --Square Fit of a Straight LineSquare Fit of a Straight Line
��ก� �!����� (xi, yi) ����%������������: β+α= ii xy
���������������: iii yye −= ii xy α−β−=
&�������������������ก'�����: ...eeeS 23
22
21r +++=
( )2ii
2ir xyeS α−β−Σ=Σ=
����������������������� �����ก�� -��12!������%�(�1(3��$�ก�,4���2
0Sr =α∂
∂
0Sr =β∂
∂
��ก��ก��!���� ����� x i ��% y i �����$����� �%����$�
0)x)(xy(2S
iiir =−β−α−Σ=
α∂∂
0)1)(xy(2S
iir =−β−α−Σ=
β∂∂
���������$�%����$�
iii2i yxxx Σ=Σβ+Σα
ii ynx Σ=β+Σα
����� n ����'����!�����
Σ
Σ=
β
α
Σ
ΣΣ
i
ii
i
i2i
y
yx
nx
xx
�����!����������ก�����(ก
�ก���ก����+��-����ก���-������$� αααα ��% ββββ
( )22
1
1
i i i i
i i
x y x yn
x xn
α∑ − ∑ ∑
=∑ − ∑
y xβ α= −
xyxy and of mean the are and where
1xy i i i iS x y x y
n= Σ − Σ ΣDefine:
( )22 1xx i iS x x
n= Σ − Σ
( )22 1yy i iS y y
n= Σ − Σ
y xα β= +
Approximated y for any x is
xy
xx
S
Sα =
Example: Fit a straight line to x and y values
xi yi
1234567
0.52.52.04.03.56.05.5 4286.3
7
24
47
28
7
==
==
=
y
x
n
2
(119.5) (28)(24) / 70.8393
(140) (28) / 7
3.4286 0.8393(4) 0.0714
α
β
−= =
−
= − =
xi2 xi yi
149
16253649
0.55.06.0
16.017.536.038.5
yi2
0.256.25
416
12.2536
30.25
Σ 28 24 140 119.5 105
Least-square fit:
0714.0x8393.0y +=
1 2 3 4 5 6 70
1
2
3
4
5
6
x
y
MATLAB:
>> x = 1:7;
>> y = [0.5 2.5 2.0 4.0 3.5 6.0 5.5];
>> yhat = 0.8393*x+0.0714;
>> plot(x,y,'o',x,yhat)
Goodness of FitGoodness of Fit
Sum of the square of the errors:
( )22
1 1
n n
r i i ii i
S e y xβ α= =
= = − −∑ ∑
Standard errors of the estimation: / 2r
y x
Ss
n=
−
Sum of the square around the mean: ( )2
1
n
t ii
S y y=
= −∑
Standard deviation:2
ty
Ss
n=
−
( )2 /xx yy xy xxS S S S= −
yyS=
Coefficient of determination 2 t r
t
S Sr
S
−=
For perfect fit Sr = 0 and r = r2 = 1
y
sy sy/x
Linear regressionsy > sy/x
2xy
xx yy
S
S S=
r2 ��������$��ก�������������$� y ����ก(���กก����������$� x
Example: Error analysis of the linear fit
xi yi
1234567
0.52.52.04.03.56.05.5
0.8393
0.0714
αβ
=
=
3.4286y =
/
22.7143
7 22.131
2.9911
7 20.773
y
y x
s
s
=−
=
=−
=
( yi - β - α xi)2
8.57650.86222.04080.32650.00516.61224.2908
0.16870.56260.34730.32650.58960.79720.1993
Σ 28 24 22.7143 2.9911
( )2
iy y−
St Sr
Since sy/x < sy , linear regression has merit.
22.7143 2.99110.868 0.932
22.7143r
−= = =
Linear model explains 86.8% of original uncertainty.
OR Example: Error analysis of the linear fit
xi yi
1234567
0.52.52.04.03.56.05.5
Σ 28 24
xi2
149
16253649
140
xi yi
0.55.06.0
16.017.536.038.5
119.5
yi2
0.256.25
416
12.2536
30.25
105
2140 28 / 7 28xxS = − =
2105 24 / 7 22.7yyS = − =
119.5 28 24 / 7 23.5xyS = − × =
2(28 22.7 23.5 ) / 28
2.977
rS = × −
=
/
22.72.131
7 2
2.9770.772
7 2
y
y x
s
s
= =−
= =−
Since sy/x < sy , linear regression has merit.
22 23.5
0.86928 22.7
r = =×
Linear model explains 86.9% of original uncertainty.
NonNon --linear Curve Fitlinear Curve Fit
��,�ก�)�� ���������$�������������� ก����� Linear regression
ก5�%��$�������ก�,����������(���� error �$��!����
0 1 2 3 4 5-10
0
10
20
30
40
50
60
70
ก����%��)��������*��������&������ก�$�
22i2i10i
2ir )xaxaay(eS −−−Σ=Σ=
...eeeS 23
22
21r +++=
Quadratic FitQuadratic Fit
��%��)� �!����� (xi, yi) ����*��ก'�����: 2i2i10i xaxaay ++=
���������������: iii yye −=
&�������������������ก'�����:
����������������������� �����ก�� -��12!������%�(�1(3��$�ก�,4���2
2i2i10i xaxaay −−−=
)xaxaay(2aS 2
i2i10i0
r −−−Σ−=∂∂
= 0
)xaxaay(x2aS 2
i2i10ii1
r −−−Σ−=∂∂
= 0
)xaxaay(x2aS 2
i2i10i2i
2
r −−−Σ−=∂∂
= 0
���������$�%����$�
i22i1i0 ya)x(a)x(an Σ=Σ+Σ+
ii23i1
2i0i yxa)x(a)x(a)x( Σ=Σ+Σ+Σ
i2i2
4i1
3i0
2i yxa)x(a)x(a)x( Σ=Σ+Σ+Σ
�����!����������ก�����(ก
Σ
Σ
Σ
=
ΣΣ
ΣΣΣ
ΣΣΣ
i
ii
i2i
0
1
2
i2i
i2i
3i
2i
3i
4i
y
yx
yx
a
a
a
nxx
xxx
xxx
�ก���ก�����(ก*����� Gaussian elimination �-������$������%�(�1(3 a0, a1 ��% a2
Example: Fit a second-order polynomial to the data.
xi yi
012345
2.17.7
13.627.240.961.1
Σ 15 152.6
( yi - a0 - a1xi - a2xi2)2
0.143321.002861.081580.804910.619510.09439
3.74657
544.44314.47140.03
3.12239.22
1272.11
2513.39
( )2
iy y−
4
2 2
3
2 15 979
6 152.6 585.6
2.5 55 585.6
25.433 225
i i
i i i
i i i
i
m x x
n y x y
x x x y
y x
= ∑ = ∑ =
= ∑ = ∑ =
= ∑ = ∑ =
= ∑ =
From the given data:
Simultaneous linear equations
0
1
2
6 15 25 152.6
15 55 225 585.6
55 225 979 2488.8
a
a
a
=
Solving these equation gives a0 = 2.47857,a1 = 2.35929, and
a2 = 1.86071.
Least-squares quadratic equation:
y = 2.47857 + 2.35929x + 1.86071x2
Coefficient of determination:
2513.39 3.746570.99851 0.99925
2513.39r
−= = =
Polynomial FitPolynomial Fit
��%��)� �!����� (xi, yi) *�����*-�(*��������ก�� m:
mim
2i2i10i xaxaxaay ++++= ⋯
�'����*��ก���ก��%,,��ก����(�����'���� m+1:
Σ
Σ
Σ
Σ
=
ΣΣΣΣ
ΣΣΣΣ
ΣΣΣΣ
ΣΣΣ
++
+
+
imi
i2i
ii
i
m
2
1
0
m2i
2mi
1mi
mi
2mi
4i
3i
2i
1mi
3i
2ii
mi
2ii
yx
yx
yx
y
a
a
a
a
xxxx
xxxx
xxxx
xxxn
⋮⋮
⋯
⋮⋱⋮⋮⋮
⋯
⋯
⋯
MATLAB MATLAB polyfitpolyfit FunctionFunction
For second-order polynomial, we can define
211 1
1222 2
2
32
-1
1
1, ,
1
and show that ( ' ) ' or =
mm m
yx xc
yx xc
cyx x
−
= = =
= 1
A Y C
C A A A Y C A Y
⋮⋮ ⋮ ⋮
>> C = polyfit(x,y,n)
>> [C,S] = polyfit(x,y,n)
x = independent variabley = dependent variablen = degree of polynomial
C = coeff. of polynomial indescending power
S = data structure for polyvalfunction
Fit norm Fit QR
Solving by MATLAB Solving by MATLAB polyfitpolyfit FunctionFunction
>> x = [0 1 2 3 4 5];
>> y = [2.1 7.7 13.6 27.2 40.9 61.1];
>> c = polyfit(x,y,2)
>> [c,s] = polyfit(x,y,2)
>> st = sum ((y-mean(y)).^2)
>> sr = sum((y-polyval(c,x)).^2)
>> r = sqrt((st-sr)/st)
MATLAB MATLAB polyvalpolyval FunctionFunction
Evaluate polynomial at the points defined by the input vector
Example: y = 1.86071x2 + 2.35929x + 2.47857
y = c(1)*x n + c(2)*x (n-1) +...+ c(n)*x + c(n+1)
>> y = polyval(c,x)
where x = Input vector
y = Value of polynomial evaluated at x
c = vector of coefficient in descending order
>> c = [1.86071 2.35929 2.47857]
0 1 2 3 4 50
10
20
30
40
50
60
70
x
y
Polynomial InterpolationPolynomial Interpolation
>> y2 = polyval(c,x)
>> plot(x,y,’o’,x,y2)
Example: Which order is the best polynomial fit?
xy
00
10.84
20.91
30.14
4-0.76
5-0.96
6-0.28
0 1 2 3 4 5 6-1
-0.5
0
0.5
1
1st order (linear) Fit :
>> x = [0 1 2 3 4 5 6];
>> y = [0 .84 .91 .14 -.76 -.96 -.28];
>> c1 = polyfit(x,y, 1);
>> st = sum ((y-mean(y)).^2);
>> sr = sum((y-polyval(c1,x)).^2);
>> r1 = sqrt((st-sr)/st)
r1 = 0.6528
>> y1 = polyval(c1,x);
>> plot(x,y,’o’,x,y1)
0 1 2 3 4 5 6-1
-0.5
0
0.5
1
3rd
0.9951
2nd
0.6695
Order
r
1st
0.6528
0 1 2 3 4 5 6-1
-0.5
0
0.5
1
Data1st2nd3rd
c3 = [0.0886 -0.8262 1.7262 -0.0362]
>> c2 = polyfit(x,y,2)
>> c3 = polyfit(x,y,3)
Exponential FitExponential Fit
0E+0
1E+5
2E+5
3E+5
4E+5
5E+5
6E+5
0 5 10 15 20Years
Pop
ulat
ion
��,�ก�)�� �!������������������ก6)%����� exponential function
����ก��7����'������%��ก�"#��-(��
��������$�� ก�8 �%��5��$����$� 15 �8
��ก��+�����ก��,���ก��$��5�ก���-(��!#+�
�������ก���+ ก����%��)!�������
��ก6)%��+�#�$��!����ก
Exponential FitExponential Fit
1
10
100
1000
10000
100000
1000000
0 10 20Years
Pop
ulat
ion
• 9�����-�5�!�������(�*����������ก� y ���� log ��ก�
• �%��5��$�ก���-(����%��ก��%������ก�������������$�������
• ����+�9��������!������������������ก$��ก5�%�����9��%��)������,, least-square �����������(�
Exponential FitExponential Fit
��%��)� �!����� (xi, yi) *�����7.ก2������ก"2*���������
ixbi eay =
���������$*����$7.ก2������กก��(�#�:��1�����(��+��!����ก��
ii xbalnyln +=
*�����$� ln yi !��$�%!����� ��������9�ก����$� a ��% b ���*������(1� least square
β+α= ii xz
����� ,ylnz ii = α = b, ��% β = ln a ���� a = eβ
Example: Exponential Fit
xy
0266
1162
298
370
439
521
612
0 1 2 3 4 5 60
50
100
150
200
250
300
x i
0123456
y i
2661629870392112
zi = ln y i
5.5845.0884.5854.2493.6643.0452.485
x i2
0149
162536
x izi
05.0889.17012.74614.65415.22314.909
zi2
31.17525.88421.02218.05013.4229.2696.175
ΣΣΣΣ 21 28.698 91 71.789 124.996
Least Square fit of x i and z i:
n = 7
3721
x ==
1.47698.28
z ==
2i
2i
iiii
)x(n1
x
zxn1
zx
Σ−Σ
ΣΣ−Σ=α 261.0
7/2191
7/698.2821789.782 −=
−
×−=
xz α−=β = 4.1 – (-0.261)(3) = 4.883
0 1 2 3 4 5 60
50
100
150
200
250
300
ixbi eay =
β+α= ii xz
�������������$�����
α = b, ��% a = eβ
= -0.261 xi + 4.883
����� = e4.883 = 132.026
ix261.0i e026.132y −=
>> xi=0:0.1:6;
>> yi=132.026*exp(-0.261*xi);
>> plot(x,y,'o',xi,yi)
Least Square fit of x i and z i: (by MATLAB)
>> clear
>> x = 0:6;
>> y = [266 162 98 70 39 21 12];
>> z = log(y);
>> c = polyfit(x,z,1);
>> b = c(1);
>> a = exp(c(2));
>> yhat = a*exp(b*x);
>> st = sum ((y-mean(y)).^2);
>> sr = sum((y-yhat).^2);
>> r = sqrt((st-sr)/st)
r = 0.9968
>> xi = 0:0.1:6;
>> yi = a*exp(b*xi);
>> plot(x,y,'o',xi,yi)
0 1 2 3 4 5 60
50
100
150
200
250
300
Lecture Lecture 88 InterpolationInterpolation
LinearLinear InterpolationInterpolation
QuadraticQuadratic InterpolationInterpolation
PolynomialPolynomial InterpolationInterpolation
PiecewisePiecewise Polynomial InterpolationPolynomial Interpolation
Numerical MethodsNumerical Methods for for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Visual InterpolationVisual Interpolation
Vehicle speed is approximately 49 km/h
0
20
4060
80
100
120
km/h
Interpolation between data points
known datay
x
Consider a set of xy data collected during an experiment.
We use interpolation technique to estimate y at x where there’s no data.
What is the corresponding value of y for this x ?
BASIC IDEASBASIC IDEAS
From the known data ( xi , yi ), interpolate
Determining coefficient a1, a2, . . . , an of basis function F(x)
F(x) = a1ΦΦΦΦ1(x) + a2ΦΦΦΦ2(x) + . . . + anΦΦΦΦn(x)
Polynomials are often used as the basis functions.
F(x) = a1 + a2 x + a3 x2 + . . . + an xn-1
ixxxFy ≠= ˆ for )ˆ(ˆ
nn
ii
ii
yx
yx
yx
yx
⋮⋮
⋮⋮
11
11
++
interpolateyx ˆˆ →
y
xx
y
F(x)
Interpolation vInterpolation v ..ss.. Curve FittingCurve Fitting
known datay
x
interpolationcurve fit
Curve fitting: fit function & data not exactly agree
Interpolation: function passes exactly through known data
Interpolation Interpolation && ExtrapolationExtrapolation
Interpolation approximate within the range of independent variable
of the given data set.
Extrapolation approximate outside the range of independent variable
of the given data set.
xx1 x2
y
Linear Interpolation
y
x
The most common way to estimate a data point between 2 points.
The function is estimated by a straight line drawn between them.
Interpolated Point
Linear, Quadratic, and Cubic Interpolations
Linear = 1st order
f(x) = a0 + a1x
2 points
Quadratic = 2nd order
f(x) = a0 + a1x + a2x2
3 points
Cubic = 3rd order
f(x) = a0 + a1x+ a2x2 + a3x3
4 points
General formula for an (General formula for an ( nn –– 1)th1)th --order polynomial:order polynomial:
f(x) = a0 + a1x+ a2x2 +…+ anxn-1
Linear InterpolationLinear Interpolation
Connect two data points with a straight line
x0 x1
y0
y1
x
yxF ˆ)ˆ( =
Using similar triangles:
01
01
0
0
ˆˆ
xxyy
xxyy
−
−=
−
−
)ˆ(ˆ)ˆ( 001
010 xx
xxyy
yyxF −−
−+==
QuadraticQuadratic InterpolationInterpolation
Second-order polynomial interpolation using 3 data points
(x0, y0)
(x1, y1)
(x2, y2)
Convenient form:
))(()()( 1020102 xxxxbxxbbxf −−+−+=
22102 )( xaxaaxf ++=
where 1020100 xxbxbba ++=
120211 xbxbba ++=
22 ba =00002 )( Substitute ybyxf =→=
01
011112 )( Substitute
xxyy
byxf−
−=→=
02
01
01
12
12
2222 xxxxyy
xxyy
b y)x(f Substitute−
−
−−
−
−
=→=
EXAMPLE : Linear & Quadratic Interpolation
Estimate ln (2) by using linear & quadratic interpolation
ln(1) = 0ln(2) = 0.6932ln(4) = 1.3863ln(6) = 1.7918
Linear interpolation from x1 = 1 to x2 = 6
3584.0)12(16
07918.10)2(1 =−
−
−+=f %3.48 =→ tε
Linear interpolation from x1 = 1 to x2 = 4
4621.0)12(14
03863.10)2(1 =−
−
−+=f %3.33 =→ tε
Quadratic interpolation from x1 = 1 to x2 = 4 and x3 = 6
05187.016
4621.046
3863.17918.1
b
4621.014
03863.1b ,0b
2
10
−=−
−−
−
=
=−
−==
Substitute b0, b1 and b2 into equation
f2(x) = 0 + 0.4621(x - 1) - 0.05187(x - 1)(x - 4)
which can be evaluated at x = 2 for
f2(2) = 0 + 0.4621(2 - 1) - 0.05187(2 - 1)(2 - 4)
f2(2) = 0.5658 → εt = 18.4%
f2(x) = – 0.05187x2 + 0.7251x – 0.6696
1 2 3 4 5 6 7
1
2
>> x = 1:0.01:7;
>> y = log(x);
>> y2= -0.05187*x.^2+ ...
0.7251*x-0.6696;
>> plot(x,y,x,y2)
MATLAB :
PolynomialPolynomial InterpolationInterpolation
In general, n+1 data points can be fitted by an nth-order
polynomial of the form
)xx()xx)(xx(b)xx(bb)x(f n10n010n −−−++−+= ⋯⋯
The following equations are used to evaluate the coefficients:
)x(fb 00 =
01
011 xx
)x(f)x(fb
−
−=
02
01
01
12
12
2 xxxx
)x(f)x(fxx
)x(f)x(f
b−
−
−−
−
−
=
]x[f 0=
]x,x[f 01=
]x,x,x[f 012=
⋮
]x,x,,x[fb 01nn …=
Finite Divided DifferenceFinite Divided Difference
The bracketed function evaluations are finite divided differences and are defined as
01
0101 xx
)x(f)x(f]x,x[f
−
−=
02
0112012 xx
]x,x[f]x,x[f]x,x,x[f
−
−=
⋮
0n
011n12n01n xx
]x,x,,x[f]x,x,,x[f]x,x,,x[f
−
−= − ……
…
1st finite divided difference
2nd finite divided difference
nth finite divided difference
These equations are recursive, i.e. higher order differences
are computed by taking differences of lower-order differences.
Divided Difference TableDivided Difference Table
The computations are organized in the divided-
difference table:
x i
x0
x1
x2
x3
f(x i)
f(x0)
f(x1)
f(x2)
f(x3)
First Second Third
f [x1,x0] f [x2,x1,x0] f [x3,x2,x1,x0]
f [x3,x2]
f [x2,x1] f [x3,x2,x1]
]x,x,,x[f)xx()xx)(xx(
]x,x,x[f)xx)(xx(]x,x[f)xx()x(f)x(f
01nn10
012100100n
…⋯⋯ −−−++
−−+−+=
Newton’s divided difference interpolating polynomial:
EXAMPLE : 3rd Order Polynomial Interpolation
Use the polynomial interpolation to estimate ln (2)
ln(1) = 0ln(2) = 0.6932ln(4) = 1.3863ln(5) = 1.6094ln(6) = 1.7918
x i
1
4
5
6
f(x i)
0
1.3863
1.6094
1.7918
First Second Third
0.4621 -0.0797 0.0133
0.1834
0.2231 -0.0397
f3(x) = 0 + 0.4621(x – 1) – 0.0797(x – 1)(x – 4) + 0.0133(x – 1)(x – 4)(x – 5)
For x = 2, f3(2) = 0.7013 εεεεt = 1.17%
Polynomial InterpolationPolynomial Interpolation
� Although the finite divided difference is well suited for
determining intermediate values between points, they
do not provide a polynomial in conventional form:
nx xaxaxaaxf ++++= ⋯
2210)(
� Since n+1 data points are required to determine n+1
coefficients, simultaneous linear systems of equations
can be used to calculate “a”s.
nnnnnn
nn
nn
xaxaxaaxf
xaxaxaaxf
xaxaxaaxf
+++=
+++=
+++=
⋯
⋮
⋯
⋯
2210
12121101
02020100
)(
)(
)(
Where “x”s are the knowns and “a”s are the unknowns.
VandermondeVandermonde Matrix EquationMatrix Equation
=
)x(f
)x(f
)x(f
a
a
a
xxx1
xxx1
xxx1
n
1
0
n
1
0
nn
2nn
n2
222
n1
211
⋮⋮
⋯
⋮⋮⋮⋮
⋯
⋯
Rewritten the equations in the matrix form:
Polynomial InterpolationPolynomial Interpolation
Finding Pn-1(x) of degree n-1 that passes through n known data pairs
Pn-1(x) = c1xn-1 + c2xn-2 + . . . + cn-1x + cn
Vandermonde Systems n pairs of (x, y)n equationsn unknowns
Polynomial pass through each of data points
Linear = 1 : 2 pt.Quadratic = 2 : 3 pt.Cubic = 3 : 4 pt.
Example: Construct a quadratic interpolating function
y = c1x2 + c2x + c3
that pass through (x, y) support points (-2, -2), (-1, 1), and (2, -1)
Substitute known points into equation:
-2 = c1 (-2)2 + c2 (-2) + c3
1 = c1 (-1)2 + c2 (-1) + c3
-1 = c1 (2)2 + c2 (2) + c3
Rewritten in matrix form:
1
2
3
4 2 1 2
1 1 1 1
2 2 1 1
c
c
c
− − − = −
VandermondeVandermonde MatrixMatrix
21 1 1 122 2 2 223 3 3 3
1
1
1
x x c y
x x c y
x x c y
=
>> x = [-2 -1 2]’;
>> A = [x.^2 x ones(size(x))];
or use the built-in vander function
>> A = vander([-2 -1 2]);
>> y = [-2 1 -1]’;
>> c = A\y
c =
-0.9167
0.2500
2.1667
MATLAB : MATLAB : polyfitpolyfit and and polyvalpolyval FunctionsFunctions
Find y = c1x2 + c2x + c3 that pass through (x, y) support points
(-2, -2), (-1, 1), and (2, -1).
>> x = [-2 -1 2];>> y = [-2 1 -1];>> c = polyfit(x,y,2)c =
-0.9167 0.2500 2.1667
We can then use the polyval function to perform an interpolation as in
Ex. To interpolate y at x = 1
>> polyval(c,1)
ans =
1.5000
Also, we can use the polyval function to plot the result as in
>> xhat = -3:0.1:3;
>> yhat = polyval(c,xhat);
>> plot(xhat,yhat,x,y,'o')
-3 -2 -1 0 1 2 3-8
-6
-4
-2
0
2
4
Polynomials WigglePolynomials Wiggle
2nd-order
3rd-order
4th-order
5th-order
y
x
xi 1 2 3 4 5 6 7 8 9 10
yi 3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0
MATLAB’s Command Lines to Demonstrate Polynomials Wi ggle
>> x = [1 2 3 4 5 6 7 8 9 10];
>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];
>> x0 = 1:0.1:10;
>> y2 = polyval(polyfit(x(4:6), y(4:6), 2), x0);
>> y3 = polyval(polyfit(x(4:7), y(4:7), 3), x0);
>> y4 = polyval(polyfit(x(3:7), y(3:7), 4), x0);
>> y5 = polyval(polyfit(x(3:8), y(3:8), 5), x0);
>> axis([0 10 -5 5])
>> plot(x, y, ‘o’)
>> hold on
>> plot(x0, y2)
>> plot(x0, y3)
>> plot(x0, y4)
>> plot(x0, y5)
Piecewise Polynomial Interpolation
Using a set of lower degree interpolants on subinterval of the whole domain
= breakpoint or knoty
x
Piecewise-linear interpolation
Piecewise-quadratic interpolation
Piecewise-cubic interpolation
f’(x) and f’’(x) continuous at breakpoint = cubic spline
MATLAB’s Built-in Interpolation Functions
Funnction Description
interp1 1-D interpolation with piecewise polynomials.
interp2 2-D interpolation with nearest neighbor, bilinear,or bicubic interpolants.
interp3 3-D interpolation with nearest neighbor, bilinear,
or bicubic interpolants.
interpft 1-D interpolation of uniformly spaced data using
Fourier Series (FFT).
interpn n-D extension of methods used by interp3.
spline 1-D interpolation with cubic-splines using
not-a-knot or fixed-slope end conditions.
interp1 Built-in Function
1-D interpolation with one of the following 4 methods:
1. Nearest-neighbor uses piecewise-constant function.
Interpolant discontinue at midpoint between knots
2. Linear interpolation uses piecewise-linear polynomials.
3. Cubic interpolation uses piecewise-cubic polynomials.
Interpolant and f’ (x) are continuous.
4. Spline interpolation uses cubic splines. This option performs
the same interpolation as built-in function spline .
How to use How to use interpinterp 11 ??
>> yhat = interp1(y, xhat)
>> yhat = interp1(x, y, xhat)
>> yhat = interp1(x, y, xhat, method)
where y = tabulated values to be interpolated.
x = independent values. If not given x=1:length(y).
xhat = values at which interpolant be evaluated.
method = ‘nearest’ , ‘linear’ , ‘cubic’ or ‘spline’
>> x = [1 2 3 4 5 6 7 8 9 10];
>> y = [3.5 3.0 2.5 2.0 1.5 -2.4 -2.8 -3.2 -3.6 -4.0];
>> xhat = 1:0.1:10; % eval interpolant at xhat
>> yn = interp1(x, y, xhat, ‘nearest’);
>> plot(x, y, ‘o’, xhat, yn); pause;
>> yl = interp1(x, y, xhat, ‘linear’);
>> plot(x, y, ‘o’, xhat, yl); pause;
>> yc = interp1(x, y, xhat, ‘cubic’);
>> plot(x, y, ‘o’, xhat, yc); pause;
>> ys = interp1(x, y, xhat, ‘spline’);
or >> ys = spline(x, y, xhat);
>> plot(x, y, ‘o’, xhat, ys);
Example: Interpolation with piecewise-polynomials
EXAMPLE : Linear interpolation by interp1 function
Estimate the value of y when x is equal to 3.5
x
y
0
15
1
10
2
9
3
6
4
2
5
0
>> x = 0:5;
>> y = [15 10 9 6 2 0];
>> interp1(x,y,3.5)
ans =
4
Plot Graph:
>> xhat = 0:0.2:5;
>> yhat = interp1(x,y,xhat);
>> plot(x,y,'bo-',xhat,yhat,'rx')
0 1 2 3 4 50
5
10
15
-- Basic IdeasBasic Ideas
-- Symbolic Symbolic vsvs.. Numerical IntegrationNumerical Integration
-- Trapezoid RuleTrapezoid Rule
-- SimpsonSimpson’’s Rules Rule
-- MATLAB quad and quadMATLAB quad and quad88 FunctionsFunctions
Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADETS U R A N A R E E INSTITUTE OF ENGINEERINGUNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Lecture Lecture 99 Numerical IntegrationNumerical Integration
BASICBASIC IDEASIDEAS
)( curveunder Area)( xfdxxfIb
a
== ∫
Approximated by piecewise-linear:
Area = Sum of trapezoidal regions
f (x)
a b
Trapezoidalregion
Node
x
Symbolic Integration with MATLAB
>> I = int(f) % Indefinite integral
>> I = int(f, v) % Designating integration variable
>> I = int(f, a, b) % Definite integral on close interval
>> I = int(f, v, a, b)where f = symbolic expression
Variables must be define as symbolic by sym or syms>> x = sym(‘x’), y = sym(‘y’), z = sym(‘z’)or >> syms x y z
Symbolic Math ToolboxSymbolic Math ToolboxSymbolic Math Toolboxes incorporate symbolic computation into the numericenvironment of MATLAB.
dxcxIb
a∫ −= )( :Example 3
>> syms x a b c>> I = int(x^3-c,x,a,b)
I =
1/4*b^4-c*b-1/4*a^4+c*a
What is Integration?What is Integration?
dxxfIb
a∫= )(
The integral is equivalent to the area under the curve.f(x)
xa b
I
Cross-sectional area of a river
Examples of how integration is used to evaluate areasExamples of how integration is used to evaluate areas
Net force due to wind blowing against the building
NewtonNewton--Cotes FormulasCotes FormulasReplace a complicated function with a polynomial that is easy to integrate
dxxfIb
a∫= )( dxxf
b
an∫≅ )(
nn
nnn xaxaxaaxf 1
110)( where +
− ++++= L
f(x)
xa b
Approximate area by a straight line
f(x)
xa b
Approximate area by a parabola
Trapezoidal RuleTrapezoidal Rule
xba
f(a)
f(b)
f(x)
f1(x)
1st degree polynomial = Linear line
)()()()()(1 axab
afbfafxf −−−
+=
2)()()()(1
bfafabdxxfIb
a
+−== ∫
Area of trapezoid = width x average height
width = b – a = h
average height = 2
)()( bfaf +
f1
f2
x1 x2
f(x)
x3 x4 x5
f3f4 f5
∫ =5
1
)(x
x
dxxf ∫ +2
1
)(x
x
dxxf ∫ +3
2
)(x
x
dxxf ∫ +4
3
)(x
x
dxxf ∫5
4
)(x
x
dxxf
Composite Trapezoidal RuleComposite Trapezoidal RuleImprove accuracy by dividing interval into subintervals
f(x)
x1 = a
x1 x2 x3 x4 x5 x6
xn = bh =
b – an
There are n segment with equal width:
h =b – a
n
dxxfdxxfdxxfIn
n
x
x
x
x
x
x∫∫∫
−
+++=1
3
2
2
1
)()()( L
2)()(
2)()(
2)()(
1
3221
nn xfxfh
xfxfhxfxfhI
++
++
++
=
−
L
22213221 nn ffhffhffhI +
+++
++
= −L
++= ∑
−
=n
n
ii fffhI
1
21 2
2
Substitute h =b – a
n
MEAN
f(x)
x1 x2 x3 x4 x5 x6
f1
f2 f3
f4 f5
f6
width average height
n
fffabI
n
n
ii
2
2)(
1
21
++
−=∑
−
=
Example 9-1 Use Trapezoidal rule to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=
from a = 0 to b = 0.8. Note that the exact value of the integral can be determinedanalytically to be 1.640533.
or using MATLAB’s Symbolic Math Toolbox :
>> x = sym(‘x’)>> I=int(0.2+25*x-200*x^2+675*x^3-900*x^4+400*x^5,x,0,0.8)
I = 3076/1875 = 1.6405
Single Trapeziod: f(0) = 0.2 and f(0.8) = 0.232
1728.02
232.02.0)08.0(
2)()()(
=+
−=
+−=
bfafabI
%5.89%1006405.1
1728.06405.1=×
−=tε
f(x)
0 0.8
Error
Integral estimate
2 Trapeziods: n = 2 (h = 0.4) :
f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232
n
fffabI
n
n
ii
2
2)(
1
21
++
−=∑
−
=
0688.14
232.0)456.2(22.0)8.0( =++
=I
%9.34%1006405.1
0688.16405.1=×
−=tε
Trapezoidal numerical integration
>> z = trapz(y) % Integral of y with unit spacing>> z = trapz(x, y) % Integral of y with respect to x
o
o
90
0
Example: sin x dx∫>> angle = 0:15:90;>> x = (pi*angle/180);>> y = sin(x);>> z = trapz(x,y)>> z =
0.9943
MATLABMATLAB’’ss trapztrapz functionfunction
Example 9-2 Use MATLAB’s trapz function to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=
from a = 0 to b = 0.8. Note that the exact value of the integral can be determinedanalytically to be 1.640533.
For n = 2 : >> x = linspace(0,0.8,3);>> fx = 0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;>> I = trapz(x,fx)I =1.0688
n
23456
I
1.06881.36391.48481.53991.5703
εt
34.916.59.496.134.28
Using higher-order polynomials to connect the points
f(x)
xSimpson’s 1/3 ruleQuadratic connecting 3 points
f(x)
xSimpson’s 3/8 ruleCubic connecting 4 points
SimpsonSimpson’’s Ruless Rules
f(x)
x0
Quadratic connecting 3 points
x2x1
f(x0)f(x2)
f(x1) f2(x)2 2
0 0
2( ) ( )x x
x x
I f x dx f x dx= ≅∫ ∫
1 22 0
0 1 0 2
0 21
1 0 1 2
0 12
2 0 2 1
( )( )( ) ( )( )( )( )( ) ( )
( )( )( )( ) ( )
( )( )
x x x xf x f xx x x xx x x x f x
x x x xx x x x f x
x x x x
− −=
− −− −
+− −
− −+
− −
[ ] 2 00 1 2( ) 4 ( ) ( ) ,
3 2x xhI f x f x f x h −
≅ + + =
SimpsonSimpson’’s 1/3 Ruless 1/3 Rules
Example 9-3 Use Simpson’s 1/3 rule to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=
from a = 0 to b = 0.8. Note that the exact value of the integral can be determinedanalytically to be 1.640533.
Solution: n = 2 (h = 0.4) :
f(0) = 0.2 f(0.4) = 2.456 f(0.8) = 0.232
3675.1)232.0)456.2(42.0(34.0
=++=I
%6.16%1006405.1
3675.16405.1=×
−=tε
Composite SimpsonComposite Simpson’’s 1/3 Ruless 1/3 Rules
number even, where)()()(2
4
2
2
0
=−
=+++= ∫∫∫−
nn
abhdxxfdxxfdxxfIn
n
x
x
x
x
x
x
L
[ ] [ ] [ ]nnn fffhfffhfffhI +++++++++= −− 12432210 43
43
43
L
+++
−= ∑ ∑
−
=
−
=
)()(2)(4)(3
)( 1
5,3,1
2
6,4,20 n
n
i
n
jji xfxfxfxf
nabI
14 2 4
24
2 4 2 41
1 4 1
1 4 1
1 4 1
4
1 4 1
1 1
a b
Example 9-4 Use composite Simpson’s 1/3 rule with n = 4 to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=
from a = 0 to b = 0.8. Note that the exact value of the integral can be determinedanalytically to be 1.640533.
Solution: n = 4 (h = 0.2) :
f(0) = 0.2 f(0.2) = 1.288 f(0.4) = 2.456 f(0.6) = 3.464 f(0.8) = 0.232
624.1)232.0)456.2(2)464.3288.1(42.0()4(3
8.0=++++=I
%04.1%1006405.1
624.16405.1=×
−=tε
+++
−= ∑ ∑
−
=
−
=
)()(2)(4)(3
)( 1
5,3,1
2
6,4,20 n
n
i
n
jji xfxfxfxf
nabI
2( ) ( )b b
a a
I f x dx f x dx= ≅∫ ∫f(x)
x
Cubic connecting 4 points
[ ]
( )0 1 2 3
3 0
3 ( ) 3 ( ) 3 ( ) ( )8
/ 3
hI f x f x f x f x
h x x
≅ + + +
= −
x0 x1 x2 x3a b
SimpsonSimpson’’s 3/8 Ruless 3/8 Rules
[ ])()(3)(3)(8
)(3210 xfxfxfxfabI +++
−=or…
31 ( )12
h f ξ′′
5 (4)1 ( )90
h f ξ
5 (4)3 ( )80
h f ξ
Trapezoid
Simpson’s 1/3
Simpson’s 3/8
Simpson’s 3/8 rule is used when number of segments is odd.
h b a= −
2b ah −
=
3b ah −
=
Method Et Et
Intervalwidth
3( ) ( )12
b a f ξ− ′′
5(4)( ) ( )
2880b a f ξ
−
5(4)( ) ( )
6480b a f ξ
−
Truncation ErrorsTruncation Errors
Apply Simpson’s 1/3 and 3/8 RulesTo handle multiple application with odd number of intervals
f(x)
x
1/3 rule 3/8 rule
Example 9-5 Use Simpson’s 1/3 + 3/8 rule with n = 5 to numerically integrate5432 400900675200252.0)( xxxxxxf +−+−+=
from a = 0 to b = 0.8. Note that the exact value of the integral can be determined analytically to be 1.640533.
f(x)
x
1/3 rule 3/8 rule
0 0.16 0.32 0.48 0.64 0.8
Solution: n = 5 (h = 0.16) :
f(0) = 0.2 f(0.16) = 1.297 f(0.32) = 1.743 f(0.48) = 3.186f(0.64) = 3.182 f(0.8) = 0.232
%274.0%1006405.1
645.16405.1=×
−=tε
For the first two segments use Simpson’s 1/3 :
380.0)743.1)297.1(42.0(316.0
=++=I
For the last three segments use Simpson’s 3/8 :
265.1
)232.0)182.3186.3(3743.1(848.0
=
+++=I
Total integral : I = 0.380 + 1.265 = 1.645
>> q = quad(’f',a,b)Approximates the integral of f(x) from a to b within a relativeerror of 1e-3 using an adaptive recursive Simpson's rule.
quad : low order method, quad8: high order method
’f' is a string containing the name of the function.
>> quad(‘sin’, 0, pi/2)
Function f must return a vector of output values if given a vector of input values.
MATLABMATLAB’’ss quadquad and and quad8quad8 FunctionsFunctions
0.82 3 4 5
0
Example: 0.2 25 200 675 900 400x x x x x dx+ − + − +∫
function p = poly5(x)p=0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;
poly5.m
>> quad(‘poly5’, 0, 0.8)
ans =
1.6405
-1-0.5
00.5
1
-1
-0.5
0
0.5
10
2
4
6
8
10
Example: Computing the Length of a Curve
[ ]
( ) sin(2 ), ( ) cos( ), ( )
where 0, 3
x t t y t t z t t
t π
= = =
∈
>> t = 0:0.1:3*pi;>> plot3(sin(2*t), cos(t), t)
Length of the curve:Norm of derivative
32 2
0
4cos(2 ) sin( ) 1t t dtπ
+ +∫
hcurve.m
function f = hcurve(t)f = sqrt(4*cos(2*t).^2 + sin(t).^2 + 1);
>> len = quad(‘hcurve’,0,3*pi)
len =17.2220
Example: Double Integrationmax max
min min
( , )y x
y x
f x y dx dy∫ ∫
>> xmin = pi; xmax = 2*pi;>> ymin = 0; ymax = pi;>> result = dblquad(‘integrnd’,xmin,xmax,ymin,ymax);result =
-9.8698
For example: f (x, y) = y sin(x) + x cos(y)integrnd.m
function out = integrnd(x,y)out = y*sin(x) + x*cos(y);
- Basic Ideas
- Euler’s Method
- Higher Order One-step Methods
- Predictor-Corrector Approach
- Runge-Kutta Methods
- Adaptive Stepsize Algorithms
Numerical Methods for Numerical Methods for CivilCivil EngineersEngineers
Lecture Lecture 1010 Ordinary Differential EquationsOrdinary Differential Equations
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Definitions and TerminologyDefinitions and Terminology
Differential equation (DE) : An equation contains the derivates
of one or more dependent variables with respect to one or more independent variables.
yx2.0dxdy
=Example:
Ordinary DE (ODE): An eq. contains only ordinary derivates
of one or more dependent variables with respect to a single independent variable.
Example: ,ey5dxdy x=+ yx2
dtdy
dtdx
+=+
( , )dy
f t ydt
=
where t = independent variable
y = dependent variable
Initial Condition y(t0) = y0
Rate of change of one variable with respect to another.
2
0d x dx
ma cv kx m c kxdt dt
+ + = + + =
Example: Mass-spring system m
k
c
x
Ordinary Differential EquationsOrdinary Differential Equations
Leonhard Euler 1707 - 1783
Leonhard Euler made a huge number of contributions to mathematics, almost half after he was totally blind.
Leohard Euler
It was Euler who originated the following notations:
e (base of natural log)
( )f x (function notation)
π (pi) i ( )1−
(summation)∑
New value = Old value + slope x step size
From Taylor series:2
00 0 0 0
( )( ) ( ) ( ) ( ) ( )
2!
t ty t y t t t y t y t
−′ ′′= + − + +⋯
Retaining only first derivative: 0 0 0( ) ( , )y t y h f t y= +
where h = (t - t0), y0 = y(t0), and f(t0, y0) = y’(t0)
t
y
t0 t1
h
Predict
Trueerror
1 0 0 0( , )y y h f t y= +
2 1 1 1( , )y y h f t y= +
1 1 1( , )i i i iy y h f t y− − −= +
EULEREULER’’S METHODS METHOD
Example: Manual Calculation with Euler’s Method
2 , (0) 1dy
t y ydt
= − = Exact solution:
( )212 1 5
4ty t e−= − +
Set step size: h = 0.2
f (ti-1, yi-1)
NA
0-2(1) = -2.0
0.2-2(0.6) = -1.0
0.4-2(0.4) = -0.4
ti
0.0
0.2
0.4
0.6
Euleryi = yi-1+ h f (ti-1, yi-1)
initial cond. = 1.0
1.0+0.2(-2.0) = 0.6
0.6+0.2(-1.0) = 0.4
0.4+0.2(-0.4) = 0.32
Exact
1.0000
0.6879
0.5117
0.4265
Error
0
-0.0879
-0.1117
-0.1065
f (t, y) = t – 2y
Implementing Euler’s Method
function [t,y] = odeEuler(diffeq,tn,h,y0)% Input: diffeq = (string) name of m-file that% evaluate right hand side of ODE% tn = stopping value of independent variable% h = stepsize% y0 = initial condition at t=0
t = (0:h:tn)’;n = length(t);y = y0*ones(n,1);
for i=2:ny(i) = y(i-1)+h*feval(diffeq,t(i-1),y(i-1));
end
odeEuler.m
Example: 2 , (0) 1, 0.2dy
t y y hdt
= − = =
function dydt = rhs1(t,y)dydt = t-2*y;
rhs1.m
>> [t,y] = odeEuler(‘rhs1’,0.6,0.2,1.0)
t =0
0.20000.40000.6000
y =1.00000.60000.40000.3200
>> y0 = (1/4)*(2*t-ones(size(t))+5*exp(-2*t))
y0 =
1.00000.68790.51170.4265
Exact Solution :
>> t0=0:0.01:0.6;>> y0 = (1/4)*(2*t0-ones(size(t0))+5*exp(-2*t0));>> plot(t0,y0,t,y)
HeunHeun ’’ss MethodMethod :: estimate slope from derivatives at the beginning and at the end of the interval.
Improvements of Improvements of EEuleruler’’ss MMethodethod
t
y
ti-1
slope:k1 = f(ti-1, yi-1)
ti
Euler:y0
i = yi-1 + hk1
h
slope:k2 = f(ti, y0
i)
t
y
ti-1 ti
h
2),(),(
2 slope Average
01121 iiii ytfytfkk +
=+
= −−
++= − 2
211
kkhyy ii
Use slope:(k1 + k2)/2
1 1 1
2 1 1 1
1 21
( , )
( , )
2
i i
i i
i i
k f t y
k f t h y hk
k ky y h
− −
− −
−
=
= + +
+= +
yi-1
ti-1
h
ti
True
Euler
Heun
HeunHeun’’ss MMethodethod
( , )dy
f t ydt
= 0iy
Heun’s Method:
1 1 1
2 1 1 1
1 21
( , )
( , )
2
i i
i i
i i
k f t y
k f t h y hk
k ky y h
− −
− −
−
=
= + +
+= +
1 1 1( , )i i i iy y h f t y− − −= +Euler’s Method:
01 1 1Predictor: ( , )i i i iy y h f t y− − −= +
01 1
1
( , ) ( , )Corrector:
2i i i i
i i
f t y f t yy y h − −
−
+= +
PredictorPredictor--Corrector ApproachCorrector Approach
Iterating the Corrector of Iterating the Corrector of HeunHeun’’ss MethodMethod
To improve the estimation
01 1
1
( , ) ( , )
2i i i i
i
f t y f t yy h − −−
++
iy
Example: Use Heun’s method to integrate
0.84 0.5 , (0) 2xy e y y′ = − =
From x = 0 to 4, step size = 1
Analytical solution: ( )0.8 0.5 0.542
1.3x x xy e e e− −= − +
01 1 1
0 01
Predictor, ( , )
2 1 (4 0.5(2)) 5
i i i iy y h f t y
y e
− − −= +
= + − =
01 1
1
0 0.8(1)
1
( , ) ( , )Corrector,
2(4 0.5(2)) (4 0.5(5))
2 1 6.70112
i i i ii i
f t y f t yy y h
e ey
− −−
+= +
− + −= + =
True value: ( )0.8(1) 0.5(1) 0.5(1)42 6.1946
1.3y e e e− −= − + =
6.1946 6.7011Relative error, 100% 8.18%
6.1946tE−
= × =
0 0.8(1)nd
1
(4 0.5(2)) (4 0.5(6.7011))2 Corrector, 2 1
26.2758, 1.31%t
e ey
E
− + −= +
= =
0 0.8(1)rd
1
(4 0.5(2)) (4 0.5(6.2758))3 Corrector, 2 1
26.3821, 3.03%t
e ey
E
− + −= +
= =
Error may increases for large step sizes.
Iteration using up-arrow in MATLAB
>> y = 2+1*(4*exp(0)-0.5*2) % Predictor of y1
y = 5
>> y = 2+(1/2)*((4*exp(0)-0.5*2)+(4*exp(0.8*1)-0.5*y) )
y = 6.7011 % 1st Corrector of y1
EnterPress and
y = 6.7011 % 2nd Corrector of y1
y = 6.2758 % 3rd Corrector of y1
y = 6.3821 % 4th Corrector of y1
y = 6.3555 % 5th Corrector of y1
y = 6.3609 % until no change
MATLAB’s Implementation
function dydx = func1(x,y)dydx = 4*exp(0.8*x)-0.5*y;
func1.m
function [x,y] = odeHeun(diffeq,xn,h,y0,iter)% Input: iter = number of corrector iterations
x = (0:h:xn)’;n = length(x);y = y0*ones(n,1);for i=2:n
y(i) = y(i-1)+h*feval(diffeq,x(i-1),y(i-1));for j=1:iter
y(i) = y(i-1)+h*(feval(diffeq,x(i-1),y(i-1)) ...
+ feval(diffeq,x(i),y(i)))/2;end
end
odeHeun.m
Et =0.002.683.093.173.18
y_true =2.00006.194614.843933.677275.3390
For x = 0 to 4, step size = 1,y(0) = 2, Iteration = 15
>> [x,y] = odeHeun(‘func1’,4,1,2,15)
x =01234
y =2.00006.360915.302234.743377.7351
Compare with Euler’s Method:
>> [x,y] = odeEuler(‘func1’,4,1,2)
x =01234
y =2.00005.000011.402225.513256.8493
Et =0.0019.2823.1924.2424.54
Comparison of True Solution with Euler’s and Heun’s Method
>> xtrue = (0:0.1:4)’;>> ytrue = (4/1.3)*(exp(0.8*xtrue)-exp(-0.5*xtrue)).. .
+ 2*exp(-0.5*xtrue);>> [xeuler,yeuler] = odeEuler(‘func1’,4,1,2);>> [xheun,yheun] = odeHeun(‘func1’,4,1,2,15);>> plot(xtrue,ytrue,xeuler,yeuler,’o’,xheun,yheun,’ +’)
0 1 2 3 40
20
40
60
80
x
y
True Euler Heun 15
11
m
i i l ll
y y kγ−=
= +∑General Formula using Weighted Average of slopes
For Heun’s method γ1 = γ2 = 0.5
The weights must satisfy1
1m
ll
γ=
=∑
Second-Order RK (Ralston’s) Method:
1 1 2
1 1 1
2 1 1 1
1 2
3 3
( , )
3 3,
4 4
i i
i i
i i
y y h k k
k f x y
k f x h y k h
−
− −
− −
= + +
=
= + +
RRUNGEUNGE--KUTTA (RK)KUTTA (RK) METHODSMETHODS
Third-Order Runge-Kutta Methods
( )1 1 2 346i i
hy y k k k−= + + +
where
( )
1 1 1
2 1 1 1
3 1 1 1 2
( , )
1 1,
2 2
, 2
i i
i i
i i
k f x y
k f x h y k h
k f x h y k h k h
− −
− −
− −
=
= + +
= + − +
Fouth-Order Runge-Kutta Methods
where
( )
1 1 1
2 1 1 1
3 1 1 2
4 1 1 3
( , )
1 1,
2 2
1 1,
2 2
,
i i
i i
i i
i i
k f x y
k f x h y k h
k f x h y k h
k f x h y k h
− −
− −
− −
− −
=
= + +
= + +
= + +
The most popular RK methods
( )1 1 2 3 42 26i i
hy y k k k k−= + + + +Classical 4th-order RK:
Example: Use 4th-order RK method to integrate
0.84 0.5 , (0) 2xy e y y′ = − =
From x = 0 to 4, step size = 1
Step 1:x0 = 0, y0 = 2
01
0.8(0.5)2
0.8(0.5)3
0.8(1)4
4 0.5(2) 3
14 0.5(2 3 1) 4.2173
21
4 0.5(2 4.2173 1) 3.91302
4 0.5(2 3.9130 1) 5.9457
k e
k e
k e
k e
= − =
= − + × × =
= − + × × =
= − + × =
( )1
12 3 2 4.2173 2 3.9130 5.9457
66.2011
y = + + × + × +
=
True value: ( )0.8(1) 0.5(1) 0.5(1)42 6.1946
1.3y e e e− −= − + =
6.1946 6.2010Relative error, 100% 0.1038%
6.1946tE−
= × =
MATLAB’s Implementation
function [x,y] = odeRK4(diffeq,xn,h,y0)
x = (0:h:xn)’;n = length(x);y = y0*ones(n,1);for i=2:n
k1 = feval(diffeq,x(i-1),y(i-1));k2 = feval(diffeq,x(i-1)+h/2,y(i-1)+k1*h/2);k3 = feval(diffeq,x(i-1)+h/2,y(i-1)+k2*h/2);k4 = feval(diffeq,x(i-1)+h,y(i-1)+k3*h);y(i) = y(i-1)+h/6*(k1+2*k2+2*k3+k4);
end
odeRK4.m
For x = 0 to 4, step size = 1, y(0) = 2
>> [x,y] = odeRK4(‘func1’,4,1,2)
x =01234
y =2.00006.201014.862533.721375.4392
y_true =2.00006.194614.843933.677275.3390
Et(%) =0.000.10380.12500.13090.1328
MATLAB’s ode23 and ode45 Routines
ode23 use second- and third-order RK simultaneously
ode45 use fourth- and fifth-order RK simultaneously
>> x0 = 0; xn = 4; y0 = 2;>> [x45,y45] = ode45(‘func1’,[x0,xn],y0);>> xtrue = (0:0.1:4)’;>> ytrue = (4/1.3)*(exp(0.8*xtrue)...
- exp(-0.5*xtrue))+ 2*exp(-0.5*xtrue);>> plot(xtrue,ytrue,x45,y45)
For x = 0 to 4,y(0) = 2
Example: 0.84 0.5 , (0) 2xy e y y′ = − = func1.m
Adaptive Adaptive StepsizeStepsize AlgorithmsAlgorithms
���� Basic Ideas
���� One-dimension- Golden-Section Search- Quadratic Interpolation- Newton’s Method
���� Multidimension- Direct Methods- Gradient Methods
NumericalNumerical Methods for Methods for CivilCivil EngineersEngineers
Mongkol JIRAVACHARADET
S U R A N A R E E INSTITUTE OF ENGINEERING
UNIVERSITY OF TECHNOLOGY SCHOOL OF CIVIL ENGINEERING
Lecture 11 Optimization
BASIC IDEAS
Root Root
Root
( ) 0f x =
Maximum
Minimum
( ) 0
( ) 0
f x
f x
′ =
′′ <
( ) 0
( ) 0
f x
f x
′ =
′′ >
0
f (x)
x
Maximum : Find x* such that f(x*) ≥≥≥≥ f(x) for all x
Minimum : Find x* such that f(x*) ≤≤≤≤ f(x) for all x
Examples of Optimization
• Minimize weight of structure subject to constraint on its
strength, or maximize its strength subject to constraint of
its weight
• Design vehicle for minimum energy consumption subject
to constraint on capacity and speed
• Design machine for maximum efficiency with less energy
consumption
• Optimal planning and scheduling of a construction project
• Budget or financial planning of any organization
0
f (x)
x
maximum
minimum
maximum
minimum
Local
LocalGlobal
Global
Global & Local Optimization
x* is global minimum if f(x*) ≤≤≤≤ f(x) for all x
x* is local minimum if f(x*) ≤≤≤≤ f(x) for all x near x*
Golden Section Search
� Similar to Bisection root finding method
� Define interval that contains one answer
� Need 3 points to detect maximum or minimum
� Pick 4th points then choose the first or last three points.
IntermediatePoint
x*xL xU
MAX
MIN
Continue the process to
narrow down the interval
Choice of Intermediate Points
0
f (x)
xx*
MAX
xL xU
L0
First iteration:L0 = L1 + L2
L2L1
- Define interval
- Pick 3 pts. from 4 pts.
Interval L0 is narrow down to L1
Choice of Intermediate Points
1 2
0 1
L L=
L LL2
Second iteration:
0
f (x)
xx*
MAX
xL
L0
First iteration:L0 = L1 + L2
L2L1
xU
- Move xU to new position
- Use same old 3 pts. sopick the new 4th pt.
From two condition, 1 2
1 2 1
L L
L L L=
+
Set R = L2 / L1,
211 or 1 0R R R
R+ = + − =
Solving for positive root,
1 1 4( 1) 5 10.61803
2 2R
− + − − −= = = …
R = 0.61803 = Golden ratio since it allows optima to be
found efficiently
The Parthenon in Athens
x
0.61803 x
This golden ratio were considered aesthetically pleasing
by the Greeks.
f (x1)f (x2)Eliminate
3) If f (x1) > f (x2),
Initial Step of the Golden-section Search
f (x)
x
Maximum
2) Choose two interior points x1 and x2
1) Guess initial bracket xL and xU
xL xUx1x2
dd
at the distance d
where )xx(2
15d LU −
−= )xx(618.0 LU −≈
x1 = xL + d
x2 = xU – d
eliminate [ xL, x2 ] and set x2 = xL for next step
Next Step to Complete the Algorithm
f (x)
x
Maximum
xUx2xL
Old x2 Old x1
4) Only new x1 need to be determined,
x1
new )xx(2
15d LU −
−=
x1 = xL + d
Example: Golden-Section Search to find maximum
2
( ) 2sin , 0 and 410 L U
xf x x x x= − = =
Solution: (1) Create two interior points
1
2
5 1(4 0) 2.472
20 2.472 2.472
4 2.472 1.528
d
x
x
−= − =
= + =
= − =
(2) Evaluate function at interior points,2
1
2
2.472( ) (2.472) 2sin(2.472) 0.63
10( ) (1.528) 1.765
f x f
f x f
= = − =
= =
xL x1 x2 xU
d d
(3) Because f(x2) > f(x1), eliminate upper part,
New xU = x1 = 2.472
New x1 = x2 = 1.528xL x2 x1 xU
OLD
xL x1 xU
NEW
x2
(4) Compute new x2
2
5 1(2.472 0) 1.528
22.472 1.528 0.944
d
x
−= − =
= − =
(5) Evaluate function at x2
1 2
2
New ( ) Old ( ) 1.765
New ( ) (0.944) 1.531
f x f x
f x f
= =
= =
Because f(x1) > f(x2), eliminate lower part, . . .
2 0 2.4721
4
5
6
7
8
0.9443
1.3050
1.3050
1.3050
1.3901
1.3050
1.5279
1.4427
1.3901
1.4427
1.7595
1.7647
1.7755
1.7742
1.7755
1.5279
1.6656
1.5279
1.4427
1.4752
1.7647
1.7136
1.7647
1.7755
1.7732
1.8885
1.8885
1.6656
1.5279
1.5279
0.5836
0.3607
0.2229
0.1378
0.0851
Tabulated results:
xL x2 f(x 2) x1 f(x 1) xU di
1.7647 0.6300
3 0.9443 2.4721
1 0 4.0000
1.5279 1.7647 1.8885 1.5432 0.9443
0.9443 1.5310 1.5279 1.7647 1.5279
1.5279 2.4721 2.4721
Quadratic Interpolation
2nd polynomial provides good approximation near optimum
xx0 x1 x3 x2
f (x) True maximum Quadraticapproximationof maximum
2 2 2 2 2 20 1 2 1 2 0 2 0 1
30 1 2 1 2 0 2 0 1
( )( ) ( )( ) ( )( )
2 ( )( ) 2 ( )( ) 2 ( )( )
f x x x f x x x f x x xx
f x x x f x x x f x x x
− + − + −=
− + − + −
Example: Quadratic Interpolation to find maximum
2
0 1 2( ) 2sin , 0, =1 and 410
xf x x x x x= − = =
Solution: (1) Evaluate function values,
0 0
1 1
2 2
0 ( ) 0
1 ( ) 1.5829
4 ( ) 3.1136
x f x
x f x
x f x
= =
= =
= = −
2 2 2 2 2 2
3
0(1 4 ) 1.5829(4 0 ) ( 3.1136)(0 1 )1.5055
2(0)(1 4) 2(1.5829)(4 0) 2( 3.1136)(0 1)x
− + − + − −= =
− + − + − −
(2) Because f(1.5055) = 1.7691, employ golden-section search,
eliminate lower guess x0 = 0,
0 0
1 1
2 2
1 ( ) 1.5829
1.5055 ( ) 1.7691
4 ( ) 3.1136
x f x
x f x
x f x
= =
= =
= = −
2 2 2 2 2 2
3
1.5829(1.5055 4 ) 1.7691(4 1 ) ( 3.1136)(1 1.5055 )
2(1.5829)(1.5055 4) 2(1.7691)(4 1) 2( 3.1136)(1 1.5055)
1.4903
x− + − + − −
=− + − + − −
=
After 5 iterations, x = 1.4276 gives maximum value of 1.7757
Newton’s Method
1
( )( ) 0
( )i
i ii
f xf x x x
f x+= → = −′
At optimal point *, ( *) ( *) 0x f x g x′ = =
Define new function, ( ) ( )g x f x′=
1 1
( ) ( )( ) 0
( ) ( )i i
i i i ii i
g x f xg x x x x x
g x f x+ +
′= → = − → = −
′ ′′
Example: Use Newton’s method to find maximum of
2
0( ) 2sin , Initial guess: 2.510
xf x x x= − =
Solution: (1) Evaluate 1st and 2nd derivatives of the function,
( ) 2cos51
( ) 2sin5
xf x x
f x x
′ = −
′′ = − −
(2) Substitute into Newton’s formula,
1
2cos / 5
2sin 1/ 5i i
i ii
x xx x
x+
−= −
− −
i x f(x)
01234
2.50.995081.469011.427641.42755
0.571941.578591.773851.775731.77573
(3) Substituting initial guess,
1
2cos 2.5 2.5 / 52.5 0.99508, (0.99508) 1.57859
2sin 2.5 1/ 5x f
−= − = =
− −
(4) Second iteration,
2
2cos0.995 0.995 / 50.995 1.46901, (1.46901) 1.77385
2sin 0.995 1/ 5x f
−= − = =
− −
After 4 iterations, result
converges rapidly to the
true value.
Multidimension Optimization
2D searches: Ascending a mountain (maximum) orDescending into a valley (minimization)
Example: 2-D Topographic Map of 3-D Mountain
x
y Line of constant f
GRADIENT METHODS
DIRECT METHODS
� Random Search
� Univariate and Pattern Searches
� Gradients and Hessians
� Steepest Ascent Method
Write an M-file objfun1.m
function f = objfun1(x,y)
f = y-x-2*x^2-2*x*y-y^2;
Random Search
Repeatedly evaluates function at randomly selected points
Example: 22 22),( yxyxxyyxf −−−−=
Domain boundaries: x = -2 to 2 and y = 1 to 3
>> rand % Uniformly distributed random numbers % on the interval (0.0,1.0).
ans =0.2311
Random search algorithm: rndsrch.m
function [maxx,maxy,maxf] = rndsrch(objfun,xmin,…xmax,ymin,ymax)
iter = 100;maxf = -inf;
for i = 1:iterx = xmin + (xmax - xmin)*rand;y = ymin + (ymax - ymin)*rand;fn = feval(objfun,x,y);if fn > maxf
maxf = fn;maxx = x;maxy = y;
endend
>>[maxx,maxy,maxf]=rndsrch('objfun1',-2,2,1,3)
100 -0.8050 1.3092 1.2120
200 -1.1409 1.7708 1.2133
500 -1.0006 1.4983 1.2500
1000 -1.0253 1.5030 1.2489
TRUE -1.0000 1.5000 1.2500
iter maxx maxy maxf
50 -0.9024 1.5913 1.2048
maxx = -1.0609 maxy = 1.7063 maxf = 1.2251
12
From (1) move along x axis with y constant to max at (2)
3
Univariate and Pattern Searches
- More efficient and still no require derivative evaluation
- Change one variable at time while others constant
y
x
4
Pattern directions
Powell’s Method
Use pattern directions to find optimum efficiently
y
x
1
2
- Pt. 1 and 2 are obtained by 1-Dsearch from different starting pts.
- Line formed by 1 and 2 will bedirected toward maximum
Conjugate direction
GRADIENT METHODS
Use derivative to locate optima
Gradient :
f∆h∆
y∆
x∆
Slope along axis f
hh
∆=
∆
f ff
x y
∂ ∂∇ = +
∂ ∂i j
The vector is called “delf “ = directional derivative of f (x , y)
= Steepest direction
To determine whether maximum, minimum or saddle
22 2 2
2 2
f f fH
x y x y
∂ ∂ ∂= − ∂ ∂ ∂ ∂
2
2
2
2
If 0 and 0 local minimum
If 0 and 0 local maximum
If 0 saddle point
fH
x
fH
xH
∂• > > →
∂
∂• > < →
∂
• < →
Hessian
SteepestSteepest AscentAscent
Steepest Ascent Method
Climbing a hill with maximum slope
y
x0 : Starting point (x0, y0)
1 Follow steepest ascent pathuntil f (x, y) stop increasing
Repeat the process untilthe maximum is reached
Determine: 1) The best direction of steepest ascent
2) The best value along that search direction
Example: Steepest ascent method
2( , ) , Starting point (2,2)f x y xy=
0 1 2 3 40
1
2
3
4
x
y
>> contour(x,y,fxy)
>> surf(x,y,fxy)
>> mesh(x,y,fxy)
x=0:0.1:4;y=0:0.1:4;fxy=zeros(length(y),length(x));for i=1:length(y)
for j=1:length(x)fxy(i,j)=x(j)*y(i)^2;
endend
M-file to create fxy: fxy.m
Determine elevation: 2(2,2) 2 2 8f = × =
Evaluate partial derivatives,
2 22 4
2 2(2)(2) 8
fy
xf
xyy
∂= = =
∂
∂= = =
∂
Determine the gradient, 4 8f∇ = +i j
Direction, 1 08tan 63.4
4θ − = =
relative to x axis
63.40
Magnitude of 2 2 is 4 8 8.944f∇ + =
We will gain 8.944 units of elevation rise for a unit distance
advanced along this steepest path.
63.4o
0 1 2 3 4x
0
1
2
3
4
y
Relationship between steepest direction and x-y coo rdinate
4 8f∇ = +i jy
x2
2
6
10
h =
0
h =
1
Starting at x0, y0 to any point x, y
in the gradient direction :
Then, we go along h axis to find a
maximum of f(x, y)
hyf
yy
hxf
xx
0
0
∂
∂+=
∂
∂+=
Define a new function g(h) = f(x, y) )hyf
y,hxf
x(f 00∂
∂+
∂
∂+
Find maximum by setting g’(h*) = 0 to find h*
Example: Developing 1-D function along a gradient direction
2 2( , ) 2 2 2 , Starting point ( 1,1)f x y xy x x y= + − − −
Partial derivative at starting point:
2 2 2 2(1) 2 2( 1) 6
2 4 2( 1) 4(1) 6
fy x
xf
x yy
∂= + − = + − − =
∂
∂= − = − − = −
∂
Function along h axis:
0 0
2 2
2
( , ) ( 1 6 ,1 6 )
2( 1 6 )(1 6 ) 2( 1 6 ) ( 1 6 ) 2(1 6 )
( ) 180 72 7
f ff x h y h f h h
x y
h h h h h
g h h h
∂ ∂+ + = − + −∂ ∂
= − + − + − + − − + − −
= − + −
Locate the maximum,
( *) 360 * 72 0
* 0.2
g h h
h
′ = − + =
=
Locate (x, y) coordinates according to h = 0.2,
1 6(0.2) 0.2
1 6(0.2) 0.2
x
y
= − + =
= − = −
Set new starting point to (0.2, -0.2) and repeat the process.
MATLAB Optimization Toolbox
fminunc : the unconstrained optimization function
2 2( , ) (4 2 4 2 1), Starting point ( 1,1)xf x y e x y xy y= + + + + −
Write an M-file objfun2.m
function f = objfun2(x,y)
f = exp(x)*(4x^2+2*y^2+4*x*y+2*y+1);
>> x0 = [-1,1]; % Starting point>> options = optimset(‘LargeScale’,’off’);>> [x,fval,flag,output]=fminunc(‘objfun2’,x0,options );
x =0.5000 -1.0000