Computer Programmingusers.utcluj.ro/~jim/CP/Resources/Lectures/L12-13.pdf · 2014. 1. 10. · T.U. Cluj-Napoca - Computer Programming - lecture 12 - M. Joldoş 3 Computation of the

T.U. Cluj-Napoca - Computer Programming - lecture 12 - M. Joldoş 1

Computer Programming

“In programming, it is not enough to be inventive and ingenious. One also needs to be disciplined and controlled in order not be become entangled in one's own complexities.”

Harlan D. Mills, Forward to Programming Proverbs by Henry F. Ledgard


Outline

Sample programs in pseudocode Computation of the determinant and inverse of a

matrix

Approximation of a function by a polynomial of degree m using least squares method

Numerical integration of the equation y ’=f (x) with initial condition y (x0)=y0

Solving linear equations

For all samples: Problem definition

Math model

Algorithm development


Computation of the determinant and inverse of a matrix

Theorem: If a non-singular matrix, A, can be reduced to the unit matrix, I, by pre-multiplication with a string of matrices, then by pre-multiplication of the unit matrix, I, with the same matrix string will result in the matrix inverse, A-1

Consider the matrix, A

nnnn

n

n

aaa

aaa

aaa

A

...............

...

...

21

22221

11211



Finding the inverse: n steps

Each step involves a series of line operations

Intermediate matrices are obtained using pivot element elimination

After step 1 the intermediate matrix A(1)

nnn

n

n

aa

aaaa

A

)1(2

)1(

2)1(

22)1(

1)1(

12)1(

)1(

...0............

...0

...1



Elements of matrix A(1) were computed as

provided that a110 . In general, at step k

nj

niaaaa

a

aa

ijijij

jj

,1

,2for11)1()1(

11

11

)1(

nnk

nkk

nk

nk

nk

knk

kkk

kk

kk

kk

k

a

a

a

a

a

a

a

a

a

a

A

,)(

,)(

,3)(

,2)(

,1)(

1,)(

1,)(

1,3)(

1,2)(

1,1)(

)(

...

...

...

...

...

...

...

...

...

...

...

0...1...

000

...

...

...

...

...

...

...

0...0...

100

0...0...

010

0...0...

001



Elements of matrix A(k) were computed as

Thus, starting with B=I and applying the same ops we get B (n)=A-1. Matrices B (k) are calculated using:

nj

kiniabbba

bb

ikk

kjk

ijk

ijk

kkk

kjk

kjk

,1

,,1for)1()()1()(

)1(

)1()(

nj

kiniaaaaa

aa

ikk

kjk

ijk

ijk

kkk

kjk

kjk

,1

,,1for)1()()1()(

)1(

)1()(



The determinant of matrix A, |A| is: |A|=a11a

(1)22a

(2)33...a

(n-1)nn

because

1||...

1||

1||

...

||1

||1

||

||1

||

)1(33

)2(22

)1(

11

)1(

)1(

)(

22)1(

11

)1(

22)1(

)2(

11

)1(

A

aaaaA

aA

Aaa

Aa

A

Aa

A

nnk

n

nnk

n



To minimize rounding errors:

Before each step, k, bring to position k, by inter-changing, the line containing the element of absolute maximum value in column k, selected from lines numbered k, and above

By interchanging two lines, the determinant must be multiplied by -1



const maxn = 10;

type matrix= array[1..maxn, 1..maxn] of real;

procedure matrix_inverse(in n:integer; inout a: matrix,

out b:matrix; out det_a:real);

var i, j, k, maxpos: integer;

aux, max: real;

begin

{Initialize matrix B=I}

for i:=1 to n do

begin

for j:=1 to n do

begin

if (i=j) then b[i, j]=1;

else b[i,j]=0;

endif;

end;

end;

det_a:=1;

k:=1; {k is the step in computing matrices A(k) and B(k)}



while (k n) and (det_a 0) do begin

{find pivot line}

max:=|a[k, k]|;

maxpos:=k;

for i:=k+1 to n do

begin

if |a[i,k]| > max then

begin

max:=|a[i,k]|; maxpos:=i;

end;

endif;

end;

{interchange line k with maxpos in matrices A and B}

if (k maxpos) then begin for j:=1 to n do

begin

aux:=a[k, j];

a[k, j]:=a[maxpos, j];

a[maxpos,j]:=aux;

aux:=b[k, j];

b[k,j]:=b[maxpos,j];

b[maxpos, j]:=aux;

end;

det_a:=-det_a;

endif;


Computation of the determinant and inverse of a matrix if (a[k, k]=0) then det_a:=0; else begin det_a:=det_a*a[k, k]; aux:=a[k, k]; for j:=1 to n do begin a[k, j]:=a[k, j]/aux; b[k, j]:=b[k, j]/aux; end;

for i:=1 to n do begin aux:=a[i, k]; if (ik) then for j:=1 to n do begin a[i, j]:=a[i, j]-a[k, j]*aux; b[i, j]:=b[i, j]-b[k, j]*aux; end; endif; end; end; endif;

k:=k+1; end;

end.


Approximation of a function by a polynomial of degree m

Problem: a function is known by its values in n points,

Approximation by a polynomial of degree m required,

such as the sum of the squares of the errors is a minimum. i.e.

By replacing P (x) it yields

niyx ii ,1),,(

m

mxpxpxppxP ...)( 2

2

1

10

minimal)])([ 2

1

i

n

i

i yxP

n

i

i

m

imii yxpxpxppE1

22

2

1

10 minimal)...(



The following conditions result:

And, finally the matrix equation

0...;;0;010

mp

E

p

E

p

E

n

i

i

m

i

i

n

i

i

n

i

i

mn

i

m

i

n

i

m

i

n

i

m

i

n

i

m

i

n

i

i

n

i

i

n

i

m

i

n

i

i

n

i

i

n

i

m

i

n

i

i

yx

yx

y

p

p

p

x

x

x

x

x

x

x

x

x

x

x

n

1

1

1

1

0

1

2

1

1

1

1

2

1

3

1

2

1

1

1

2

1

1

1

......

...............

.........



Notation: A = matrix of coefficients

P = matrix of unknown parameters

B = right hand side

Rewritten:

AP=B

Hence

P=A-1B

As matrix inverse already has an algorithm, we have to: Generate matrix A and vector B

Multiply two matrices



program least_squares_method; const maxn = 11; type vector= array[1..maxn] of real; matrix= array[1..maxn, 1..maxn] of real; var x, y, P, B: vector; {x, y have n elements, P and B have m+1

elements} A, A_inv: matrix; {matrices of order (m+1)*(m+1)} n, m, m1, i: integer; det_A: real; begin read n, ((x[i], y[i]), i=1,n); {n is the number of points} read m; {m is the degree of the polynomial} m1:=m+1; call generate_A(n, x, y, m1, A); call generate_B(n, x, y, m1, B); call matrix_inverse(m1, A, A_inv, det_A); if (det_A0) then begin call matrix_product(m1, m1, A_inv, B, P); write (P[i], i=1,m1); end; else write ”Non-determination”; endif; end.



procedure matrix_product(in n, m, p: integer,; in A, B:matrix; out C:matrix); {compute product of matrices A (of size n*m) and B (of size

m*p) into C} var i, j, k: integer; s: real; begin for i:=1 to n do begin for j:=1 to p do begin s:=0; for k:=1 to m do begin s:=s+A[i, k] * B[k, j]; end; C[i, j]:=s; end; end; end;



procedure generate_A(in n: integer,; in x: vector; out A:matrix); var i, j, m: integer; s: vector; sum: real; begin s[1]:=n; m:=m1-1; for i:=1 to 2*m do begin sum:=0; for j:=1 to n do begin sum:=sum+x[j]^i; end; s[i+1]:=sum; end; {generate matrix A of size m1*m1} for i:=1 to m1 do begin for j:=1 to m1 do begin A[i, j]:=s[i+j-1]; end; end; end;

}...,,,{generate22 m

iii xxx



procedure generate_B(in n: integer,; in x, y: vector; out B:matrix); var i, j: integer; sum: real; begin sum:=0; for i:=1 to n do begin sum:=sum+y[i]; end; B[1]:=sum; {compute B[2],...,B[m+1]} m:=m1-1; for i:=1 to m do begin sum:=0; for j:=1 to n do begin sum:=sum+x[j]^i*y[i]; end; B[i+1]:=sum; end; end;

}]1[{calculate iyB


Numerical integration of the equation y ’=f (x)

Problem: integrate y ’=f (x)

Solution: for x[x0, xr] with step h : find n values, y1, y2,..., yn, which approximate the true values on points x1, x2,..., xn, where xi =x0+i *h of the function y ’=f (x) which includes the initial point (x0, y0)

Euler’s method based on Taylor’s approximation:

From which the first two terms are retained

i.e.

)(')()( 1 iii xyhxyxy

...)(''!2

)(')()(2

1 iiii xyh

xyhxyxy

),(1 iiii yxfhyy



const maxn=100; type vector=array[0..maxn] of real; function f(in x, y: real); begin f:=...; {expression of f} end; procedure Euler1(in n:integer; in x0, y0, xf:real; out x, y: vector); var h: real; i: integer; begin h:=(xf - x0) / n; x[0]:=x0; y[0]:=y0; for i:=1 to n do begin x[i]:=x[i-1]+h; y[i]=h*f(x[i-1], y[i-1]); end; end;

For an acceptable precision, step h must be sufficiently small



A better approximation results if first three terms are taken.

Thus we have procedure Euler2(in n:integer; in x0, y0, xf:real;

out x,y: vector);

var h, fi: real;

i: integer;

begin

h:=(xf - x0) / n;

x[0]:=x0;

y[0]:=y0;

for i:=1 to n do

begin

x[i]:=x[i-1]+h;

fi:=f(x[i-1], y[i-1]);

y[i]=y[i-1]+h/2*[fi+f(x[i], y[i-1]+h*fi);

end;

end;

))],(,(),([2

11 iiiiiiii yxfhyxfyxfh

yy



The predictor-corrector method combines the two methods as is computes the predicted value

from the first two terms of Taylor’s series and a corrected value

calculated from Taylor’s series from the first three terms The last formula is applied repeatedly till a predefined precision,

(which we will call eps), is met.

At each repetition, yci+1 becomes the predicted value.

A maximum number of repetitions will be denoted by max_rep

),(1 iii

p

i yxfhyy

),(),(2

111

p

iiiii

c

i yxfyxfh

yy



procedure Euler3(in max_rep: integer, in x0, y0, xf, eps: real; out x, y: vector); var i, nb_rep, j: integer; h, fi, ypred, ycorr, dif: real; go_on: Boolean; begin h:=(xf-x0) / n; x[0]:=x0; y[0]:=y0; go_on:=true; while ((i<n) and go_on) do begin x[i]:=x[i-1]+h; fi:=f(x[i-1], y[i-1]); ypred:=y[i-1]+h*fi; nb_rep:=0; repeat nb_rep:=nb_rep+1; ycorr:=y[i-1]+h/2*[fi+f(x[i], ypred)]; dif:=|ycorr-ypred|; ypred:=ycorr; until (dif <eps) or (nb_rep > max_rep); if (nb_rep > max_rep) then begin go_on:=false; write ”No convergence within the given number of

iterations”; end else begin y[i]:=ycorr; i:=i+1; end endif end; end;


Numerical integration of the equation y ’=f (x). Runge-Kutta

Another method: proposed by Runge and Kutta

No details. Just formulas for a precision of the order h4:

where

43211 226

1kkkkyy ii

),(1 ii yxfhk

)2

,2

( 23

ky

hxfhk ii

)2

,2

( 12

ky

hxfhk ii

),( 34 kyhxfhk ii


Numerical integration of the equation y ’=f (x). Runge-Kutta

procedure Runge_Kutta(in n: integer; in x0, y0, xf: real;

out x, y: vector);

var h, k1, k2, k3, k4: real;

i: integer;

begin

h:= (xf - x0)/ n;

x[0]=x0;

y[0]:=y0;

for i:=0 to n-1 do

begin

x[i+1]:=x[i]+h;

k1:=h*f(x[i], y[i]);

k2:=h*f(x[i]+h/2, y[i]+k1/2);

k3:=h*f(x[i]+h/2, y[i]+k2/2);

k4:=h*f(x[i]+h,y[i]+k3);

y[i+1]:=y[i]+1/6*(k1+2*k2+2*k3+k4);

end;

end.


Sample programs. Solving linear equations

Problem: solve a system of n linear equations with n variables (1)

Method used: Gauss, i.e. use linear transformations to obtain the row echelon (upper triangular) form (2)

(1),11

nibxa i

n

j

jij

1,1and,2for0'where

(2),1''1

ijnia

nibxa

ij

i

n

j

jij



The equivalent system (2) is solved as:

(3)1,1for

'

''

'

'

1

ni

a

xab

x

a

bx

ii

n

ij

jiji

i

nn

nn



The algorithms is: begin

*read input data

*bring equation system (1) to the row echelon

(triangle) form (2)

if system is compatible and determinate

then

begin

*solve system using formulas (3)

*display solution

end

endif

end.



Elementary row operations (also known as Gaussian operations) are simple manipulations performed on a linear system that have no effect on the system's solution set. The elementary row operations are: swapping: an equation is swapped with another

rescaling: both sides of an equation are multiplied by a nonzero constant

pivoting: an equation is replaced by the sum of itself and a multiple of another



Bringing equation system (1) to the row echelon (triangle) form (2) can be done as follows: Consider lines to be pivot lines by bringing in

position p (by performing interchanges) the line containing the maximum absolute value element |a[p, p]|

Eliminate unknown variable x[p] from equations by multiplying the pivot line p by constant

and adding that line to line i

1,1 np

1,1 npi

(4)1for],[

],[, npi

ppa

piac



Then we have: *bring equation system (1) to the triangle form (2) is

for p:=1 to n-1 do

begin

*build pivot line, p;

for i:=p+1 to n do

begin

for j:=p to n do

begin

a[i, j]:=a[i, j]+c*a[p, j];

end

end

b[i]:=b[i]+c*b[p];

end

end.

;p]a[p,

p]a[i,c



The nonstandard pivot line building

*build pivot line, p is

max:=|a[p, p]|;

q:=p;

for i:=p+1 to n do

begin

if |a[i, p]| > max then begin

max:=a[i, p]; q:=i;

end

endif

end

{interchange of lines p and q if that is the case}

if p q then begin

for j:=p to n do

begin

t:=a[p, j]; a[p, j]:=a[q, j]; a[q, j]:=t;

end

endif



The non-standard statement for solving: *solve system using formulas (3)is

for i:=n-1 downto 1 do

begin

s:=0;

for j:=i+1 to n do

begin

s:=s+a[i, j]*x[j];

end

end

;n]a[n,

b[n]x[n]

;i]a[i,

s-b[i]x[i]



Modules of the algorithm – main program: program Gauss;

{solve a n linear equation system, ax=b, with n unknown values}

const maxn=10;

type vector=array[1..maxn] of real;

matrix=array[1..maxn, 1..maxn] of real;

var i, j, n, error: integer;

a: matrix;

b, x: vector; begin read n, (((a[i, j], j=1,n), b[i]), i=1,n); call row_echelon(n, a, b, error); {error=0 for a uniquely

determined system, and 1 otherwise} if error=0 then begin call solve(n, a, b,x); write(x[i], i=1,n); {solution}

end else write ”undetermined or impossible system: determinant is zero.” endif end.


Sample programs. Solving linear equations Modules of the algorithm – procedure row_echelon procedure row_echelon(in n:integer; inout a: matrix; inout b: vector; out error: integer); var p, q, i, j: integer; c, max, t: real; begin p:=1; repeat {build pivot line, p}

max:=|a[p, p]|; q:=p;{q contains maximum absolute value in column p} for i:=p+1 to n do begin if |a[i, p]| > max then begin max:=a[i, p]; q:=i; end endif end if p q then begin {interchange of lines p and q} for j:=p to n do begin t:=a[p, j]; a[p, j]:=a[q, j]; a[q, j]:=t;

end t:=b[p]; b[p]:=b[q]; b[q]:=t; end endif


Sample programs. Solving linear equations Modules of the algorithm – procedure row_echelon (cont’d) {triangle form}

error:=0; if (a[p, p]=0 then error:=1 else

for i:=p+1 to n do

begin

for j:=p to n do

begin

a[i, j]:=a[i, j]+c*a[p, j];

end

end

b[i]:=b[i]+c*b[p];

endif

p:=p+1;

until ((p=n) or (error=1)

if (a[n, n]=0 then error:=1; endif

end;



Modules of the algorithm – procedure solve

procedure solve(in n: integer; in a: matrix; in b: vector; out x:

vector);

var i, j: integer;

s: real;

begin

for i:=n-1 downto 1 do

begin

s:=0;

for j:=i+1 to n do

begin

s:=s+a[i, j]*x[j];

end

end

end.

;n]a[n,

b[n]x[n]

;i]a[i,

s-b[i]x[i]


Summary

Sample programs in pseudocode Computation of the determinant and inverse of a

matrix

Approximation of a function by a polynomial of degree m using least squares method

Numerical integration of the equation y ’=f (x) with initial condition y (x0)=y0

Solving linear equations

For all samples: Problem definition

Math model

Algorithm development

Documents

Computer Programmingusers.utcluj.ro/~jim/CP/Resources/Lectures/L12-13.pdf · 2014. 1. 10. · T.U. Cluj-Napoca - Computer Programming - lecture 12 - M. Joldoş 3 Computation of the