EIGEN-VALUES, EIGEN-VECTORS
QR FACTORIZATION (1)
ELM1222 Numerical Analysis
1
Some of the contents are adopted from
Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999
ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Eigen-values, Eigen-vectors
Why do we need eigen-values, eigen-vectors?
โข Eigen-values and eigen-vectors give us useful and important information
about a matrix ( a special matrix representing a system, or a data):
Some examples:
1. determine whether or not a matrix is positive definite(matrix A is positive
definite if and only if the eigenvalues of A are positive)
2. determine whether or not a matrix is invertible as well as to indicate how
sensitive the determination of the inverse will be to numerical errors.
3. provide important representation for matrices known as the eigenvalue
decomposition
2 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Eigen-values, Eigen-vectors
โข Eigenvalues of an matrix A are obtained by solving its
โข characteristic equation
โข ๐๐ + ๐๐โ1๐๐โ1 + ๐๐โ2๐๐โ2 + โฏ + ๐1๐1 + ๐0 = 0
โข For large values of n, polynomial equations like this one are difficult and time-
consuming to solve.
โข Moreover, numerical techniques for approximating roots of polynomial
equations of high degree are sensitive to rounding errors.
โข We need alternative methods for approximating eigenvalues
โข As presented here, the method can be used only to find the eigenvalue of A
that is largest in absolute value โthe dominant eigenvalue of A.
โข Although this restriction may seem severe, dominant eigenvalues are of
primary interest in many physical applications.
3 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Dominant Eigen-value
โข Let ๐1, ๐2, ๐3,โฆ, ๐๐ be the eigen-values of an A matrix with size ๐๐ฅ๐.
โข If the eigen values in magnitude will be sorted like
๐1 < ๐2 < ๐3 < โฏ < ๐๐ , ๐๐ is called the dominant eigen-value
Example 1:
Find the dominant eigen value of the following matrix.
2 โ121 โ5
The characteristic equation will be
(2 โ ๐)(โ5 โ ๐) + 12 = 0
โ10 + 3 ๐ + ๐ 2 + 12 = 0
๐ = {โ1, โ2}
The dominant one is -2 and corresponding eigen-vector is ๐ฅ = ๐ก 3 1 ๐ where ๐ก โ 0
4 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
NUMERICAL TECHNIQUES FOR EIGEN-
VALUES, EIGEN-VECTOR APPROXIMATION
5 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
โข Accelerated Power Method
โข Shifted Power Method
โข Inverse Power Method
Power Method
โข Like the Jacobi and Gauss-Seidel methods, the power method for
approximating eigenvalues is iterative.
โข We chose an initial approximation of one of the dominant eigenvectors of A.
โข This initial approximation can be a nonzero vector z
โข We will have
w = Az
If z is an eigen-vector, then for any component we will have
ฮป zk = wk
If z is not an eigen-vector we will use w as the next approximation of z but
in the scaled form such that the largest component of z will be 1
6 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
The iteration pattern will be as following;
w(1) = A z(1),
---
z(2) = w(1)
๐ค(1)๐
=A z(1)
๐ค(1)๐ w(2) = A z(2) = A
A z(1)
๐ค(1)๐
= A2 z(1)
๐ค(1)๐
,
z(3) =w(2)
๐ค(2)๐
=A z(2)
๐ค(2)๐
= A2 z(1)
๐ค(2)๐. ๐ค(1)
๐
w(3) = A z(3) = AA z(2)
๐ค(2)๐
= A3 z(1)
๐ค(2)๐. ๐ค(1)
๐
---
w(i) = A z(i), z(i) =w(i)
w(i)๐
7 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
Example 2:
Given the A matrix find the dominant eigen-value and corresponding eigen-
vector using Power method
8 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
๐ณ = 1 1 1 ๐ Initial eigen-vector
First Iteration
๐ฐ = ๐๐ณ = 27 19 20 ๐ ๐ฐ๐ = 27
๐ณ = ๐ฐ/๐ค1 = 1.000 0.7037 0.7407 ๐
Second Iteration
๐ฐ = ๐๐ณ = 25.1852 15.1111 16.0000 ๐ ๐ฐ๐ = 25.1852
๐ณ = ๐ฐ/๐ค1 = 1.000 0.6000 0.6353 ๐
Power Method
9 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Third Iteration
๐ฐ = ๐๐ณ = 24.5647 13.6471 14.3059 ๐ ๐ฐ๐ = 24.5647
๐ณ = ๐ฐ/๐ค1 = 1.000 0.5556 0.5824 ๐
Fourth Iteration
๐ฐ = ๐๐ณ = 24.3065 12.9655 13.4253 ๐ ๐ฐ๐ = 24.3065 ๐ณ = ๐ฐ/๐ค1 = 1.000 0.5334 0.5523 ๐
๐ โ ๐ค1, ๐ = 24.3065
๐ณ = 1.000 0.5334 0.5523 ๐
๐๐ณ โ ฮป๐ณ = โ0.1249 โ0.3653 โ0.5123 ๐
๐๐ณ โ ฮป๐ณ โ = 0.5123
Accelerated Power Method
โข In some cases, when A is symmetric power method with
โข Rayleigh quotient converges more rapidly than classic power method.
โข If x is an eigen-vector matrix A then its corresponding eigenvalue is given by
ฮป =๐ง๐Az
๐ง๐. z=
๐ง๐w
๐ง๐. z
โข This is called Rayleigh quotient
10 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Accelerated Power Method
Example 3:
Given the A matrix find the dominant eigen-value and corresponding eigen-
vector using Rayleigh quotient
11 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Shifted Power Method
โข If we already know an eigenvalue ฮป of a matrix A , we can find another
eigenvalue of A by applying the power method to the matrix B = A โฮปI .
โข Denote the dominant eigenvalue of the shifted matrix B as ฮผ
Example 4
If one eigen-value of the following matrix is 6 to find another eigen value apply
power method to the shifted matrix
12 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Shifted Power Method
13 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Initialize iterations with
๐ณ = 1 1 1 ๐
Apply Rayleigh quotient approximation
First Iteration
๐ฐ = ๐๐ณ = โ3 โ5 0 ๐
๐ =๐ณT๐ฐ
๐ณT๐ณ= โ
3
8
w2 = โ5 ๐ณ = ๐ฐ/๐ค2 = 3/5 1 0 ๐
Second Iteration
๐ฐ = ๐๐ณ = โ13/5 โ21/5 0 ๐
๐ =๐ณT๐ฐ
๐ณT๐ณ= โ
72
17
w2 = โ21
5
๐ณ = ๐ฐ/๐ค2 = 13/21 1 0 ๐
๐๐ณ โ ฮป๐ณ ๐ < 0.0001 we can use this as a stopping criterion
Inverse power method
โข Provides an estimate of the eigenvalue of A that is of smallest magnitude
โข Based on the fact that eigenvalues of B = Aโ1 are the reciprocals of the
eigenvalues of A.
โข Thus, we apply the power method to B = Aโ1 to find its dominant eigenvalue
ฮผ .
โข Then, reciprocal of ฮผ (i.e. ๐ = 1/๐) will give the smallest magnitude.
โข It is not desirable to actually compute Aโ1 .instead where the power method
is normally Aโ1 z = w, we will use the form A w = z
14 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Inverse power method
Example 5: Given the following A matrix calculate the smallest eigen value
using inverse Power Method
15 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Initialize iterations with
๐ณ = 1 1 1 ๐
First Iteration
๐๐ฐ = ๐ณ ๐ฐ = 0.0286 0.0651 0. 0573 ๐
๐ =๐ค2
๐ง2= 0.0651
๐ =1
๐= 15.3601
w2 = โ5 ๐ณ = ๐ฐ/๐ค2 = 0.4400 1 0.8801 ๐
Inverse power method
16 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Second Iteration
๐๐ฐ = ๐ณ ๐ฐ = 0.0042 0.0842 0. 0617 ๐
๐ =๐ค2
๐ง2= 0.0842
(๐ค2 is the largest component of w)
๐ =1
๐= 11.8777
๐ณ = ๐ฐ/๐ค2 = โ0.0495 1 0.7324 ๐
Third Iteration
๐๐ฐ = ๐ณ ๐ฐ = โ0.0336 0.1029 0. 0639 ๐
๐ =๐ค2
๐ง2= 0.1029
๐ =1
๐= 9.7153
The largest eigen-value for ๐โ1
The smallest eigen-value for A
Summary of Distance Metrics
17 ELM1222 Numerical Analysis | Dr Muharrem Mercimek