EIGEN-VALUES, EIGEN-VECTORS QR FACTORIZATION (1) · PDF fileDominant Eigen-value • Let 1, 2, 3,, 𝑛 be the eigen-values of an A matrix with size 𝑛 𝑛. • If the eigen values

EIGEN-VALUES, EIGEN-VECTORS

QR FACTORIZATION (1)

ELM1222 Numerical Analysis

1

Some of the contents are adopted from

Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999

ELM1222 Numerical Analysis | Dr Muharrem Mercimek

Eigen-values, Eigen-vectors

Why do we need eigen-values, eigen-vectors?

• Eigen-values and eigen-vectors give us useful and important information

about a matrix ( a special matrix representing a system, or a data):

Some examples:

1. determine whether or not a matrix is positive definite(matrix A is positive

definite if and only if the eigenvalues of A are positive)

2. determine whether or not a matrix is invertible as well as to indicate how

sensitive the determination of the inverse will be to numerical errors.

3. provide important representation for matrices known as the eigenvalue

decomposition

2 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

Eigen-values, Eigen-vectors

• Eigenvalues of an matrix A are obtained by solving its

• characteristic equation

• 𝜆𝑛 + 𝑐𝑛−1𝜆𝑛−1 + 𝑐𝑛−2𝜆𝑛−2 + ⋯ + 𝑐1𝜆1 + 𝑐0 = 0

• For large values of n, polynomial equations like this one are difficult and time-

consuming to solve.

• Moreover, numerical techniques for approximating roots of polynomial

equations of high degree are sensitive to rounding errors.

• We need alternative methods for approximating eigenvalues

• As presented here, the method can be used only to find the eigenvalue of A

that is largest in absolute value —the dominant eigenvalue of A.

• Although this restriction may seem severe, dominant eigenvalues are of

primary interest in many physical applications.


Dominant Eigen-value

• Let 𝜆1, 𝜆2, 𝜆3,…, 𝜆𝑛 be the eigen-values of an A matrix with size 𝑛𝑥𝑛.

• If the eigen values in magnitude will be sorted like

𝜆1 < 𝜆2 < 𝜆3 < ⋯ < 𝜆𝑛 , 𝜆𝑛 is called the dominant eigen-value

Example 1:

Find the dominant eigen value of the following matrix.

2 −121 −5

The characteristic equation will be

(2 − 𝜆)(−5 − 𝜆) + 12 = 0

−10 + 3 𝜆 + 𝜆 2 + 12 = 0

𝜆 = {−1, −2}

The dominant one is -2 and corresponding eigen-vector is 𝑥 = 𝑡 3 1 𝑇 where 𝑡 ≠ 0


NUMERICAL TECHNIQUES FOR EIGEN-

VALUES, EIGEN-VECTOR APPROXIMATION


Power Method

• Accelerated Power Method

• Shifted Power Method

• Inverse Power Method

Power Method

• Like the Jacobi and Gauss-Seidel methods, the power method for

approximating eigenvalues is iterative.

• We chose an initial approximation of one of the dominant eigenvectors of A.

• This initial approximation can be a nonzero vector z

• We will have

w = Az

If z is an eigen-vector, then for any component we will have

λ zk = wk

If z is not an eigen-vector we will use w as the next approximation of z but

in the scaled form such that the largest component of z will be 1


Power Method

The iteration pattern will be as following;

w(1) = A z(1),

---

z(2) = w(1)

𝑤(1)𝑘

=A z(1)

𝑤(1)𝑘 w(2) = A z(2) = A

A z(1)

𝑤(1)𝑘

= A2 z(1)

𝑤(1)𝑘

,

z(3) =w(2)

𝑤(2)𝑘

=A z(2)

𝑤(2)𝑘

= A2 z(1)

𝑤(2)𝑘. 𝑤(1)

𝑘

w(3) = A z(3) = AA z(2)

𝑤(2)𝑘

= A3 z(1)

𝑤(2)𝑘. 𝑤(1)

𝑘

---

w(i) = A z(i), z(i) =w(i)

w(i)𝑘


Power Method

Example 2:

Given the A matrix find the dominant eigen-value and corresponding eigen-

vector using Power method


𝐳 = 1 1 1 𝑇 Initial eigen-vector

First Iteration

𝐰 = 𝐀𝐳 = 27 19 20 𝑇 𝐰𝟏 = 27

𝐳 = 𝐰/𝑤1 = 1.000 0.7037 0.7407 𝑇

Second Iteration

𝐰 = 𝐀𝐳 = 25.1852 15.1111 16.0000 𝑇 𝐰𝟏 = 25.1852

𝐳 = 𝐰/𝑤1 = 1.000 0.6000 0.6353 𝑇

Power Method


Third Iteration

𝐰 = 𝐀𝐳 = 24.5647 13.6471 14.3059 𝑇 𝐰𝟏 = 24.5647

𝐳 = 𝐰/𝑤1 = 1.000 0.5556 0.5824 𝑇

Fourth Iteration

𝐰 = 𝐀𝐳 = 24.3065 12.9655 13.4253 𝑇 𝐰𝟏 = 24.3065 𝐳 = 𝐰/𝑤1 = 1.000 0.5334 0.5523 𝑇

𝜆 ≅ 𝑤1, 𝜆 = 24.3065

𝐳 = 1.000 0.5334 0.5523 𝑇

𝐀𝐳 − λ𝐳 = −0.1249 −0.3653 −0.5123 𝑇

𝐀𝐳 − λ𝐳 ∞ = 0.5123

Accelerated Power Method

• In some cases, when A is symmetric power method with

• Rayleigh quotient converges more rapidly than classic power method.

• If x is an eigen-vector matrix A then its corresponding eigenvalue is given by

λ =𝑧𝑇Az

𝑧𝑇. z=

𝑧𝑇w

𝑧𝑇. z

• This is called Rayleigh quotient


Accelerated Power Method

Example 3:

Given the A matrix find the dominant eigen-value and corresponding eigen-

vector using Rayleigh quotient


Shifted Power Method

• If we already know an eigenvalue λ of a matrix A , we can find another

eigenvalue of A by applying the power method to the matrix B = A −λI .

• Denote the dominant eigenvalue of the shifted matrix B as μ

Example 4

If one eigen-value of the following matrix is 6 to find another eigen value apply

power method to the shifted matrix


Shifted Power Method


Initialize iterations with

𝐳 = 1 1 1 𝑇

Apply Rayleigh quotient approximation

First Iteration

𝐰 = 𝐁𝐳 = −3 −5 0 𝑇

𝜇 =𝐳T𝐰

𝐳T𝐳= −

3

8

w2 = −5 𝐳 = 𝐰/𝑤2 = 3/5 1 0 𝑇

Second Iteration

𝐰 = 𝐁𝐳 = −13/5 −21/5 0 𝑇

𝜇 =𝐳T𝐰

𝐳T𝐳= −

72

17

w2 = −21

5

𝐳 = 𝐰/𝑤2 = 13/21 1 0 𝑇

𝐀𝐳 − λ𝐳 𝟐 < 0.0001 we can use this as a stopping criterion

Inverse power method

• Provides an estimate of the eigenvalue of A that is of smallest magnitude

• Based on the fact that eigenvalues of B = A−1 are the reciprocals of the

eigenvalues of A.

• Thus, we apply the power method to B = A−1 to find its dominant eigenvalue

μ .

• Then, reciprocal of μ (i.e. 𝜆 = 1/𝜇) will give the smallest magnitude.

• It is not desirable to actually compute A−1 .instead where the power method

is normally A−1 z = w, we will use the form A w = z



Example 5: Given the following A matrix calculate the smallest eigen value

using inverse Power Method


Initialize iterations with

𝐳 = 1 1 1 𝑇

First Iteration

𝐀𝐰 = 𝐳 𝐰 = 0.0286 0.0651 0. 0573 𝑇

𝜇 =𝑤2

𝑧2= 0.0651

𝜆 =1

𝜇= 15.3601

w2 = −5 𝐳 = 𝐰/𝑤2 = 0.4400 1 0.8801 𝑇



Second Iteration

𝐀𝐰 = 𝐳 𝐰 = 0.0042 0.0842 0. 0617 𝑇

𝜇 =𝑤2

𝑧2= 0.0842

(𝑤2 is the largest component of w)

𝜆 =1

𝜇= 11.8777

𝐳 = 𝐰/𝑤2 = −0.0495 1 0.7324 𝑇

Third Iteration

𝐀𝐰 = 𝐳 𝐰 = −0.0336 0.1029 0. 0639 𝑇

𝜇 =𝑤2

𝑧2= 0.1029

𝜆 =1

𝜇= 9.7153

The largest eigen-value for 𝐀−1

The smallest eigen-value for A

Summary of Distance Metrics


Documents

EIGEN-VALUES, EIGEN-VECTORS QR FACTORIZATION (1) · PDF fileDominant Eigen-value • Let 1, 2, 3,, 𝑛 be the eigen-values of an A matrix with size 𝑛 𝑛. • If the eigen values