§9.2 Orthogonal Matrices and Similarity Transformations

Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)

}form an orthonormal set in Rn.

Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.

Ex

H =

12

12

12

12

−12

12 −1

212

−12 −1

212

12

12 −1

2 −12

12

, HT H = I .

§9.2 Orthogonal Matrices and Similarity Transformations

Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)

}form an orthonormal set in Rn.

Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.

Ex

H =

12

12

12

12

−12

12 −1

212

−12 −1

212

12

12 −1

2 −12

12

, HT H = I .

§9.2 Orthogonal Matrices and Similarity Transformations

Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)

}form an orthonormal set in Rn.

Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.

Ex

H =

12

12

12

12

−12

12 −1

212

−12 −1

212

12

12 −1

2 −12

12

, HT H = I .

Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .

Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.

Proof: Let x 6= 0 be such that

A x =(S−1 B S

)x = λ x.

It follows that B (S x) = λ (S x)

Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .

Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.

Proof: Let x 6= 0 be such that

A x =(S−1 B S

)x = λ x.

It follows that B (S x) = λ (S x)

Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .

Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.

Proof: Let x 6= 0 be such that

A x =(S−1 B S

)x = λ x.

It follows that B (S x) = λ (S x)

Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.

Proof:

A = S D S−1,(S =

(v(1), v(2), · · · , v(n)

), D = diag (λ1, λ2, · · · , λn)

)⇐⇒ A

(v(1), v(2), · · · , v(n)

)=(v(1), v(2), · · · , v(n)

)diag (λ1, λ2, · · · , λn) ,

⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.

Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.

Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.

Proof:

A = S D S−1,(S =

(v(1), v(2), · · · , v(n)

), D = diag (λ1, λ2, · · · , λn)

)⇐⇒ A

(v(1), v(2), · · · , v(n)

)=(v(1), v(2), · · · , v(n)

)diag (λ1, λ2, · · · , λn) ,

⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.

Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.

Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.

Proof:

A = S D S−1,(S =

(v(1), v(2), · · · , v(n)

), D = diag (λ1, λ2, · · · , λn)

)⇐⇒ A

(v(1), v(2), · · · , v(n)

)=(v(1), v(2), · · · , v(n)

)diag (λ1, λ2, · · · , λn) ,

⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.

Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.

Def: A matrix U ∈ Cn×n is unitary if ‖U x‖2 = ‖x‖2 forany vector x.

Schur Thm: Let A ∈ Rn. A unitary matrix U exists such that

T = U−1 AU =

is upper-triangular.

The diagonal entries of T are the eigenvalues of A.

Def: A matrix U ∈ Cn×n is unitary if ‖U x‖2 = ‖x‖2 forany vector x.

Schur Thm: Let A ∈ Rn. A unitary matrix U exists such that

T = U−1 AU =

is upper-triangular.

The diagonal entries of T are the eigenvalues of A.

Def: The complex conjugate of a complex vectoru = a +

√−1b ∈ Cn is u = a−

√−1b.

Def: ‖u‖2 =√aTa + bTb =

√uTu.

Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.

Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,

Au = λu, −→ Au = λu.

λ(uT u

)= uT (Au)

= (Au)T u = λ(uT u

).

Therefore λ = λ ∈ R.

Def: The complex conjugate of a complex vectoru = a +

√−1b ∈ Cn is u = a−

√−1b.

Def: ‖u‖2 =√aTa + bTb =

√uTu.

Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.

Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,

Au = λu, −→ Au = λu.

λ(uT u

)= uT (Au)

= (Au)T u = λ(uT u

).

Therefore λ = λ ∈ R.

Def: The complex conjugate of a complex vectoru = a +

√−1b ∈ Cn is u = a−

√−1b.

Def: ‖u‖2 =√aTa + bTb =

√uTu.

Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.

Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,

Au = λu, −→ Au = λu.

λ(uT u

)= uT (Au)

= (Au)T u = λ(uT u

).

Therefore λ = λ ∈ R.

Def: The complex conjugate of a complex vectoru = a +

√−1b ∈ Cn is u = a−

√−1b.

Def: ‖u‖2 =√aTa + bTb =

√uTu.

Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.

Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,

Au = λu, −→ Au = λu.

λ(uT u

)= uT (Au)

= (Au)T u = λ(uT u

).

Therefore λ = λ ∈ R.

Thm: A matrix A ∈ Rn is symmetric if and only if there exists adiagonal matrix D ∈ Rn and an orthogonal matrix Q so that

A = Q D QT = Q

QT .

Proof: I By induction on n. Assume theorem true for n − 1.I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un

are unit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

(λ 0T

0 UT(A U

) ).

I Matrix UT(A U

)∈ R(n−1)×(n−1) is symmetric.

Thm: A matrix A ∈ Rn is symmetric if and only if there exists adiagonal matrix D ∈ Rn and an orthogonal matrix Q so that

A = Q D QT = Q

QT .

Proof: I By induction on n. Assume theorem true for n − 1.I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un

are unit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

(λ 0T

0 UT(A U

) ).

I Matrix UT(A U

)∈ R(n−1)×(n−1) is symmetric.

Thm: A matrix A ∈ Rn×n is symmetric if and only if there exists adiagonal matrix D ∈ Rn×n and an orthogonal matrix Q so

that A = Q D QT = Q

QT .

Proof: I

UT AU =

(λ 0T

0 UT(A U

) ).

I By induction, there exist diagonal matrix D and orthogonalmatrix Q ∈ R(n−1)×(n−1),

UT(A U

)= Q D QT .

I therefore

UT AU =

(λ 0T

0 Q D QT

).

A =

(U

(1

Q

))(λ

D

)(U

(1

Q

))Tdef= Q D QT .

Thm: A matrix A ∈ Rn×n is symmetric if and only if there exists adiagonal matrix D ∈ Rn×n and an orthogonal matrix Q so

that A = Q D QT = Q

QT .

Proof: I

UT AU =

(λ 0T

0 UT(A U

) ).

I By induction, there exist diagonal matrix D and orthogonalmatrix Q ∈ R(n−1)×(n−1),

UT(A U

)= Q D QT .

I therefore

UT AU =

(λ 0T

0 Q D QT

).

A =

(U

(1

Q

))(λ

D

)(U

(1

Q

))Tdef= Q D QT .

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.

Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .

I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.

A is positive definite

⇐⇒ xT A x > 0 for any non-zero x

⇐⇒(QT x

)TD(QT x

)> 0 for any non-zero x

⇐⇒ yT D y > 0 for any non-zero y

⇐⇒ diagonal entries of D are positive.

§9.3 The Power Method for Google PageRank (I)

I The PageRank Principle: The importance of each Webpageis proportional to the total size of the other Webpages whichare pointing to it.

§9.3 The Power Method for Google PageRank (II)

I random surf with jump: A Websurfer surfs the nextWebpage

I either jumping to a page chosen at random from the entireWeb at 15% likelihood,

I or choosing a random link from the Webpage at 85%likelihood.

§9.3 The Power Method for Google PageRank (III)

I Google Matrix G : each row/column represents awebpage, each G entry models Web connectivity and Webuser surf patterns,

I PageRank vector x is eigenvector for G :

G x = 1 · x,

where 1 is always a simple eigenvalue of G .

I Power Method for iteratively computing x, given x(0),

x(k+1) = G x(k), k = 0, 1, · · · ,

The Power Method, in general

Given: Matrix A ∈ Rn×n, with n eigenvalues

|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .

(A has precisely one eigenvalue, λ1, that is largest in magnitude.)

Task: Compute λ1 and corresponding eigenvector v1.

Despite condition on λ1, PM usually first method to try.

The Power Method, in general

Given: Matrix A ∈ Rn×n, with n eigenvalues

|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .

(A has precisely one eigenvalue, λ1, that is largest in magnitude.)

Task: Compute λ1 and corresponding eigenvector v1.

Despite condition on λ1, PM usually first method to try.

The Power Method, in general

Given: Matrix A ∈ Rn×n, with n eigenvalues

|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .

(A has precisely one eigenvalue, λ1, that is largest in magnitude.)

Task: Compute λ1 and corresponding eigenvector v1.

Despite condition on λ1, PM usually first method to try.

The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.

I Given initial vector x 6= 0. Then

x =n∑

j=1

βj vj

for some coefficients β1, · · · , βn. Assume β1 6= 0.

I For any k > 0

Ak x =n∑

j=1

βj λkj vj

= β1λk1

v1 +n∑

j=2

(λjλ1

)k (βjβ1

vj

)= β1λ

k1

(v1 + O

((λ2λ1

)k))

Ak x points to the direction of v1 for large k .

The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.

I Given initial vector x 6= 0. Then

x =n∑

j=1

βj vj

for some coefficients β1, · · · , βn. Assume β1 6= 0.I For any k > 0

Ak x =n∑

j=1

βj λkj vj

= β1λk1

v1 +n∑

j=2

(λjλ1

)k (βjβ1

vj

)= β1λ

k1

(v1 + O

((λ2λ1

)k))

Ak x points to the direction of v1 for large k .

The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.

I Given initial vector x 6= 0. Then

x =n∑

j=1

βj vj

for some coefficients β1, · · · , βn. Assume β1 6= 0.I For any k > 0

Ak x =n∑

j=1

βj λkj vj

= β1λk1

v1 +n∑

j=2

(λjλ1

)k (βjβ1

vj

)= β1λ

k1

(v1 + O

((λ2λ1

)k))

Ak x points to the direction of v1 for large k .

Rayleigh quotient

Given: Approximate eigenvector x.

Task: Find approximate eigenvalue λ.LS for λ: Choose λ in LS sense

minλ ‖A x− λ x‖2 .

LS Solution:

λ =xT (A x)

xT x

Rayleigh quotient

Given: Approximate eigenvector x.Task: Find approximate eigenvalue λ.

LS for λ: Choose λ in LS sense

minλ ‖A x− λ x‖2 .

LS Solution:

λ =xT (A x)

xT x

Algorithm 1 The Power Method

Input: Matrix A ∈ Rn×n,initial guess x(0) ∈ Rn, and tolerance τ > 0.

Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)

/∥∥x(0)∥∥2, y(0) = A x(0), k = 0.

λ =(x(0))T

y(0).

while∥∥y(k) − λ x(k)∥∥

2≥ τ do

x(k+1) = y(k)/∥∥y(k)∥∥

2, y(k+1) = A x(k+1).

λ =(x(k+1)

)Ty(k+1).

k = k + 1.end while

Algorithm 2 The Symmetric Power Method

Input: Symmetric matrix A ∈ Rn×n,initial guess x(0) ∈ Rn, and tolerance τ > 0.

Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)

/∥∥x(0)∥∥2, y(0) = A x(0), k = 0.

λ =(x(0))T

y(0).

while∥∥y(k) − λ x(k)∥∥

2≥ τ do

x(k+1) = y(k)/∥∥y(k)∥∥

2, y(k+1) = A x(k+1).

λ =(x(k+1)

)Ty(k+1).

k = k + 1.end while

Same PM, but Symmetric PM converges much faster.

I Ex 1: A =

−4 14 0−5 13 0−1 0 2

with x0) =

111

for λ1 = 6.

I Ex 2: A =

−4 −1 1−1 3 −2

1 −2 3

with x0) =

100

for λ1 = 6.

Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then

min1≤j≤n |λ− λj | ≤ τ.

Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix

Qdef= (v1, v2, · · · , vn) is orthogonal, and

x = β1 v1 + β2 v2 + · · ·+ βn vn

with

β1...βn

def= QT x unit vector.

‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2

=

√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2

≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |

Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then

min1≤j≤n |λ− λj | ≤ τ.

Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix

Qdef= (v1, v2, · · · , vn) is orthogonal, and

x = β1 v1 + β2 v2 + · · ·+ βn vn

with

β1...βn

def= QT x unit vector.

‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2

=

√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2

≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |

Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then

min1≤j≤n |λ− λj | ≤ τ.

Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix

Qdef= (v1, v2, · · · , vn) is orthogonal, and

x = β1 v1 + β2 v2 + · · ·+ βn vn

with

β1...βn

def= QT x unit vector.

‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2

=

√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2

≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n

= min1≤j≤n |λ− λj |

Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then

min1≤j≤n |λ− λj | ≤ τ.

Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix

Qdef= (v1, v2, · · · , vn) is orthogonal, and

x = β1 v1 + β2 v2 + · · ·+ βn vn

with

β1...βn

def= QT x unit vector.

‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2

=

√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2

≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |

The Inverse Power Method (I)

Given: Matrix A ∈ Rn×n, with n eigenvalues λ1, λ2, · · · , λn; andgiven shift q.

Task: Compute λi that is closest to q, and correspondingeigenvector vi .

Apply Power Method to (A− q I )−1.

The Inverse Power Method (I)

Given: Matrix A ∈ Rn×n, with n eigenvalues λ1, λ2, · · · , λn; andgiven shift q.

Task: Compute λi that is closest to q, and correspondingeigenvector vi .

Apply Power Method to (A− q I )−1.

I Matrix (A− q I )−1 has eigenvalues

1

λ1 − q,

1

λ2 − q, · · · , 1

λn − q.

I Assume q closest to λi and λk , but closer to λi .

I IPM converges to λi at order

(λi − q

λk − q

)k

.

I Matrix (A− q I )−1 has eigenvalues

1

λ1 − q,

1

λ2 − q, · · · , 1

λn − q.

I Assume q closest to λi and λk , but closer to λi .

I IPM converges to λi at order

(λi − q

λk − q

)k

.

I Matrix (A− q I )−1 has eigenvalues

1

λ1 − q,

1

λ2 − q, · · · , 1

λn − q.

I Assume q closest to λi and λk , but closer to λi .

I IPM converges to λi at order

(λi − q

λk − q

)k

.

Algorithm 3 The Inverse Power Method

Input: Matrix A ∈ Rn×n, shift q,initial guess x(0) ∈ Rn, and tolerance τ > 0.

Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)

/∥∥x(0)∥∥2, y(0) = (A− q I )−1 x(0).

λ =(x(0))T

y(0), k = 0.

while∥∥y(k) − λ x(k)∥∥

2≥ τ do

x(k+1) = y(k)/∥∥y(k)∥∥

2, y(k+1) = (A− q I )−1 x(k+1).

λ =(x(k+1)

)Ty(k+1).

k = k + 1.end while

I Symmetric/Non-symmetric PM Errors

I Symmetric IPM Errors

ReviewThm: A matrix A ∈ Rn×n is symmetric if and only if there exists a

diagonal matrix D ∈ Rn and an orthogonal matrix Q so that

A = Q D QT = Q

QT .

Proof: I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un

are unit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

(λ 0T

0 UT(A U

) ).

I Repeat on symmetric matrix UT(A U

)∈ R(n−1)×(n−1).

ReviewThm: A matrix A ∈ Rn×n is symmetric if and only if there exists a

diagonal matrix D ∈ Rn and an orthogonal matrix Q so that

A = Q D QT = Q

QT .

Proof: I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un

are unit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

(λ 0T

0 UT(A U

) ).

I Repeat on symmetric matrix UT(A U

)∈ R(n−1)×(n−1).

Computing all eigenvalues of matrix A ∈ Rn×n

I Compute one approximate eigenvalue λ of A with uniteigenvector u: Au = λu.

I Extend u into an orthonormal basis for Rn: u,u2, · · · ,un areunit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.

I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

λ uT(A U

)0 UT

(A U

) . (Deflation)

I Continue on matrix Adef= UT

(A U

)∈ R(n−1)×(n−1).

Computing all eigenvalues of matrix A ∈ Rn×n

I Compute one approximate eigenvalue λ of A with uniteigenvector u: Au = λu.

I Extend u into an orthonormal basis for Rn: u,u2, · · · ,un areunit, mutually orthogonal vectors.

Udef= (u,u2, · · · ,un)

def=(u, U

)∈ Rn×n is orthogonal.

I

UT AU =

(uT

UT

)(Au,A U

)=

uT (Au) uT(A U

)UT (Au) UT

(A U

) =

λ uT(A U

)0 UT

(A U

) . (Deflation)

I Continue on matrix Adef= UT

(A U

)∈ R(n−1)×(n−1).

Householder ReflectionLet v ∈ Rn be a unit vector. Define Householder Reflection matrix

H = I − 2v vT ∈ Rn×n.

I H is symmetric and orthogonal

H = HT , H2 = I − 4v vT + 4v vT = I .

I For any vector x ∈ Rn, x†def= H x = x− 2v vTx reflects x in

the direction v⊥:

Deflation with Householder Reflection (I)

I Given eigenvalue λ of A with unit eigenvector u: Au = λu.

I Extend u into an orthonormal basis with a Householderreflection

U = I − 2v vTdef= (u,u2, · · · ,un)

def=(u, U

)I

UT AU =

λ uT(A U

)0 UT

(A U

) .

Find unit vector v so first column of I − 2v vT is u.

Deflation with Householder Reflection (II)I Partition

u =

(µu

), v =

(νv

).

I First column of I − 2v vT is(µu

)=

(10

)− 2

(νv

)ν.

I If µ ≤ 0, then

ν =

√1− µ

2, v = − u

2 ν, U = I − 2v vT =

(u, U

). (1)

I If µ > 0, then −u is also unit eigenvector. Compute v withequation (1) on −u:

ν =

√1 + µ

2, v =

u

2 ν, U = I − 2v vT =

(−u, U

). (2)

Equations (1) and (2) ensure numerical stability

Householder Reflection (II)Let v ∈ Rn be a unit vector. Define Householder Reflection matrix

H = I − 2 v vT ∈ Rn×n.

I For any vector x ∈ Rn, choose v so that

H x =

(± ‖x‖2

0

), (sign to be chosen for numerical stability.)

I

Partition x =

(ξx

),

(± ‖x‖2

0

)= H x =

(ξx

)− 2 v vTx,

udef=

(± ‖x‖2 − ξ−x

)choose===== −

(sign (ξ) (‖x‖2 + |ξ|)

x

)and v = u /‖u‖2