Upload
arden-glover
View
69
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Lecture 13 - Eigen-analysis. CVEN 302 July 1, 2002. Lecture’s Goals. Shift Method Inverse Power Method Accelerated Power Method QR Factorization Householder Hessenberg Method. Shift method. - PowerPoint PPT Presentation
Citation preview
Lecture 13 - Eigen-analysisLecture 13 - Eigen-analysis
CVEN 302
July 1, 2002
Lecture’s GoalsLecture’s Goals
– Shift Method– Inverse Power Method– Accelerated Power Method– QR Factorization– Householder – Hessenberg Method
Shift methodShift method
It is possible to obtain another eigenvalue from the set equations by using a technique known as shifting the matrix.
xxA Subtract the a vector from each side, thereby changing the maximum eigenvalue
xsxIsxA
Shift methodShift method
The eigenvalue, s, is the maximum value of the matrix A. The matrix is rewritten in a form.
IAB max
Use the Power method to obtain the largest eigenvalue of [B].
Example of Shift MethodExample of Shift Method
Consider the follow matrix A
500
120
010
100
010
001
4
100
120
014
B
Assume an arbitrary vector x0 = { 1 1 1}T
Example of Shift MethodExample of Shift Method
Multiply the matrix by the matrix [A] by {x}
5
1
1
1
1
1
500
120
010
Normalize the result of the product
1
6.0
2.0
5-
5
1
1
Example of Shift MethodExample of Shift Method
1
12.0
04.0
5
5
6.0
2.0
5
6.0
2.0
1
2.0
2.0
500
120
010
Continue with the iteration and the final value is = -5. However, to get the true you need to shift back by:
145max
Inverse Power MethodInverse Power Method
The inverse method is similar to the power method, except that it finds the smallest eigenvalue. Using the following technique.
xxA xAxAA 11
xAx 11
xBx
Inverse Power MethodInverse Power Method
The algorithm is the same as the Power method and the “eigenvector” is not the eigenvector for the smallest eigenvalue. To obtain the smallest eigenvalue from the power method.
1
1
Inverse Power MethodInverse Power MethodThe inverse algorithm use the technique avoids calculating the inverse matrix and uses a LU decomposition to find the {x} vector.
xxA xxUL 1
ExampleExample
512
131
024
A
The matrix is defined as:
82.1
2535.5
9264.4
Matlab ProgramMatlab Program
There are set of programs Power and InversePower.
The InversePower(A, x0,iter,tol) does the inverse method.
Accelerated Power MethodAccelerated Power MethodThe Power method can be accelerated by using the Rayleigh Quotient instead of the largest wk value.
The Rayleigh Quotient is defined as:
11 zA
zz
wz
'
'1
Accelerated Power MethodAccelerated Power MethodThe values of the next z term is defined as:
The Power method is adapted to use the new value.
12
wz
Example of Accelerated Power Example of Accelerated Power MethodMethod
Consider the follow matrix A
100
120
014
A
Assume an arbitrary vector x0 = { 1 1 1}T
Example of Accelerated Power Example of Accelerated Power MethodMethod
Multiply the matrix by the matrix [A] by {x}
1
3
5
1
1
1
100
120
014
333.23
7
1
1
1
111
1
3
5
111
1
4286.0
2857.1
1429.2
1
12
wz
Example of Accelerated Power Example of Accelerated Power MethodMethod
Multiply the matrix by the matrix [A] by {x}
4286.0
1429.2
8571.9
4286.0
2857.1
1429.2
100
120
014
6857.3
429.0
2857.1
143.2
429.0286.1143.2
429.0
143.2
857.9
429.02857.1142.2
2
1163.0
5814.0
6744.2
2
23
wz
Example of Accelerated Power Example of Accelerated Power MethodMethod
1163.0
2791.1
2791.11
1163.0
5814.0
6744.2
100
120
014
1171.43
0282.0
3107.0
7396.2
3
34
wz
Example of Accelerated Power Example of Accelerated Power MethodMethod
0282.0
5931.0
2689.11
0282.0
3107.0
7396.2
100
120
014
0849.44
0069.0
1452.0
7587.2
4
45
wz
And so on ...
QR FactorizationQR Factorization
The technique can be used to find the eigenvalue using a successive iteration using Householder transformation to find an equivalent matrix to [A] having an eigenvalues on the diagonal
QR FactorizationQR Factorization
Another form of factorization
A = Q*RProduces an orthogonal matrix (“Q”) and a right upper triangular matrix (“R”)
Orthogonal matrix - inverse is transpose
T1 QQ
Why do we care?
We can use Q and R to find eigenvalues
1. Get Q and R (A = Q*R)2. Let A = R*Q3. Diagonal elements of A are eigenvalue approximations 4. Iterate until converged
QR FactorizationQR Factorization
Note: QR eigenvalue method gives all eigenvalues simultaneously, not just the dominant
In practice, QR factorization on any given matrix requires a number of steps
First transform A into Hessenberg form
xx
xxx
xxxx
xxxxx
xxxxxx
xxxxxxx
xxxxxxxx
xxxxxxxx
M
Hessenberg matrix - upper triangular plus first sub-diagonal
Special properties of Hessenberg matrix make it easier to find Q, R, and eigenvalues
QR Eigenvalue MethodQR Eigenvalue Method
QR FactorizationQR Factorization
• Construction of QR Factorization
matrixngular upper tria:R
; orthogonal:Q
1 IQQQQ
QRATT
QR FactorizationQR Factorization
• Use Householder reflections and given rotations to reduce certain elements of a vector to zero.
• Use QR factorization that preserve the eigenvalues.• The eigenvalues of the transformed matrix are much
easier to obtain.
Jordan Canonical FormJordan Canonical Form• Any square matrix is orthogonally similar to a
triangular matrix with the eigenvalues on the diagonal
matrix nn an
1
1
1
J
J
n
1n
2
1
r
1
Similarity TransformationSimilarity Transformation• Transformation of the matrix A of the form H-1AH is known as
similarity transformation.• A real matrix Q is orthogonal if QTQ = I. • If Q is orthogonal, then A and Q -1AQ are said to be orthogonally
similar
• The eigenvalues are preserved under the similarity transformation.xQAxQxAQQQ
xxAQQAx
then ,xAx If
1111
1
Upper Triangular MatrixUpper Triangular Matrix• The diagonal elements Rii of the upper triangular matrix
R are the eigenvalues
0rrrr
r000
rr00
rrr0
rrrr
IAIADet
r000
rr00
rrr0
rrrr
R
nn332211
nn
n333
n22322
n1131211
nn
n333
n22322
n1131211
)())()((
iii r
Householder ReflectorHouseholder Reflector• Householder reflector is a matrix of the form
• It is straightforward to verify that Q is symmetric and orthogonal
1
22
2
www
wwIQT
T
Iwwww4ww4IQQ
Qww2Iww2IQTTTT
TTTT
Householder MatrixHouseholder Matrix• Householder matrix reduces zk+1 ,…,zn to zero
• To achieve the above operation, v must be a linear combination of x and ek
00yyyHxy
xxxxxx
v
v w;ww2IH
k21
n1kk21
2
n1kk1k21k
Tk
xxxxxxexv
001000e
,,,,,,,
....,,,,...,,
Householder TransformationHouseholder Transformation
k
2
2
2nkk
22
21
2k
2
2
2n
2k
22
21
kn1kk21
n1kk21
xxxx)x(xxxv
x2xx)x(xxvv
e x xxαxxxv
xxxxxx
vvv
xv2xx
vv
vv2IHx
00002
12
222 choose weIf 2
2
2
2
2
22
αvxvvv
xvxHx
xxx
xx
vv
xvx
k
k
αxx) sign(xα k2k ofon cancellati avoid to choose
y
x
x
x
x
x
x
x
HxHk
n
k
k
kkk
0
0
1
1
1
1
)()(
Householder matrixHouseholder matrix• Corollary (kth Householder matrix): Let A be an
nxn matrix and x any vector. If k is an integer with 1< k<n-1 we can construct a vector w(k) and matrix H(k) = I - 2w(k)w’(k) so that
Householder matrixHouseholder matrix• Define the value so that
• The vector w is found by
• Choose = sign(xk)g to reduce round-off error
22yx
22
2
221k
22
21
2n
21k
2k
21k
22
21
2
2
2n
21k
2k
2
gyxxx
xx xxxxx
xxx
2
1k2n
21k
2k
2k
2n
21k
2k
2
n1kk
2
2x 2xxxx 2
xxxs
xx)x(00s
1
yx
yxw
1ww
Householder MatricesHouseholder Matrices
n,,1ki ,s/xw
;/sgxsignx/sxw set:4Step
xgg2x2g2s
gx2gx2g
xxxs compute:3 Step
g xsign
xxxg compute:2 Step
0www set:1 Step
ii
kkkk
kk2
2k
22k
2
2n
21k
2k
2
k
2n
21k
2k
1k21
1www ;
s
x,,
s
x,
s
gxsignx,0,,0w
2
2
n1kkk
Example: Householder MatrixExample: Householder Matrix
2/31/32/3
1/32/32/3
2/32/31/3
224
2
2
4
24
1
100
010
001
ww2IH
22424
12231
24
1ww
24)13)(3(2)xg(g2s
3gxsign ;3221xxxg
2
2
1
1:,Ax ;
212
122
121
A
)1(
)1(
1
12222
322
21
Example: Householder MatrixExample: Householder Matrix
1240.09923.00
9923.01240.00
001
6618.07497.00
6618.0
7497.0
0
100
010
001
ww2IH
6618.07497.003
8
3
6510
s
1ww
0294.4)xg(g2s
/3;65g xsign /3;658/3)((1/3)g
7/38/30
2/31/30
1/34/33
212
122
121
2/31/32/3
1/32/32/3
2/32/31/3
AH
)2(
)2(
2
222
)1(
Basic QR FactorizationBasic QR Factorization• [A] = [Q] [R]• [Q] is orthogonal, QTQ = I• [R] is upper triangular• QR factorization using Householder matrices• Q = H(1)H(2)….H(n-1)
end
RHRDefine
zero toreduce R in
n,,1k positions ithw H Find
1n:1k for
AR Define
)1k()k((k)
1)-(k
)k(
)0(
1)-(n
(k)
RR Define
end
QHQ
1:-1:1-nk for
IQ Define
Example: QR FactorizationExample: QR Factorization
AAHH HH QR
RR
HHQ
AHHRHR
'ww2IH
AHRHR
ww2IH
212
122
121
AR
)1()2()2()1(
)2(
)2()1(
)1()2()1()2()2(
)2(
)1()0()1()1(
)1(
)0(
• Similarity transformation B = QTAQ preserve the eigenvalues
QR FactorizationQR Factorization
2481.07029.06667.0
6202.04134.06667.0
7442.05788.03333.0
1240.09923.00
9923.01240.00
001
2/31/32/3
1/32/32/3
2/32/31/3
HHQ
3721.000
3980.26874.20
3333.03333.13
AHHR
)2()1(
)1()2(
AB
QR = A
Finding Eigenvalues Using Finding Eigenvalues Using QR FactorizationQR Factorization
• Generate a sequence A(m) that are orthogonally similar to A
• Use Householder transformation H-1AH
• the iterates converge to an upper triangular matrix with the eigenvalues on the diagonal
)k()k()1k(
)k()k((k)
QRA
RQA general In
Find all eigenvalues simultaneously!
QR Eigenvalue MethodQR Eigenvalue Method
• QR factorization: A = QR
• Similarity transformation: A(new) = RQ
)3()3()4(
)3()3((3)
)2()2()3(
)2()2((2)
)1()1()2(
)1()1()1(
QRA
RQA
QRA
RQA
QRA
RQA
QAQA
RQA
AQAQR
QRA
)k(T)1k(
)1k(
T1
Example: QR EigenvalueExample: QR Eigenvalue
092302615024810
261527966219290
488410535211112
248107029066670
620204134066670
744205788033330
3721000
39802687420
33330333313
RQA
3721000
39802687420
33330333313
248107029066670
620204134066670
744205788033330
QR
212
122
121
AA
1
0
...
...
...
...
...
...
.
..
..
.
..
..
...
...
...
)(
)(
Example: QR EigenvalueExample: QR Eigenvalue
4178.00038.00017.0
8930.10021.30010.0
3937.18579.14157.2
A
3948.00191.00099.0
9203.19892.20056.0
3930.18691.14056.2
A
5161.01047.00616.0
7694.10527.30310.0
3865.18104.14636.2
A
)4(
)3(
)2(
4142.00000.00000.0
8974.10000.30000.0
3934.18597.14142.2
A
4143.00001.00001.0
8974.10001.30000.0
3934.18596.14143.2
A
4136.00007.00003.0
8982.19996.20002.0
3933.18600.14140.2
A
)7(
)6(
)5(
4142.0 ,0000.3 ,4142.2 321
» A=[1 2 -1; 2 2 -1; 2 -1 2]A = 1 2 -1 2 2 -1 2 -1 2
» [Q,R]=QR_factor(A)Q = -0.3333 -0.5788 -0.7442 -0.6667 -0.4134 0.6202 -0.6667 0.7029 -0.2481R = -3.0000 -1.3333 -0.3333 0.0000 -2.6874 2.3980 0.0000 0.0000 -0.3721
» e=QR_eig(A,6);A = 2.1111 2.0535 1.4884 0.1929 2.7966 -2.2615 0.2481 -0.2615 0.0923
A = 2.4634 1.8104 -1.3865 -0.0310 3.0527 1.7694 0.0616 -0.1047 -0.5161A = 2.4056 1.8691 1.3930 0.0056 2.9892 -1.9203 0.0099 -0.0191 -0.3948A = 2.4157 1.8579 -1.3937 -0.0010 3.0021 1.8930 0.0017 -0.0038 -0.4178A = 2.4140 1.8600 1.3933 0.0002 2.9996 -1.8982 0.0003 -0.0007 -0.4136A = 2.4143 1.8596 -1.3934 0.0000 3.0001 1.8972 0.0001 -0.0001 -0.4143
e = 2.4143 3.0001 -0.4143
MATLAB Example
QR factorization
eigenvalue
Improved QR MethodImproved QR Method
• Using similarity transformation to form an upper Hessenberg Matrix (upper triangular matrix & one nonzero band below diagonal) .
• More efficient to form Hessenberg matrix without explicitly forming the Householder matrices (not given in textbook).
function A = Hessenberg(A)[n,nn] = size(A);for k = 1:n-2 H = Householder(A(:,k),k+1); A = H*A*H;end
» A=[1 2 -1; 2 2 -1; 2 -1 2]A = 1 2 -1 2 2 -1 2 -1 2» [Q,R]=QR_factor_g(A)Q = 0.4472 0.5963 -0.6667 0.8944 -0.2981 0.3333 0 -0.7454 -0.6667R = 2.2361 2.6833 -1.3416 -1.4907 1.3416 -1.7889 -1.3333 0 -1.0000» e=QR_eig_g(A,6);A = 2.1111 -2.4356 0.7071 -0.3143 -0.1111 -2.0000 0 0.0000 3.0000A = 2.4634 2.0523 -0.9939 -0.0690 -0.4634 -1.8741 0.0000 0.0000 3.0000
Improved QR MethodImproved QR Method A = 2.4056 -2.1327 0.9410 -0.0114 -0.4056 -1.9012 0.0000 0.0000 3.0000A = 2.4157 2.1194 -0.9500 -0.0020 -0.4157 -1.8967 0.0000 0.0000 3.0000A = 2.4140 -2.1217 0.9485 -0.0003 -0.4140 -1.8975 0.0000 0.0000 3.0000A = 2.4143 2.1213 -0.9487 -0.0001 -0.4143 -1.8973 0.0000 0.0000 3.0000e = 2.4143 -0.4143 3.0000» eig(A)ans = 2.4142 -0.4142 3.0000
Hessenberg matrix
MATLAB function
eigenvalue
SummarySummary
• Single value eigen-analysis– Power Method– Shifting technique– Inverse Power Method
• QR Factorization– Householder matrix– Hessenberg matrix
HomeworkHomework
• Check the Homework webpage