Understanding the QR algorithm,Part X
David S. [email protected]
Department of Mathematics
Washington State University
Glasgow 2009 – p. 1
1. Understanding the QR algorithm, SIAM Rev., 1982
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
4. QR-like algorithms—an overview of convergencetheory and practice, AMS proceedings, 1996
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
4. QR-like algorithms—an overview of convergencetheory and practice, AMS proceedings, 1996
5. QR-like algorithms for eigenvalue problems, JCAM,2000
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
4. QR-like algorithms—an overview of convergencetheory and practice, AMS proceedings, 1996
5. QR-like algorithms for eigenvalue problems, JCAM,2000
6. Fundamentals of Matrix Computations, Wiley, 2002
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
4. QR-like algorithms—an overview of convergencetheory and practice, AMS proceedings, 1996
5. QR-like algorithms for eigenvalue problems, JCAM,2000
6. Fundamentals of Matrix Computations, Wiley, 2002
7. The Matrix Eigenvalue Problem: GR and KrylovSubspace Methods, SIAM, 2007.
Glasgow 2009 – p. 2
1. Understanding the QR algorithm, SIAM Rev., 1982
2. Fundamentals of Matrix Computations, Wiley, 1991
3. Some perspectives on the eigenvalue problem, 1993
4. QR-like algorithms—an overview of convergencetheory and practice, AMS proceedings, 1996
5. QR-like algorithms for eigenvalue problems, JCAM,2000
6. Fundamentals of Matrix Computations, Wiley, 2002
7. The Matrix Eigenvalue Problem: GR and KrylovSubspace Methods, SIAM, 2007.
8. The QR algorithm revisited, SIAM Rev., 2008.
Glasgow 2009 – p. 2
Some names associated withthe QR algorithm
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Francis
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Francis
Implicitly Shifted QR algorithm
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Francis
Implicitly Shifted QR algorithmHow should we understand it?
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Francis
Implicitly Shifted QR algorithmHow should we understand it? . . . view it?
Glasgow 2009 – p. 3
Some names associated withthe QR algorithm (short list)
Rutishauser
Kublanovskaya
Francis
Implicitly Shifted QR algorithmHow should we understand it? . . . view it?. . . teach it to our students?
Glasgow 2009 – p. 3
The Standard Approach . . .
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
This is simple,
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
This is simple, appealing,
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
This is simple, appealing, does not require muchpreparation,
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
This is simple, appealing, does not require muchpreparation, but . . .
Glasgow 2009 – p. 4
The Standard Approach . . .. . . dating from the work of Francis
Start with the basic algorithm . . .
A = QR RQ = A repeat!
This is simple, appealing, does not require muchpreparation, but . . .
. . . it is far removed from versions of theQRalgorithm that are actually used.
Glasgow 2009 – p. 4
Refinements
Glasgow 2009 – p. 5
Refinementsshifts of origin
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
implicit-Q theorem
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
implicit-Q theoremvs.Krylov subspaces
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
implicit-Q theoremvs.Krylov subspaces
Introducing Krylov subspaces improvesunderstanding,
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
implicit-Q theoremvs.Krylov subspaces
Introducing Krylov subspaces improvesunderstanding, allows more general results,
Glasgow 2009 – p. 5
Refinementsshifts of origin
reduction to Hessenberg form
implicit shift technique (Francis)
double shiftQR
multiple shiftQR
implicit-Q theoremvs.Krylov subspaces
Introducing Krylov subspaces improvesunderstanding, allows more general results, andprepares students for Krylov subspace methods.
Glasgow 2009 – p. 5
The Implicitly Shifted QR Iteration
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI)
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Perform similarity transformA → Q∗0AQ0.
Glasgow 2009 – p. 6
The Implicitly Shifted QR Iterationmatrix is in upper Hessenberg form
pick some shiftsρ1, . . . ,ρm (m = 1, 2, 4, 6)
p(A) = (A − ρ1I) · · · (A − ρmI) expensive!
computep(A)e1 cheap!
Build unitaryQ0 with q1 = αp(A)e1.
Perform similarity transformA → Q∗0AQ0.
Hessenberg form is disturbed.
Glasgow 2009 – p. 6
An Upper Hessenberg Matrix@
@@
@@
@@
@@
@@
@@
Glasgow 2009 – p. 7
After the Transformation ( Q∗0AQ0)
@@
@@
@@
@@
@@
Glasgow 2009 – p. 8
After the Transformation ( Q∗0AQ0)
@@
@@
@@
@@
@@
Now return the matrix to Hessenberg form.
Glasgow 2009 – p. 8
Chasing the Bulge@
@@@
@@
@@
@@@
Glasgow 2009 – p. 9
Chasing the Bulge@
@@
@@
@@
@@
@
Glasgow 2009 – p. 10
Done@
@@
@@
@@
@@
@@
@@
Glasgow 2009 – p. 11
Done@
@@
@@
@@
@@
@@
@@
The implicitQR step is complete!
Glasgow 2009 – p. 11
Summary of Implicit QR Iteration
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q∗0AQ0)
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q∗0AQ0)
Chase the bulge. (return to Hessenberg form)
Glasgow 2009 – p. 12
Summary of Implicit QR IterationPick some shifts.
Computep(A)e1. (p determined by shifts)
Build Q0 with first columnq1 = αp(A)e1.
Make a bulge. (A → Q∗0AQ0)
Chase the bulge. (return to Hessenberg form)
A = Q∗AQ
Glasgow 2009 – p. 12
Question
Glasgow 2009 – p. 13
QuestionThis differs a lot from the basicQR step.
Glasgow 2009 – p. 13
QuestionThis differs a lot from the basicQR step.
A = QR RQ = A
Glasgow 2009 – p. 13
QuestionThis differs a lot from the basicQR step.
A = QR RQ = A
Can we carve a reasonable pedagogical path thatleads directly to the implicitly-shiftedQR algorithm,
Glasgow 2009 – p. 13
QuestionThis differs a lot from the basicQR step.
A = QR RQ = A
Can we carve a reasonable pedagogical path thatleads directly to the implicitly-shiftedQR algorithm,bypassing the basicQR algorithm entirely?
Glasgow 2009 – p. 13
QuestionThis differs a lot from the basicQR step.
A = QR RQ = A
Can we carve a reasonable pedagogical path thatleads directly to the implicitly-shiftedQR algorithm,bypassing the basicQR algorithm entirely?
That’s what we are going to do today.
Glasgow 2009 – p. 13
Ingredients
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces and subspace iteration
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces and subspace iteration
(unitary) similarity transformation(change of coordinate system)
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces and subspace iteration
(unitary) similarity transformation(change of coordinate system)
reduction to Hessenberg form
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces and subspace iteration
(unitary) similarity transformation(change of coordinate system)
reduction to Hessenberg form
Hessenberg form and Krylov subspaces(instead of implicit-Q theorem)
Glasgow 2009 – p. 14
Ingredientssubspace iteration (power method)
Krylov subspaces and subspace iteration
(unitary) similarity transformation(change of coordinate system)
reduction to Hessenberg form
Hessenberg form and Krylov subspaces(instead of implicit-Q theorem)
No Magic Shortcut!
Glasgow 2009 – p. 14
Power Method, Subspace Iteration
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
S, p(A)S, p(A)2S, p(A)3S, . . .
Glasgow 2009 – p. 15
Power Method, Subspace Iterationv, Av, A2v, A3v, . . .
convergence rate|λ2/λ1 |
S, AS, A2S, A3S, . . .
subspaces of dimensionj (|λj+1/λj |)
Substitutep(A) for A (shifts, multiple steps)
S, p(A)S, p(A)2S, p(A)3S, . . .
convergence rate|p(λj+1)/p(λj) |
Glasgow 2009 – p. 15
Krylov Subspaces . . .
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace Iteration
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
. . . becausep(A)A = Ap(A)
Glasgow 2009 – p. 16
Krylov Subspaces . . .. . . and Subspace IterationDef: Kj(A, q) = span
{
q, Aq,A2q, . . . , Aj−1q}
j = 1, 2, 3, . . . (nested subspaces)
Kj(A, q) are “determined byq”.
p(A)Kj(A, q) = Kj(A, p(A)q)
. . . becausep(A)A = Ap(A)
Conclusion: Power method induces nested subspaceiterations on Krylov subspaces.
Glasgow 2009 – p. 16
power method: p(A)kq
Glasgow 2009 – p. 17
power method: p(A)kq
nested subspace iterations:
p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .
Glasgow 2009 – p. 17
power method: p(A)kq
nested subspace iterations:
p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .
convergence rates:
|p(λj+1)/p(λj) |, j = 1, 2, 3, . . .
Glasgow 2009 – p. 17
(Unitary) Similarity Transforms
Glasgow 2009 – p. 18
(Unitary) Similarity TransformsA → Q∗AQ preserves eigenvalues
Glasgow 2009 – p. 18
(Unitary) Similarity TransformsA → Q∗AQ preserves eigenvalues
transforms eigenvectors in a simple way(w → Q∗w)
Glasgow 2009 – p. 18
(Unitary) Similarity TransformsA → Q∗AQ preserves eigenvalues
transforms eigenvectors in a simple way(w → Q∗w)
is a change of coordinate system (v → Q∗v)
Glasgow 2009 – p. 18
(Unitary) Similarity TransformsA → Q∗AQ preserves eigenvalues
transforms eigenvectors in a simple way(w → Q∗w)
is a change of coordinate system (v → Q∗v)
triangular form (eigenvalues)
Glasgow 2009 – p. 18
(Unitary) Similarity TransformsA → Q∗AQ preserves eigenvalues
transforms eigenvectors in a simple way(w → Q∗w)
is a change of coordinate system (v → Q∗v)
triangular form (eigenvalues)
relationship of invariant subspaces to triangular form
Glasgow 2009 – p. 18
Subspace Iterationwith change of coordinate system
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q∗AQ
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q∗AQ
qk → Q∗qk = ek
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q∗AQ
qk → Q∗qk = ek
span{q1, . . . , qj} → span{e1, . . . , ej}
Glasgow 2009 – p. 19
Subspace Iterationwith change of coordinate system
takeS = span{e1, . . . , ej}
p(A)S = span{p(A)e1, . . . , p(A)ej}
= span{q1, . . . , qj} (orthonormal)
build unitaryQ = [q1 · · · qj · · ·]
change coordinate system:A = Q∗AQ
qk → Q∗qk = ek
span{q1, . . . , qj} → span{e1, . . . , ej}
ready for next iterationGlasgow 2009 – p. 19
This version of subspace iteration . . .
Glasgow 2009 – p. 20
This version of subspace iteration . . .
. . . holds the subspace fixed
Glasgow 2009 – p. 20
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
Glasgow 2009 – p. 20
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
. . . moving toward a matrix under which
span{e1, . . . , ej}
is invariant.
Glasgow 2009 – p. 20
This version of subspace iteration . . .
. . . holds the subspace fixed
while the matrix changes.
. . . moving toward a matrix under which
span{e1, . . . , ej}
is invariant.
A →
[
A11 A12
0 A22
]
(A11 is j × j.)
Glasgow 2009 – p. 20
Reduction to Hessenberg form
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
can always be done (direct method,O(n3) flops)
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
can always be done (direct method,O(n3) flops)
brings us closer to triangular form
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
can always be done (direct method,O(n3) flops)
brings us closer to triangular form
makes computations cheaper
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
can always be done (direct method,O(n3) flops)
brings us closer to triangular form
makes computations cheaper
First columnq1 can be chosen “arbitrarily”.
Glasgow 2009 – p. 21
Reduction to Hessenberg formQ → Q∗AQ = H (a similarity transformation)
can always be done (direct method,O(n3) flops)
brings us closer to triangular form
makes computations cheaper
First columnq1 can be chosen “arbitrarily”.
Example: q1 = αp(A)e1
Glasgow 2009 – p. 21
Krylov Subspaces . . .
Glasgow 2009 – p. 22
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
Glasgow 2009 – p. 22
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
Glasgow 2009 – p. 22
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
A properly upper Hessenberg=⇒
Kj(A, e1) = span{e1, . . . , ej}.
Glasgow 2009 – p. 22
Krylov Subspaces . . .. . . and Hessenberg matrices . . .
. . . go hand in hand.
A properly upper Hessenberg=⇒
Kj(A, e1) = span{e1, . . . , ej}.
More generally . . .
Glasgow 2009 – p. 22
Krylov-Hessenberg Relationship
Glasgow 2009 – p. 23
Krylov-Hessenberg RelationshipIf H = Q∗AQ, andH is properly upper Hessenberg,then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Glasgow 2009 – p. 23
Krylov-Hessenberg RelationshipIf H = Q∗AQ, andH is properly upper Hessenberg,then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Proof (sketch):
Glasgow 2009 – p. 23
Krylov-Hessenberg RelationshipIf H = Q∗AQ, andH is properly upper Hessenberg,then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Proof (sketch): Induction onj.
Glasgow 2009 – p. 23
Krylov-Hessenberg RelationshipIf H = Q∗AQ, andH is properly upper Hessenberg,then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Proof (sketch): Induction onj. AQ = QH
Glasgow 2009 – p. 23
Krylov-Hessenberg RelationshipIf H = Q∗AQ, andH is properly upper Hessenberg,then forj = 1, 2, 3, . . . ,
span{q1, . . . , qj} = Kj(A, q1).
Proof (sketch): Induction onj. AQ = QH
Aqj =n
∑
i=1
qihij =
j∑
i=1
qihij + qj+1hj+1,j
Glasgow 2009 – p. 23
Aqj =
j∑
i=1
qihij + qj+1hj+1,j
Glasgow 2009 – p. 24
Aqj =
j∑
i=1
qihij + qj+1hj+1,j
qj+1hj+1,j = Aqj −
j∑
i=1
qihij
Glasgow 2009 – p. 24
Aqj =
j∑
i=1
qihij + qj+1hj+1,j
qj+1hj+1,j = Aqj −
j∑
i=1
qihij
Proof by induction follows immediately.
Glasgow 2009 – p. 24
Aqj =
j∑
i=1
qihij + qj+1hj+1,j
qj+1hj+1,j = Aqj −
j∑
i=1
qihij
Proof by induction follows immediately.
This also gives the student a preview of the Arnoldiprocess,
Glasgow 2009 – p. 24
Aqj =
j∑
i=1
qihij + qj+1hj+1,j
qj+1hj+1,j = Aqj −
j∑
i=1
qihij
Proof by induction follows immediately.
This also gives the student a preview of the Arnoldiprocess,the most important Krylov subspacemethod.
Glasgow 2009 – p. 24
and now,
Glasgow 2009 – p. 25
and now, the Implicit QR Iteration
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
. . . efficiency.
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
. . . efficiency.
. . . automatic nested subspace iterations.
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
. . . efficiency.
. . . automatic nested subspace iterations.
Get some shiftsρ1, . . . ,ρm to definep.
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
. . . efficiency.
. . . automatic nested subspace iterations.
Get some shiftsρ1, . . . ,ρm to definep.
Computep(A)e1. (power method)
Glasgow 2009 – p. 25
and now, the Implicit QR IterationWork with Hessenberg form to get . . .
. . . efficiency.
. . . automatic nested subspace iterations.
Get some shiftsρ1, . . . ,ρm to definep.
Computep(A)e1. (power method)
TransformA to upper Hessenberg form:
A = Q∗AQ
by a matrixQ that hasq1 = αp(A)e1.
Glasgow 2009 – p. 25
A = Q∗AQ where q1 = αp(A)e1.
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem
j = 1, 2, 3, . . . ,n − 1
Glasgow 2009 – p. 26
A = Q∗AQ where q1 = αp(A)e1.
q1 → Q∗q1 = e1
power method with a change of coordinate system.Moreover . . .
p(A)Kj(A, e1) = Kj(A, p(A)e1)
i.e.p(A)span{e1, . . . , ej} = span{q1, . . . , qj}
subspace iteration with a change of coordinatesystem
j = 1, 2, 3, . . . ,n − 1
|p(λj+1)/p(λj) | j = 1, 2, 3, . . . ,n − 1
Glasgow 2009 – p. 26
Details
Glasgow 2009 – p. 27
Detailschoice of shifts
Glasgow 2009 – p. 27
Detailschoice of shifts
We change the shifts at each step.
Glasgow 2009 – p. 27
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Glasgow 2009 – p. 27
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Other Questions
Glasgow 2009 – p. 27
Detailschoice of shifts
We change the shifts at each step.
⇒ quadratic or cubic convergence
Other Questions. . . how to get BLAS 3 speed?
. . . how to parallelize?
Glasgow 2009 – p. 27
In Conclusion
Glasgow 2009 – p. 28
In ConclusionA careful study of
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspaces
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspacesleads directly to the implicitly shiftedQR algorithm.
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspacesleads directly to the implicitly shiftedQR algorithm.
The basic, explicitQR algorithm is skipped.
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspacesleads directly to the implicitly shiftedQR algorithm.
The basic, explicitQR algorithm is skipped.
The implicit-Q theorem is avoided.
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspacesleads directly to the implicitly shiftedQR algorithm.
The basic, explicitQR algorithm is skipped.
The implicit-Q theorem is avoided.
Krylov subspaces are emphasized.
Glasgow 2009 – p. 28
In ConclusionA careful study of the power method and its extensions,similarity transformations,Hessenberg form,andKrylov subspacesleads directly to the implicitly shiftedQR algorithm.
The basic, explicitQR algorithm is skipped.
The implicit-Q theorem is avoided.
Krylov subspaces are emphasized.
Krylov subspace methods are foreshadowed.
Glasgow 2009 – p. 28
One Last Question
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithm
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name?
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name? Some possibilities: . . .
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name? Some possibilities: . . .
. . . unitary bulge-chasing algorithm
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name? Some possibilities: . . .
. . . unitary bulge-chasing algorithm
. . . Hessenberg-Krylov nonstationary progressivenested subspace iteration
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name? Some possibilities: . . .
. . . unitary bulge-chasing algorithm
. . . Hessenberg-Krylov nonstationary progressivenested subspace iteration
. . . Francis’s algorithm
Glasgow 2009 – p. 29
One Last QuestionIn the implicitly shiftedQR algorithmtheQR decomposition is nowhere to be seen.
Should the implicitly-shiftedQR algorithm be givena different name? Some possibilities: . . .
. . . unitary bulge-chasing algorithm
. . . Hessenberg-Krylov nonstationary progressivenested subspace iteration
. . . Francis’s algorithm
Thank you for your attention.
Glasgow 2009 – p. 29