View
235
Download
1
Category
Preview:
Citation preview
EECS 275 Matrix Computation
Ming-Hsuan Yang
Electrical Engineering and Computer ScienceUniversity of California at Merced
Merced, CA 95344http://faculty.ucmerced.edu/mhyang
Lecture 12
1 / 18
Overview
QR decomposition by Householder transformation
QR decomposition by Givens rotation
2 / 18
Reading
Chapter 10 of Numerical Linear Algebra by Llyod Trefethen andDavid Bau
Chapter 5 of Matrix Computations by Gene Golub and Charles VanLoan
Chapter 5 of Matrix Analysis and Applied Linear Algebra by CarlMeyer
3 / 18
Householder triangularization
In Gram-Schmidt process
AR1R2 · · ·Rn︸ ︷︷ ︸R−1
= Q
has orthonormal columns. The product R = R−1n · · ·R−12 R−11 is uppertriangular
In Householder triangularization, a series of elementary orthogonalmatrices Qk is applied to the left of A
Qn · · ·Q2Q1︸ ︷︷ ︸Q>
A = R
is upper triangular. The product Q = Q>1 Q>2 · · ·Q>n is orthogonal,and therefore A = QR is a QR factorization of A
4 / 18
Geometry of elementary projectors
For u, x ∈ IRn, s.t. ‖u‖ = 1
Orthogonal projectors onto span{u} and u⊥ are
Pu =uu>
u>u, and Pu⊥ = I − uu>
u>u
For u 6= 0, the Householder transformation or the elementary reflectorabout u⊥ is
R = I − 2uu>
u>uor
R = I − 2uu>
when ‖u‖ = 1, andR = R> = R−1
5 / 18
Triangularization by introducing zeros
The matrix Qk is chosen to introduce zeros below the diagonal in thek-th column while preserving all the zeros previously introduced× × ×× × ×× × ×× × ×× × ×
Q1−→
� � �0 � �0 � �0 � �0 � �
Q2−→
× × ×
� �0 �0 �0 �
Q3−→
× × ×× ×
�00
A Q1A Q2Q1A Q3Q2Q1A
Qk operates on row k, . . . ,m (changed entries are denoted byboldface or � and blank entries are zero)
At beginning of step k , there is a block of zeros in the first k − 1columns of these rows
The application of Qk forms linear combinations of these rows, andthe linear combination of the zero entries remain zero
After n steps, all the entries below the diagonal have been eliminatedand Qn · · ·Q2Q1A = R is upper triangular
6 / 18
Householder reflectors
At beginning of step k , there is a block of zeros in the first k − 1columns of these rows
Each Qk is chosen to be
Qk =
[I 00 F
]where I is the (k − 1)× (k − 1) identity matrix and F is an(m − k + 1)× (m − k + 1) orthogonal matrix
Multiplication by F has to introduce zeros into the k-th column
The Householder algorithm chooses F to be a particular matrix calledHouseholder reflector
At step k , the entries k, . . . ,m of the k-th column are given by vectorx ∈ IRm−k+1
7 / 18
Householder transformation (cont’d)
To introduce zeros into k-th column (x ∈ IRm−k+1), the Householdertransformation F should
x =
××...×
F−→ Fx =
‖x‖
0...0
= ‖x‖e1 = αe1
The reflector F will reflect the space IRm−k+1 across the hyperplaneH orthogonal to u = ‖x‖e1 − x
A hyperplane is characterized by a vector u = ‖x‖e1 − x
8 / 18
Householder transformation (cont’d)
Every point x ∈ IRm is mapped to a mirror point
Fx = (I − 2uu>
u>u)x = x− 2u(
u>x
u>u)
and hence
F = (I − 2uu>
u>u)
Will fix the +/- sign in the next slide
9 / 18
The better of two Householder reflectors
Two Householder reflectors (transformations)
For numerical stability pick the one that moves reflect x to the vector‖x‖e1 that is not to close to x itself, i.e., −‖x‖e1 − x in this case
In other words, the better of the two reflectors is
u = sign(x1)‖x‖e1 + x
where x1 is the first element of x (sign(x1) = 1 if x1 = 0)
10 / 18
Householder QR factorization
Algorithm:
for k = 1 to n dox = Ak:m,k
uk = sign(x1)‖x‖2e1 + xuk = uk
‖uk‖2Ak:m,k:n = (I − 2uku
>k )Ak:m,k:n
end for
Recall
Qk =
[I 00 F
]Upon completion, A has been reduced to upper triangular form, i.e.,R in A = QR
Q> = Qn · · ·Q2Q1 or Q = Q>1 Q>2 · · ·Q>n
11 / 18
QR decomposition with Householder transformation
Want to compute QR decomposition A with Householdertransformation
A =
12 −51 46 167 −68−4 24 −41
Need to find a reflector for first column of A, x = [12, 6,−4]> to‖x‖e1 = [14, 0, 0]>
u=‖x‖e1 − x = [2,−6, 4]> = 2[1,−3, 2]>
F1=I − 2uu>
u>u=
6/7 3/7 −2/73/7 −2/7 6/7−2/7 6/7 3/7
,F1A =
14 21 −140 −49 −140 168 −77
Next need to zero out A32 and apply the same process to
A′ =
[−49 −14168 −77
]12 / 18
QR decomposition with Householder (cont’d)
With the same process
F2 =
1 0 00 −7/25 24/250 24/25 7/25
Thus, we have
Q = Q1Q2 =
6/7 −69/175 58/1753/7 158/175 −6/175−2/7 6/35 33/35
R = Q2Q1A = Q>A =
14 21 −140 175 −700 0 −35
The matrix Q is orthogonal and R is upper triangular
13 / 18
Givens rotationsGivens rotation: orthogonal transform to zero out elements selectively
G (i , k , θ) =
1 · · · 0 · · · 0 · · · 0...
. . ....
......
0 · · · c · · · s · · · 0...
... · · ·...
...0 · · · −s · · · c · · · 0...
......
. . ....
0 · · · 0 · · · 0 · · · 1
i
k
i kwhere c = cos(θ) and s = sin(θ) for some θPre-multiply G (i , k , θ) amounts to a counterclockwise rotation θ inthe (i , k) coordinate plane, y = G (i , k , θ)x
yj =
cxi − sxk j = i
sxi + cxk j = k
xj j 6= i , k14 / 18
Givens rotations (cont’d)
Can zero out yk = sxi + cxk = 0 by setting
c =xi√
x2i + x2k
, s =−xk√x2i + x2k
, θ = arctan(xk/xi )
QR decomposition can be computed by a series of Givens rotations
Each rotation zeros an element in the subdiagonal of the matrix,forming R matrix, Q = G1 . . .Gn forms the orthogonal Q matrix
Useful for zero out few elements off diagonal (e.g., sparse matrix)
Example
A =
12 −51 46 167 −68−4 24 −41
Want to zero out A31 = −4 with rotation vector (6,−4) to pointalong the x-axis, i.e., θ = arctan(−4/6)
15 / 18
QR factorization with Givens rotation
With θ we have the orthogonal Givens rotation G1
G1 =
1 0 00 cos(θ) sin(θ)0 −sin(θ) cos(θ)
=
1 0 00 0.83205 −0.554700 0.55470 0.83205
Pre-multiply A with G1
G1A =
12 −51 47.2110 125.6396 −33.83671
0 112.6041 −71.83368
Continue to zero out A21 and A32 and form a triangular matrix R
The orthogonal matrix Q> = G3G2G1, and G3G2G1A = Q>A = R forQR decomposition
16 / 18
Gram-Schmidt, Householder and Givens
Householder QR is numerically more stable
Gram-Schmidt computes orthonormal basis incrementally
Givens rotation is more useful for zero out few selective elements
17 / 18
Eigendecomposition
Also known as spectral decomposition
A is a square matrixA = QDQ−1
where Q is a square matrix whose columns are eigenvector and D is adiagonal matrix whose elements are the corresponding eigenvalues
With eigendecomposition
AQ = QDAqi = λiqi
where λi and qi are eigenvalues and eigenvectors of
Ax = λx
The eigenvectors are usually normalized but not necessarily
If A can be eigendecomposed with all non-zero eigenvalues
A−1 = QD−1Q−1
18 / 18
Recommended