Upload
m-rasheed
View
219
Download
0
Embed Size (px)
Citation preview
7/21/2019 Householder Matrices
1/10
Householder Matrices
DEFINITION. A linear transformation P of nis said to be a projection if
P2= P* = P.
PROPOSITION. If P is a projection, then 1 - P is a projection.
PROOF. We have that(1 - P)* = 1 - P
and
(1 - P)(1 - P) = 1 - (1)P - P(1) + PP = 1 - P - P + P = 1 - P.
Thus, the linear transformation 1 - P is a projection. Q.E.D.
EXAMPLE. Let a be a unit vector in n . Let a!a denote the linear transformation
a!a(x) = a.
Then a!a is a projection. This was demonstrated in Homework 8. The matrix of this linear transformation is A =
aa* since
Ax = aa*x = (a*x) a = a.
EXAMPLE. More generally, let a and b be vectors in n. Let a!b denote the linear transformation
a!b(x) = a.
The matrix of a!b is given by ab*.
DEFINITION. Let a be a unit vector in n. The linear transformations of the type Ua given by
Ua(x) = x - 2a
is called a Householder transformations.
PROPOSITION. Let a be a unit vector in n. Then the Householder transformation Uais a selfadjoint unitary
transformation.
PROOF. The Householder transformation Uacan be expressed as
Ua = (1 - Pa) - Pa.
where Pa= P is the projection
P(x) = a.
So we have that
Ua* = ((1 - P) - P)*
= (1 - P)* - P*
= (1 - P) - P = Ua
and
Ua*Ua = UaUa
= ((1 - P) - P)((1 - P) - P)
= (1 - P)2-P(1 - P) - (1 - P)P + P 2
= (1 - P) - (P - P2 ) - (P - P2) + P
= (1 - P) - (P - P) - (P - P) + P
= 1 - P + P = 1.
So we have that Uais a selfadjoint unitary. Q.E.D.
Householder Matrices page 1
7/21/2019 Householder Matrices
2/10
PROPOSITION. Let w = (0, , 0, cos ", 0, , 0, sin ", 0, , 0). Then
A Householder transformation has a simple geometric description.
PROPOSITION. Let a be a unit vector. Then Ua(a) = - a and Ua(x) = x for every x perpendicular to a. Here x is
perpendicular to a if = 0.
PROOF. We have that
Ua(a) = a - 2a = a - 2a = -aand
Ua(x) = x - a = x
if x is perpendicular to a. Q.E.D.
The diagram for the Householder transformation is the following:
The geometry of the Householder transformation is useful in deriving formulae for moving vectors aroud via a
Householder transformation.
PROPOSITION. Let x and y be vectors of the same length in nwith x #y. Then Householder transformation Uw
with
w =x - y
||x - y||
takes x onto y.
REMARK. If x = y in the preceding formula, then the Householder transformation U0= 1 maps x onto y = x.
PROOF. We have that v = x + y is perpendicular to w since
=x - y
||x - y||, x + y =
1
||x - y|| + - - = 0.
Here we used the fact that
=
from the hypothesis on the length and the fact that
=
since x and y are in $n. So we have that
Uw(x + y) = x + y.
Thus, we get that
Householder Matrices page 2
7/21/2019 Householder Matrices
3/10
Uw(x) = Uwx - y
2+
x + y
2=
1
2Uw(x - y) +
1
2Uw(x + y) = -
1
2(x - y) +
1
2(x + y) = y.
Q.E.D.
A Householder matrix can be used to drive a matrix to so-called Hessenberg form.
DEFINITION. An n" n matrix A = (ai j) is said to be in (upper) Hessenberg form if ai j= 0 if i > j + 1.
The definition means that A is in Hessenberg from if everything below the subdiagonal of a matrix is 0. The
subdiagonal is the diagonal from the upper left hand corner to the lower right hand corner that lies immediately
below the main diagonal.
We now show that there is a Householder transformation that will "zero-out" the lower entries of a vector.
LEMMA (Zeroing Out Lemma). Let a = (a1, , an)Tbe a vector in !n. Then there is a Householder matrix
Uwsuch that
U
w
a = ||a|| e
1(respectively, Uwa = - ||a||e1).
PROOF. First we see that the vector w%= a - ||a|| e1is perpendicular to a + ||a||e1since
= ||a||2- ||a|| + ||a|| < e1, a> - ||a||
2 = 0
Setting
w = w%/||w%||,
if w%#0, we get that
Uw
(a) = 1
2(U
w(a - ||a||e1) +
1
2U
w(a + ||a||e1)) = -
1
2(a - ||a||e1) +
1
2(a + ||a||e1) = ||a||e1.
If w%= 0, then we take U equal to the identity.
A similar proof holds for the second case. Q.E.D.
REMARK. A same Lemma holds for complex vectors. We show that there is a complex number &of modulus 1
and a unit vector w with the propery
Uwa = &||a||e1.
Here we assume that there is some need for the zeroing out, i.e., that
|a2| + + |a
n| > 0.
For every complex number &of modulus 1, the vector
a - &||a||e1= (a1- &||a||, a2, , an)
is nonzero. We shall give the proper choice of&presently. We form the Householder transform Uw
where w is the
vector
w = a - &||a||e1
|| a - &||a||e1||.
By the geometry of the Householder matrix, we get
Uw
(a - &||a||e1) = - (a - &||a||e1)
and
Uw
(a + &||a||e1) = a + &||a||e1
since
Householder Matrices page 3
7/21/2019 Householder Matrices
4/10
= ||a||2- & +& - |&|
2 ||a||2.
Now we see that the proper choice of &is the complex number of modulus 1 with the property
& '$.
This is the number
&=
if &is nonzero or 1 if & = 0. Then we have that
& = &
so that
- & +& = 0.
Now we can compute Uw
a as
Uw
(a) = 1
2(U
w(a - &||a||e1) +
1
2U
w(a + &||a||e1)) = -
1
2(a - &||a||e1) +
1
2(a + &||a||e1) = &||a||e1.
To prove the second part we use the vector a + &||a||e1
THEOREM. Let A be an n "n complex matrix. There are Householder matrices U1, , U
n -1such that
Un -1
U2U
1A
is upper triangular.
PROOF. We do this in the ususal way by a deflation process. We examine the first column a1of A and find aHouseholder matrix U
1such that
U1a
1=
(
0
0
.
Here U1is found as in the previous part so that
U1a
1= ||a||e
1.
Then we have that
U1A = [U
1a
1, U
1a
2, , U
1a
n]
where aiis the ith column of i. So we have that
U1a =
b11 b12 b1n
0 b22 b2n
0 bnw b2nnn
= B.
We find an (n - 1) "(n - 1) dimensional Householder matrix U%2with
U%2B =
c22 c23 c2n
0 c33 c3n
0 cn3 cnn
.
We then have that
U%2= 1
n -1- 2a!a
for some a in subspace spanned by e2, , e
n. So we have that
U2=
1 0
0 U%2
= 1 - 2&a!a
is a Householder matrix. But now we have that
Householder Matrices page 4
7/21/2019 Householder Matrices
5/10
U2B =
1 0
0 U%2
b11 b12 b1n
0 b22 b2n
0 bn2 b2n
=
b11 b12 b1n
0 c22 c23 c2n
0 c33 c3n
0
0 0 cn3 cnn
.
So we have set up a recurrence and we can get to upper triangular form in n - 1 steps. Q.E.D.
Now we show that we can reach Hessenberg form with n - 1 successive transforms
ad Ui
where Uiare Householder matrices. We need a little modification of the preceding zeroing-out Lemma to preserve
the Hessenberg form under the right multiplication of Ui.
LEMMA. Let a = (a1, , an)Tbe a vector in n. Then there is a unit vector w (respectively, w%) of the form such
(0, w2, , w
n) such that the Householder matrix Uwsatisfies
Uwa = (a1, ||a%||, 0, , 0)T
(respectively, Uw%a = (a1, - ||a%||, 0, , 0)T) where a%= (a2, , an)).
PROOF. This a modification of the previous zeroing-out lemma. By the previous zeroing out lemma, there is a
unit vector
w%= ( w1, , wn - 1)
in n - 1such that
Uw%a%= (||a%||, 0, , 0)T.
However, the matrix
U =1 0
0 Uw%
is a Householder matrix for the unit vector
w = (0, w1, , wn - 1)
in n. To verify this we only need to compute as follows:
1 - 2(w!w) =
1 0 0
0 1 0
0 0 1
- 2
0
w1
wn - 1
0, w1, , wn - 1
Householder Matrices page 5
7/21/2019 Householder Matrices
6/10
=
1 0 0
0 1- 2w1w1 - 2w1wn - 1
0 - 2wn -1w1 1- 2wn -11wn - 1
=1 0
0 Uw%
.
Finally we have that
Ua =a1
Uw%a%= (a1, ||a%||, 0, , 0)
T.
Q.E.D.
Now we have the reduction to upper Hessenberg form.
THEOREM. Let A be an n " n matrix. Then there are at most n - 1 Householder matrices U1, , Un - 1
such that Un - 1Un - 2U1AU1Un - 2Un - 1is a Hessenberg matrix. If A is selfadjoint, then
Un - 1Un - 2U1AU1Un - 2Un - 1
is a tridiagonal matrix.
PROOF. We start a recurrence using the previous zeroing-out lemma. We can find a Householder matrix
U1= Uw1with w1 a unit vector of the form
w1= (0, w12, , w1n)
such that U1a1has the form
U1a1= (($ ($0, , 0)T.
Here a1is the first column of A and in general aiis column i of A. Since U1has the form
U1=
1 0
0 Uw%
we get that
U1A = [U1a1, , U1an] =
* * * *
* * * *
0 * * *
0 * * *
.
Consequently, we have that
Householder Matrices page 6
7/21/2019 Householder Matrices
7/10
U1AU
1= (U
1A)U
1=
* * * * ** * * * *0 * * * *
0 * * * *
0 * * * *
1 0
0 Uw%
=
b11 b12 b13 b14 b1n
b11 b22 b33 b34 b2n
0 b32 b33 b34 b3n
0 b42 b43 b44 b4n
0 bn2 bn3 bn4 bnn
.
Here (indicates that the place is filled with an entry which may not be 0.
Now we do the recurrence step l Using the matrix
B =
b 22 b 23 b 24 % b 2n
b 32 b 33 b 34 % b 3n
b 42 b 43 b 44 % b 4n
% % % % %
b n2 b n3 b n4 % b nn
,
there is a unit vector w%%' n -1with 0 in the first coordinate such that
Uw%%
b 11 b 12 b 13 b 14 % b 1n
b 21 b 22 b 23 b 24 %
b 2n
0 b 32 b 33 b 34 % b 3n
0 b 42 b 43 b 44 % b 4n
% % % % % %
0 b n2 b n3 b n4 % b nn
Uw%%=
c 11 c 12 c 13 % c 1n
c 21 c 22 c 23 % c 2n
0 c 32 c 33 % c 3n
% % % % %
0 c n2 c n3 % c nn
Since the first coordinate of w %%is 0, we have as before that
Uw%%=1 0
0 Uw%%%
and that the n "n matrix
U2=
1 0 0
0 1 0
0 0 Uw%%
is also a Householder matrix. Here w%%%are the last n - 2 coordinates of w%%. Pre and post multiplying U1AU
1by U
2
does not disturb the first column. So we have that
U2U
1AU
1U
2= U
2
b 11 b 12 b 13 b 14 % b 1n
b 21 b 22 b 23 b 24 % b 2n
0 b 32 b 33 b 34 % b 3n
0 b 42 b 43 b 44 %
b 4n
% % % % % %
0 b n2 b n3 b n4 % b nn
U2
=
b 11 c 12 c 13 c 14 % c 1n
c 21 c 22 c 23 c 24 % c 2n
0 c 32 c 33 c 34 % c 3n
0 0 c
43
c
44
% c
4n% % % % % %
0 0 c n3 c n4 % c nn
.
So now we get the proof of the theorem from recurrence.
If A s selfadjoint, then U1A has first column equal to (b
11$ b
21, 0 , 0)Tand has first row equal to the original
first row of A. So we have ththe first row of
Householder Matrices page 7
7/21/2019 Householder Matrices
8/10
U1AU
1= (U
1A)U
1
is equal to
(b11$ b
21, 0 , 0).
So we are heading to a tridiagonal matrix and the further iterations confirm this Q.E.D.
Homework
Find Householder matrices U1and U2such that U2U1AU1U2is tridiagonal where
A =
120 80 40 - 16
80 120 16 -40
40 16 120 -80
-16 -40 -80 120
.
Householder Matrices page 8
7/21/2019 Householder Matrices
9/10
Homework 9
Find Householder matrices U1and U2such that U2U1AU1U2is tridiagonal where
A =
120 80 40 - 16
80 120 16 -40
40 16 120 -80
-16 -40 -80 120
.
SOLUTION. We let
w%1=1
|| a%1- ||a%1||e1||a%1- ||a%1||e1
where a1is the first column of A and a%1= (80, 40, -16)T. We calculate
||a%1|| =
80
40
16
&
80
40
16
= 90.863
w%1=80
40
16
- 90.863
0
0
=10.863
40
16
and
w1=1
02
+ (10.863)2
+ 402+ (- 16)
2
0
10.863
40
16
=
0
0.24449
0.9003
0.36012
.
This gives the first Householder matrix as
U1=
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
2w1 w1T
=
1 0 0 0
0 0.88045 0.44023 0.17609
0 0.44023 0.62108 0.64843
0 0.17609 0.64843 0.74063
.
We then have
U1 A U1=
120 90.863 0 0
90.863 157.21 56.97 36.378
0 56.97 152.85 1.0047
0 36.378 1.0047 49.94
.
Now we find U2. We have that
56.97
36.378&
56.97
36.378= 67.59
and
Householder Matrices page 9
7/21/2019 Householder Matrices
10/10
0
0
56.97
36.378
0
0
67.594
0
=
0
0
124.56
36.378
so that
1
0
0
124.56
36.378
&
0
0
124.56
36.378
00
124.56
36.378
00
124.56
36.378
=
0 0 0 00 0 0 0
0 0 0.92141 0.26909
0 0 0.26909 0.078588
with
U
2
=
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
2
0 0 0 0
0 0 0 0
0 0 0.92141 0.26909
0 0 0.26909 0.078588
=
1 0 0 0
0 1 0 0
0 0 0.84282 0.53819
0 0 0.53819 0.84282
.
This produces
U2U1AU1U2=
120 90.863 0 0
90.863 157.21 67.594 0
0 67.594 123.95 46.257
0 0 46.257 78.836
which is tridiagonal.
Householder Matrices page 10