Lecture 37: Quick review from previous lecture

• Suppose that A 2 Mm⇥n, b 2 Rm. Let x⇤ = A+b. Then x⇤ is the leastsquares solution to the linear system Ax = b.

If A has n linearly independent columns(kerA = {0}), then

x⇤ = A+b = (ATA)�1ATb,

whereA+ = (ATA)�1AT.

• If A =Pr

j=1 �jujvTj , then the matrix Ak =

Pkj=1 �jujv

Tj , k < r minimizes

the distance to A, that is,

minrank(B)=k

kA� Bk2 = kA� Akk2.

—————————————————————————————————Today we will discuss condition number and incomplete matrix.

- Lecture will be recorded -

—————————————————————————————————

• Information for Final Exam and Course Grade has been posted on Canvas, see”Announcements”.

MATH 4242-Week 15-2 1 Spring 2020

O-I

=

A =Amxn with rank CA ) =r.

ai. in . -- union.tl : fi:÷÷÷v÷...= la - - - um ] µ÷÷÷÷ ) = aunit . . - tourist#

= O, U. V,Tt -- - t Tr Ur Vst

.

= ¥, Tj Uj Vjt

§ Condition numberA very useful quantity for understanding the behavior of a matrix is its conditionnumber.

For simplicity, let’s take a square matrix A = An⇥n, and suppose the rank isr = n (so the matrix is nonsingular). Call the singular values

�1 � �2 � · · · � �n > 0.

Definition: The condition number of a nonsingular n ⇥ n matrix is theratio between its largest and smallest singular values, namely,

(A) =�1�n

.

The condition number (A) measures the “sensitivity of operations” we performwith A to changes in the input data.

Example. Suppose we apply A to a vector x. Write the SVD A = U⌃V T ; thenwe can write

Ax =nX

i=1

�ihx,viiui

What if we perturb x a little bit, along the direction of v1? How much wouldthe product Ax change?

MATH 4242-Week 15-2 2 Spring 2020

-

=-

= =

-

S..

I -- xtsvsyep.AI-ax-sr.ir.=

jvjt

× I *

AxAx - j.E.9-ujy.TL'' II. 9- ex- Vj>y. .

Pick a small number 8 , define I = X t SV , .=

Then AI = A ( x + su , )

= Ax + S Afv, ) Av, = Ioi ( vi. Vi >Ui --JI= AX +80

, U ,

so, 11 AI - A x 112=11 So, 4.1/2 = 181 T, since Hu . 112=1

.

The "related"between Ax and AI is the ratio

"AI-n.IT# =

.QKX, V, >

'

tr -- tonCx, ✓u>2

[Continue]

We have shown the following:

Fact: For certain vectors x, perturbing x by relative amount � along v1 canresult in a change in Ax by relative amount |�|(A).

So, if (A) is very large, then small changes in x can result in large changes inAx.

Matrices with large condition numbers are said to be ill-conditioned. Onemust be careful when working with ill-conditioned matrices, since small errors ininputs can become inflated.

MATH 4242-Week 15-2 3 Spring 2020

For example , we take x=V Then ( x,v; > =D if it n .

So,the relative difference is

11 A E - Axl,ta

- 181mL = 151k€On the other hand

,

"Ej*× =k = Isl.

since 11414=1

- -

Fact: If A is n⇥n nonsingular matrix, then A and A�1 have the same conditionnumber.

Fact: If A is n⇥ n nonsingular matrix, then

(A) = kAk2kA�1k2.

MATH 4242-Week 15-2 4 Spring 2020

[ To see this ] Since A is nxn nonsmgulav matrix,f-here are singular values 0, Z Tz z - - - Z Jn 70

.

Then singular values of A " are

or a. f. e - - - Eton .

So. KCA ) = I

Ju .

KIA" ) = %

,

= Ion .

Thus.HAKKA'¥

It All , = T, ( largest singular value of A )

" A- ' Il.= In ( s of A")

.

HAH,Ils-' th = f÷ -

-KCAL #.

8.6 Incomplete Matrices

If a matrix is not complete, then it is not similar to a diagonal matrix.For example,

A =

✓2 10 2

◆

is not complete and so is not diagonalizable.

However, it can be shown that a not complete matrix is similar to a matrix ofthe following form, called a Jordan matrix:

Here, each matrix J�i,ni is a Jordan block matrix, which is the followingni-by-ni matrix:

J�i,ni =

0

BBBBBBB@

�i 1�i 1

�i 1. . . . . .

�i 1�i

1

CCCCCCCA

The numbers �1, . . . ,�k are the eigenvalues of A. The eigenvalue �i has multi-plicity ni, the size of its Jordan block J�i,ni.

Note that if each Jordan block were 1-by-1 (i.e. ni = 1 for all i), then the Jordanmatrix would be diagonal, and A would be complete.

MATH 4242-Week 15-2 5 Spring 2020

,M = 2

,2

.

A -ZI =/ ? :)

Recall : A diagonalize able matrix A can be written as A = VDV".

where D= ( ! " :p! ).

A is similar to D,

a

-

§ The Jordan Canonical Form. Let’s see how we get this Jordan matrix.Throughout this section, we suppose that A is an n⇥ n matrix. Let

�1, . . . ,�k

be the eigenvalues of A.

Definition: A Jordan chain of length j for a square matrix A is a sequenceof nonzero vectors w1, . . . ,wj that satisfies

Aw1 = �w1, Awi = �wi +wi�1, i = 2, . . . , j,

where � is an eigenvalue of A.

The initial vector w1 in a Jordan chain is a “genuine” eigenvector. The restw2, . . . ,wj are generalized eigenvectors, in the following definition:

Definition: A nonzero vector w 6= 0 such that

(A� �I)kw = 0 for some k > 0

and � is a generalized eigenvector of the matrix A.

MATH 4242-Week 15-2 6 Spring 2020

-

EI: g- =3 ,( A - XI ) w. -- O

.

(A -RI ) wz = w . ⇒ CA -AIT w, -_CA- a -4W,= 0

( A -RI) Ws = Wz ⇒ (A - A -43W, = O.

A- Cw . wit (I ⑤ IfAws = Ahhh +1¥

Example. Find the Jordan form of A =

✓1 1�1 3

◆

Example. Find the Jordan form of A =

0

@�1 �1 00 �1 �20 0 �1

1

A

MATH 4242-Week 15-2 7 Spring 2020

O = doe ( A - 172 ) , a = I, I .

A- 2 I = ( I, f ) .

dm@CA- 22 )) = I.

Jordan t.info i ).

*

A = - l,-1,

- l.

dm ( NC At Il ) = I.

Jordan form is f ! ! !)--

#