View
2
Download
0
Category
Preview:
Citation preview
Math 504 Fall 2016 NotesWeek 3, Lecture 1
Emre Mengi
Department of MathematicsKoç University
Istanbul, Turkey
Emre Mengi Week 3, Lecture 1
Outline
Orthogonal Rank 1 Representation
Column Space, Rank, Null Space in terms of SVD
Matrix Norms
Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
It follows from the SVD
A = UΣV ∗
where
U =[
u(1) . . . u(m)],
V =[
v (1) . . . v (n)],
Σ = diag(σ1, σ2, . . . , σp)m×n
that
A = σ1u(1)[v (1)
]∗+ σ2u(2)
[v (2)
]∗+ · · ·+ σpu(p)
[v (p)
]∗where p := min{m,n}.
Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
A = σ1u(1)[v (1)
]∗+ σ2u(2)
[v (2)
]∗+ · · ·+ σpu(p)
[v (p)
]∗Rank
(u(j) [v (j)]∗) = 1 for j = 1, . . . ,p.(
u(j) [v (j)]∗)∗ u(k) [v (k)]∗ = 0, whenever j 6= k .
Example
A =
[1 −33 −1
]=
[1/√
2 −1/√
21/√
2 1/√
2
] [4 00 2
] [1/√
2 −1/√
21/√
2 1/√
2
]= 4
[1/√
21/√
2
] [1/√
2 −1/√
2]
+ 2[−1/√
21/√
2
] [1/√
2 1/√
2]
=
[2 −22 −2
]+
[−1 −1
1 1
]Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
A = σ1u(1)[v (1)
]∗+ σ2u(2)
[v (2)
]∗+ · · ·+ σpu(p)
[v (p)
]∗Rank
(u(j) [v (j)]∗) = 1 for j = 1, . . . ,p.(
u(j) [v (j)]∗)∗ u(k) [v (k)]∗ = 0, whenever j 6= k .
Example
A =
[1 −33 −1
]=
[1/√
2 −1/√
21/√
2 1/√
2
] [4 00 2
] [1/√
2 −1/√
21/√
2 1/√
2
]= 4
[1/√
21/√
2
] [1/√
2 −1/√
2]
+ 2[−1/√
21/√
2
] [1/√
2 1/√
2]
=
[2 −22 −2
]+
[−1 −1
1 1
]Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
A = σ1u(1)[v (1)
]∗+ σ2u(2)
[v (2)
]∗+ · · ·+ σpu(p)
[v (p)
]∗Rank
(u(j) [v (j)]∗) = 1 for j = 1, . . . ,p.(
u(j) [v (j)]∗)∗ u(k) [v (k)]∗ = 0, whenever j 6= k .
Example
A =
[1 −33 −1
]=
[1/√
2 −1/√
21/√
2 1/√
2
] [4 00 2
] [1/√
2 −1/√
21/√
2 1/√
2
]= 4
[1/√
21/√
2
] [1/√
2 −1/√
2]
+ 2[−1/√
21/√
2
] [1/√
2 1/√
2]
=
[2 −22 −2
]+
[−1 −1
1 1
]Emre Mengi Week 3, Lecture 1
Orthogonal Rank 1 Representation
A = σ1u(1)[v (1)
]∗+ σ2u(2)
[v (2)
]∗+ · · ·+ σpu(p)
[v (p)
]∗Rank
(u(j) [v (j)]∗) = 1 for j = 1, . . . ,p.(
u(j) [v (j)]∗)∗ u(k) [v (k)]∗ = 0, whenever j 6= k .
Example
A =
[1 −33 −1
]=
[1/√
2 −1/√
21/√
2 1/√
2
] [4 00 2
] [1/√
2 −1/√
21/√
2 1/√
2
]= 4
[1/√
21/√
2
] [1/√
2 −1/√
2]
+ 2[−1/√
21/√
2
] [1/√
2 1/√
2]
=
[2 −22 −2
]+
[−1 −1
1 1
]Emre Mengi Week 3, Lecture 1
Column and Null Spaces, Rank in terms of SVD
Emre Mengi Week 3, Lecture 1
Column Space in terms of SVD
Sort the singular values of A ∈ Cm×n from the largest to thesmallest. (Below p := min{m,n}.)
σ1 ≥ σ2 ≥ · · · ≥ σr > σr+1 = · · · = σp = 0
If all singular values are nonzero, then r = p and
σ1 ≥ σ2 ≥ · · · ≥ σp > 0.
Column Space in terms of SVD
Col(A) = {Ax | x ∈ Cn}= span{u(1),u(2), . . . ,u(r)}
In other words, {u(1),u(2), . . . ,u(r)} is an orthonormalbasis for Col(A).
Emre Mengi Week 3, Lecture 1
Column Space in terms of SVD
Sort the singular values of A ∈ Cm×n from the largest to thesmallest. (Below p := min{m,n}.)
σ1 ≥ σ2 ≥ · · · ≥ σr > σr+1 = · · · = σp = 0
If all singular values are nonzero, then r = p and
σ1 ≥ σ2 ≥ · · · ≥ σp > 0.
Column Space in terms of SVD
Col(A) = {Ax | x ∈ Cn}= span{u(1),u(2), . . . ,u(r)}
In other words, {u(1),u(2), . . . ,u(r)} is an orthonormalbasis for Col(A).
Emre Mengi Week 3, Lecture 1
Column Space in terms of SVD
Sort the singular values of A ∈ Cm×n from the largest to thesmallest. (Below p := min{m,n}.)
σ1 ≥ σ2 ≥ · · · ≥ σr > σr+1 = · · · = σp = 0
If all singular values are nonzero, then r = p and
σ1 ≥ σ2 ≥ · · · ≥ σp > 0.
Column Space in terms of SVD
Col(A) = {Ax | x ∈ Cn}= span{u(1),u(2), . . . ,u(r)}
In other words, {u(1),u(2), . . . ,u(r)} is an orthonormalbasis for Col(A).
Emre Mengi Week 3, Lecture 1
Rank in terms of SVD
Rank in terms of SVDRank(A) = # of nonzero singular values.
Example
A =
10 54 −32 14
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 5
√3
0 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 5√
3 > 0, so
{u(1), u(2)
}=
1/
√2
01/√
2
, 1/
√3
1/√
3−1/√
3
an orthonormal basis for Col(A). Furthermore, Rank(A) = 2.
Emre Mengi Week 3, Lecture 1
Rank in terms of SVD
Rank in terms of SVDRank(A) = # of nonzero singular values.
Example
A =
10 54 −32 14
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 5
√3
0 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 5√
3 > 0, so
{u(1), u(2)
}=
1/
√2
01/√
2
, 1/
√3
1/√
3−1/√
3
an orthonormal basis for Col(A). Furthermore, Rank(A) = 2.
Emre Mengi Week 3, Lecture 1
Rank in terms of SVD
Rank in terms of SVDRank(A) = # of nonzero singular values.
Example
A =
10 54 −32 14
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 5
√3
0 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 5√
3 > 0, so
{u(1), u(2)
}=
1/
√2
01/√
2
, 1/
√3
1/√
3−1/√
3
an orthonormal basis for Col(A). Furthermore, Rank(A) = 2.
Emre Mengi Week 3, Lecture 1
Rank in terms of SVD
Rank in terms of SVDRank(A) = # of nonzero singular values.
Example
A =
10 54 −32 14
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 5
√3
0 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 5√
3 > 0, so
{u(1), u(2)
}=
1/
√2
01/√
2
, 1/
√3
1/√
3−1/√
3
an orthonormal basis for Col(A). Furthermore, Rank(A) = 2.
Emre Mengi Week 3, Lecture 1
Rank in terms of SVD
Rank in terms of SVDRank(A) = # of nonzero singular values.
Example
A =
10 54 −32 14
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 5
√3
0 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 5√
3 > 0, so
{u(1), u(2)
}=
1/
√2
01/√
2
, 1/
√3
1/√
3−1/√
3
an orthonormal basis for Col(A). Furthermore, Rank(A) = 2.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVD
Null space of A ∈ Cm×n is defined by
Null(A) := {x ∈ Cn | Ax = 0}.
Example
A =
[1 −11 −1
], Null(A) = span
{[11
]}
Proposition
Null(A) is a subspace of Cn.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVD
Null space of A ∈ Cm×n is defined by
Null(A) := {x ∈ Cn | Ax = 0}.
Example
A =
[1 −11 −1
], Null(A) = span
{[11
]}
Proposition
Null(A) is a subspace of Cn.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVD
Null space of A ∈ Cm×n is defined by
Null(A) := {x ∈ Cn | Ax = 0}.
Example
A =
[1 −11 −1
], Null(A) = span
{[11
]}
Proposition
Null(A) is a subspace of Cn.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVD
Sort the singular values of A ∈ Cm×n from the largest to thesmallest. (Below p := min{m,n}.)
σ1 ≥ σ2 ≥ · · · ≥ σr > σr+1 = · · · = σp = 0
If all singular values are nonzero, then r = p and
σ1 ≥ σ2 ≥ · · · ≥ σp > 0.
Null Space in terms of SVD
Null(A) = {x ∈ Cn | Ax = 0}= span{v (r+1), v (2), . . . , v (n)}
In other words, {v (r+1), v (2), . . . , v (n)} is anorthonormal basis for Null(A).
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVD
Sort the singular values of A ∈ Cm×n from the largest to thesmallest. (Below p := min{m,n}.)
σ1 ≥ σ2 ≥ · · · ≥ σr > σr+1 = · · · = σp = 0
If all singular values are nonzero, then r = p and
σ1 ≥ σ2 ≥ · · · ≥ σp > 0.
Null Space in terms of SVD
Null(A) = {x ∈ Cn | Ax = 0}= span{v (r+1), v (2), . . . , v (n)}
In other words, {v (r+1), v (2), . . . , v (n)} is anorthonormal basis for Null(A).
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVDExample
A =
6 80 06 8
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 00 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 0, so{v (2)}=
{[4/5−3/5
]}an orthonormal basis for Null(A).
Corollary (Rank-Nullity)
For a matrix A ∈ Cm×n, we have
Rank(A) + dim Null(A) = n.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVDExample
A =
6 80 06 8
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 00 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 0, so{v (2)}=
{[4/5−3/5
]}an orthonormal basis for Null(A).
Corollary (Rank-Nullity)
For a matrix A ∈ Cm×n, we have
Rank(A) + dim Null(A) = n.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVDExample
A =
6 80 06 8
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 00 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 0, so{v (2)}=
{[4/5−3/5
]}an orthonormal basis for Null(A).
Corollary (Rank-Nullity)
For a matrix A ∈ Cm×n, we have
Rank(A) + dim Null(A) = n.
Emre Mengi Week 3, Lecture 1
Null Space in terms of SVDExample
A =
6 80 06 8
=
1/√
2 1/√
3 1/√
60 1/
√3 −2/
√6
1/√
2 −1/√
3 −1/√
6
10√
2 00 00 0
[ 3/5 4/54/5 −3/5
].
We have σ1 = 10√
2, σ2 = 0, so{v (2)}=
{[4/5−3/5
]}an orthonormal basis for Null(A).
Corollary (Rank-Nullity)
For a matrix A ∈ Cm×n, we have
Rank(A) + dim Null(A) = n.
Emre Mengi Week 3, Lecture 1
Matrix Norms
Emre Mengi Week 3, Lecture 1
Common Matrix Norms
Let A ∈ Cm×n.
Frobenius norm
‖A‖F :=
√√√√ m∑j=1
n∑k=1
|ajk |2
2-norm‖A‖2 := max
x∈Cn,‖x‖2=1‖Ax‖2
p-norm (p ∈ R, p ≥ 1)
‖A‖p := maxx∈Cn,‖x‖p=1
‖Ax‖p
Emre Mengi Week 3, Lecture 1
Common Matrix Norms
Let A ∈ Cm×n.
Frobenius norm
‖A‖F :=
√√√√ m∑j=1
n∑k=1
|ajk |2
2-norm‖A‖2 := max
x∈Cn,‖x‖2=1‖Ax‖2
p-norm (p ∈ R, p ≥ 1)
‖A‖p := maxx∈Cn,‖x‖p=1
‖Ax‖p
Emre Mengi Week 3, Lecture 1
Common Matrix Norms
Let A ∈ Cm×n.
Frobenius norm
‖A‖F :=
√√√√ m∑j=1
n∑k=1
|ajk |2
2-norm‖A‖2 := max
x∈Cn,‖x‖2=1‖Ax‖2
p-norm (p ∈ R, p ≥ 1)
‖A‖p := maxx∈Cn,‖x‖p=1
‖Ax‖p
Emre Mengi Week 3, Lecture 1
Characterizations
For a matrix A ∈ Cm×n of the form
A =[
a(1) a(2) . . . a(n)]
=
[a(1)]T[a(2)]T
...[a(m)
]T
we have
‖A‖1 = max{‖a(1)‖1, ‖a(2)‖1, . . . , ‖a(n)‖1},‖A‖2 = σ1 (largest singular value of A),
‖A‖∞ = max{‖a(1)‖1, ‖a(2)‖1, . . . , ‖a(m)‖1}
Emre Mengi Week 3, Lecture 1
Definition and Properties
A matrix norm ‖ · ‖ : Cm×n → R just like a vector norm satisfiesthe following properties.
(i) ‖A‖ > 0 for all nonzero A ∈ Cm×n
(ii) ‖αA‖ = |α|‖A‖ for all α ∈ C and all A ∈ Cm×n
(iii) ‖A + B‖ ≤ ‖A‖+ ‖B‖ for all A,B ∈ Cm×n
Additionally, most commonly used matrix norms on Cn×n alsosatisfy the following submultiplicative property.
‖AB‖ ≤ ‖A‖‖B‖ for all A,B ∈ Cn×n
Emre Mengi Week 3, Lecture 1
Definition and Properties
A matrix norm ‖ · ‖ : Cm×n → R just like a vector norm satisfiesthe following properties.
(i) ‖A‖ > 0 for all nonzero A ∈ Cm×n
(ii) ‖αA‖ = |α|‖A‖ for all α ∈ C and all A ∈ Cm×n
(iii) ‖A + B‖ ≤ ‖A‖+ ‖B‖ for all A,B ∈ Cm×n
Additionally, most commonly used matrix norms on Cn×n alsosatisfy the following submultiplicative property.
‖AB‖ ≤ ‖A‖‖B‖ for all A,B ∈ Cn×n
Emre Mengi Week 3, Lecture 1
Invariance of the 2-norms
Let U ∈ Cm×m and V ∈ Cn×n be unitary matrices.(i) For every v ∈ Cm, we have ‖Uv‖2 = ‖v‖2.
(ii) For every A ∈ Cm×n, we have ‖UAV‖2 = ‖A‖2.
Emre Mengi Week 3, Lecture 1
Recommended