Upload
evangeline-lucas
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
11
Matrix Algebra and Matrix Algebra and Random VectorsRandom Vectors
Shyh-Kang JengShyh-Kang JengDepartment of Electrical Engineering/Department of Electrical Engineering/Graduate Institute of Communication/Graduate Institute of Communication/
Graduate Institute of Networking and MultiGraduate Institute of Networking and Multimediamedia
22
General Statistical DistanceGeneral Statistical Distance
)])((2
))((2))((2
)(
)()([
),(
]222
[),(
),,,(),0,,0,0(),,,,(
11,1
331113221112
2
22222
21111
1,131132112
22222
2111
2121
pppppp
pppp
pppp
ppp
pp
yxyxa
yxyxayxyxa
yxa
yxayxa
QPd
xxaxxaxxa
xaxaxaPOd
yyyQOxxxP
33
General Statistical DistanceGeneral Statistical Distance
Axx'
]222
[),(
2
1
21
22212
11211
21
1,131132112
22222
2111
2
ppppp
p
p
p
pppp
ppp
x
x
x
aaa
aaa
aaa
xxx
xxaxxaxxa
xaxaxaPOd
44
Quadratic FormQuadratic Form
2332
2221
21
3
2
1
321
2221
21
2
121
1 1
246
220
213
031
)(
211
11)(
')(
xxxxxxx
x
x
x
xxxxQ
xxxxx
xxxxQ
xxaQk
i
k
jjiij
Axxx
55
Statistical Distance under Rotated Statistical Distance under Rotated Coordinate SystemCoordinate System
22222112
2111
212
211
22
22
11
21
21
2),(
cossin~sincos~
~
~
~
~),(
)~,~(),0,0(
xaxxaxaPOd
xxx
xxx
s
x
s
xPOd
xxPO
66
Rotated Coordinate SystemRotated Coordinate System
2
1
2
1
212
211
cossin
sincos~
~cossin~
sincos~
x
x
x
x
xxx
xxx
77
Coordinate TransformationCoordinate Transformation
x
b
b
b
y
Bxy
'
'2
'1
2
1
21
22221
11211
2
1
k
kkkkk
k
k
k x
x
x
bbb
bbb
bbb
y
y
y
88
Quadratic Form in Transformed Quadratic Form in Transformed Coordinate SystemsCoordinate Systems
ΛBAB
ΛABB
yΛyy'
yABBy
yBAyB
Axxx
'
)(
'
)()'(
')(
1
1'1
1'1
11
Q
Q
99
Diagonalized Quadratic FormDiagonalized Quadratic Form
k
iii
k
yQ1
2
2
1
')(
00
00
00
Λyyy
Λ
1010
Orthogonal MatrixOrthogonal Matrix
cossin
sincos
cossin
sincos
10
01
cossin
sincos
cossin
sincos
'
''
1
1 AA
IAAAA
1111
Diagonalization Diagonalization
iii
k
k
k
kkkk
k
k
aaa
aaa
aaa
bAb
bbb
bbb
ΛBAB
BB
00
00
00
''
'
2
1
21
21
21
22221
11211
1
1212
Concept of MappingConcept of Mapping
Axy x
1313
EigenvaluesEigenvalues
3,1
0)3)(1(31
01
31
01
0)(
21
IA
A
xIA
xAx
1414
EigenvectorsEigenvectors
1
0,
0
0
01
02
3
1
2,
0
0
21
00
1
0)(
22
1
2
12
1
1
x
x
xIA
x
x
x
x
ii
1515
Spectral DecompositionSpectral Decomposition
k
iiii
kk
k
1
'
'
'2
'1
2
1
21
00
00
00
'
''
ee
e
e
e
eeeA
ΛBBA
ΛBAB
1616
Positive Definite MatrixPositive Definite MatrixMatrix Matrix AA is non-negative definite if is non-negative definite if
for all for all xx’=[’=[xx11, , xx22, …, , …, xxkk]]Matrix A is positive definite if Matrix A is positive definite if
for all non-zero for all non-zero xx
0' Axx
0' Axx
1717
Positive Definite MatrixPositive Definite Matrix
definite positive is
' allfor 0'
' and orthogonal ,
','
04''4'
4
1,4
22
232223
1-
'222
'111
22
21
'22
'11
'22
'11
21
2
12121
22
21
A
0xAxx
EEEExy
xeexxeex
xeexxeexAxx
eeeeA
xAx
yy
yy
x
xxxxxxx
1818
Inverse and Square-Root MatrixInverse and Square-Root Matrix
12/12/1
2/12/1
2/12/1
2/1
1
'2/1
11
1
'
'
'
,,1,0
)'()')('(
'
,'
AA
AA
AAA
PPΛeeA
PPΛPΛΛPΛΛPΛΛA
PPΛA
k
iiii
i
nn
k
iiii
ki
PPeeA
1919
Random Vectors and Random Vectors and Random MatricesRandom Matrices
Random vectorRandom vector– Vector whose elements are random Vector whose elements are random
variablesvariables
Random matrixRandom matrix– Matrix whose elements are random Matrix whose elements are random
variablesvariables
2020
Expected Value of a Expected Value of a Random MatrixRandom Matrix
BXAAXB
X
)()(
)(
)()(
)()()(
)()()(
)()()(
)(
21
22221
11211
EE
xpx
dxxfxXE
XEXEXE
XEXEXE
XEXEXE
E
ijxallijijij
ijijijij
ij
pppp
p
p
2121
Population Mean VectorsPopulation Mean Vectors
piii
ii
i
p
p
E
XE
XE
xf
xxxff
XXX
21
22
21
21
)(
)(
)(
)(on distributiy probabilit Marginal
),,,()(
function density y probabilitJoint
' vector Random
Xμ
x
X
2222
CovarianceCovariance
ik
xall xallkiikkkii
kikiikkkii
kkiiki
i k
xxpxx
dxdxxxfxx
XXEXX
),())((
),())((
))((),Cov(
2323
Statistically IndependentStatistically Independent
general)in not true is converse (the
tindependen are , if 0),Cov(
)()()(),,,(
)()()(
][][] and[
22112112
,
kiki
pppp
kkiikiik
kkiikkii
XXXX
xfxfxfxxxf
xfxfxxf
xXPxXPxXxXP
2424
Population Variance-Covariance Population Variance-Covariance MatricesMatrices
pppp
p
p
pp
pp
XXX
X
X
X
E
E
21
22221
11211
221122
11
)Cov(
)')((
X
μXμXΣ
2525
Population Correlation CoefficientsPopulation Correlation Coefficients
1
21
22221
11211
ii
kkii
ikik
pppp
p
p
ρ
2626
Standard Deviation MatrixStandard Deviation Matrix
2/12/1
2/12/1
22
11
2/1
00
00
00
ΣVVρ
ρVVΣ
V
pp
2727
Correlation Matrix from Correlation Matrix from Covariance MatrixCovariance Matrix
15/15/1
5/116/1
5/16/11
5/100
03/10
002/1
500
030
002
00
00
00
2532
391
214
2/12/1
2/1
33
22
112/1
333231
232221
131211
ΣVVρ
V
V
Σ
2828
Partitioning Covariance MatrixPartitioning Covariance Matrix
)')(()')((
)')(()')((
)')((
,
)2()2()2()2()1()1()2()2(
)2()2()1()1()1()1()1()1(
)2(
)1(
)2(
)1(
1
1
μXμXμXμX
μXμXμXμX
μXμX
μ
μ
μ
X
X
X
p
q
q
X
X
X
X
2929
Partitioning Covariance MatrixPartitioning Covariance Matrix
'1221
1,1
,11,1,11,1
1,1
11,1111
2221
1211
|
|
|
|
|
|
|
|
|
|
)')((
ΣΣ
ΣΣ
ΣΣ
μXμXΣ
ppqppqp
pqqqqqq
qpqqqqq
pqq
E
3030
Linear Combinations of Linear Combinations of Random VariablesRandom Variables
)Cov( and )( where
')'Var( ariance v
')'(mean
has
'n combinatioLinear 11
XΣXμ
ΣccXc
μcXc
Xc
E
E
XcXc pp
3131
Example of Linear Combinations of Example of Linear Combinations of Random VariablesRandom Variables
ΣccXcVar
μcXc
Xc
'][)'(
')'(
],['],,['
2
)]()([
)]()[()Var(
)()()(
2212
1211
21
12222
112
22211
2212121
212121
b
aba
E
XXba
abba
XbXaE
babXaXEbXaX
baXbEXaEbXaXE
3232
Linear Combinations of Linear Combinations of Random VariablesRandom Variables
')Cov(
)()(
21
22221
11211
CCΣCXΣ
CμCXZμ
CXXZ
XZ
XZ
EE
ccc
ccc
ccc
pppp
p
p
3333
Sample Mean Vector and Sample Mean Vector and Covariance MatrixCovariance Matrix
n
jpjp
n
jpjpj
n
jpjpj
n
jj
ppp
p
n
p
xxn
xxxxn
xxxxn
xxn
ss
ss
xxx
1
2
111
111
1
211
1
111
21
11
11
],,['
S
x
3434
Partitioning Sample Mean VectorPartitioning Sample Mean Vector
)2(
)1(
1
1
x
x
x
p
q
q
x
x
x
x
3535
Partitioning Sample Partitioning Sample Covariance MatrixCovariance Matrix
'1221
1,1
,11,1,11,1
1,1
11,1111
2221
1211
|
|
|
|
|
|
|
|
|
|
SS
SS
SS
S
ppqppqp
pqqqqqq
qpqqqqq
pqq
n
ssss
ssss
ssss
ssss
3636
Cauchy-Schwarz InequalityCauchy-Schwarz Inequality
dd
dbbb
dd
dbdd
dd
dbbb
dddbbbdbdb
db
dbddbbdb
'
''0
'
')'(
'
''0
)'()'(2')()'(0
0
:Proof
hen equality w with ),')('()'(
2
22
2
2
x
xxxx
-x
c
3737
Extended Cauchy-Schwarz Extended Cauchy-Schwarz InequalityInequality
'
1
2/1
12
2/12/1'2/12/122/12/1
2/1'2/12/12/1
112
)')('()'(
''')''(
'''
:Proof
hen equality w with )')('('
definite positive
ii
p
ii
c
eeB
dBdBbbdb
dBBdbBBbdBbB
dBbBdBBbbIddb
dBbdBdBbbdb
B
3838
Maximization LemmaMaximization Lemma
dBd
Bxx
dx
Bxx
dBdBxxdx
dBx
dBdBxx
dx
dB
x
12
12
1
12
0
''
'
0'
)')('('
:Proof
0for when attained maximum
''
'max
orgiven vect matrix, definite positive
cc
3939
Maximization of Quadratic Forms Maximization of Quadratic Forms for Points on the Unit Spherefor Points on the Unit Sphere
) when (attained'
'max
) when (attained'
'min
) when (attained'
'max
,,,
rseigenvecto associated and 0
eseigen valuh matrix wit definite positive
11,,
0
110
21
21
1
kk
pp
p
p
k
exxx
Bxx
exxx
Bxx
exxx
Bxx
eee
B
eex
x
x
4040
Maximization of Quadratic Forms foMaximization of Quadratic Forms for Points on the Unit Spherer Points on the Unit Sphere
11
1111
1
1
2
1
2
1
1
2
1
2
2/12/12/12/1
2/12/1
'
'
'
'],001[)'(',
'
'
'
'''
''
'
'
'
','
:Proof
ee
Bee
yy
ΛyyPeyex
yy
Λyy
yy
xPPΛPPΛx
xPPx
xBBx
xx
Bxx
xPyPPΛB
p
ii
p
ii
p
ii
p
iii
y
y
y
y
4141
Maximization of Quadratic Forms foMaximization of Quadratic Forms for Points on the Unit Spherer Points on the Unit Sphere
maximum asserted
thegives0,1 Taking
'
'
,0
implies,,
21
1
1
2
1
2
'2
'21
'1
'
1
2211
pkk
kp
kii
p
kiii
ipipiii
k
pp
yyy
y
y
kiyyyy
yyy
xx
Bxx
eeeeeexe
eex
eeePyx
4242
Maximization of Quadratic Forms foMaximization of Quadratic Forms for Points on the Unit Spherer Points on the Unit Sphere
10
'0
0'0
,,,0,,,1'
0'0
0'0
01'
10
'0
0'0
01'
0'0
0'0
0'0
0
1001
0
0
max'max
min'min
max'max
',1'
k
p
kk
xx
BxxBxx
xx
BxxBxx
xx
BxxBxx
xx
BxxBxxxx
xx
xx
eexxeexxx
xxx
xxx