Determinant

Alexander Litvinenko

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

http://sri-uq.kaust.edu.sa/

Extreme Computing Research Center, KAUST

Alexander Litvinenko Determinant

4*

The structure of the talk

1. Motivation2. History3. Geometrical interpretation4. Properties5. Definition

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

2

4*

History

1. A notion similar to determinant appears in old-chinesebook ”The Nine Chapters on the Mathematical Art”,10th-2nd century BCE

2. for 2× 2 matrices — by Girolamo Cardano in XVI century,3. Japanese mathematician Seki Takakazu, 16834. for higher dimensions by Gottfried W. Leibnitz in 1693,

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

3

4*

Geometrical Interpretation

Matrix is a linear transformation which squish space down orstretch it out

Determinant measures how the volume of the original do-main changed.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

4

4*

Motivation from spatial statistics

Goal: To improve estimation of unknown statistical parametersin a spatial soil moisture field, Mississippi basin,[−84.8◦ − 72.9◦]× [32.446◦,43.4044◦].

Log-likelihood function:

L(θ) = −n2

log(2π)− 12

log |C(θ)| − 12

Z>C(θ)−1Z .

where C = e−|x−y|

θ is a large matrix and Z available (satellite)data.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

5

4*

Properties. Part I.

1. det(I) = |I| = 1, I -identity matrix2. exchange two rows: reverse the sign of det(A)

3. ∣∣∣∣ ta tbc d

∣∣∣∣ = t∣∣∣∣ a b

c d

∣∣∣∣4. ∣∣∣∣ a + a′ b + b′

c d

∣∣∣∣ =

∣∣∣∣ a bc d

∣∣∣∣+

∣∣∣∣ a′ b′

c d

∣∣∣∣5. two equal rows result in det(A) = 0 (by P2)

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

6

4*

Properties. Part II.

6. Subtract `× row-i from row k , then det(A) doesn’t change(P4, P3, P5)

7. row of zeros, results in det(A) = 0 (P3 with t = 0)8.

det(A) =

∣∣∣∣∣∣∣∣∣d1 ∗ ∗ ∗0 d2 ∗ ∗...

.... . . ∗

0 0 0 dn

∣∣∣∣∣∣∣∣∣ = d1d2 · ... · dn

(P6, P3, P1)

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

7

4*

Properties III

9. det(A) = 0 exactly when A is singular AND det(A) 6= 0when A is non-singular

10. det(A · B) = det(A) · det(B);learning thatdet(A−1) = 1

det(A)anddet(cA) = cn det(A) (volume!)

11. det(AT ) = det(A)

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

8

4*

Definitions

for n = 2 : det(A) =

∣∣∣∣ a bc d

∣∣∣∣ =

∣∣∣∣ a 0c d

∣∣∣∣+

∣∣∣∣ 0 bc d

∣∣∣∣=

∣∣∣∣ a 0c 0

∣∣∣∣+

∣∣∣∣ a 00 d

∣∣∣∣+

∣∣∣∣ 0 bc 0

∣∣∣∣+

∣∣∣∣ 0 b0 d

∣∣∣∣= ad − bc.

for n = 3 : det(A) =

will have 27 terms , many of them will be zeros.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

9

4*

Formula for 3× 3 case

for n = 3 : det(A) =

∣∣∣∣∣∣a b cd e fg h i

∣∣∣∣∣∣ = a(ei − fh)− b(di − fg) + c(dh − eg)

= a ·∣∣∣∣ e f

h i

∣∣∣∣− b ·∣∣∣∣ d f

g i

∣∣∣∣+ c ·∣∣∣∣ d e

g h

∣∣∣∣Here we colored with red: the (1,1)-minor, (1,2)-minor and(1,3)-minor correspondingly.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

10

4*

Formula for n × n

If A is n × n matrix, then |A| =∑n

j=1(−1)(1+j)a1j |A1j |.

Here A1j is (n− 1)× (n− 1) matrix, obtained by deleting the 1strow and jth column of A.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

11

4*

Other applications

|Σ| is used inI Kullback-Leibler divergence (KLD) (distance between two

Gaussian distributions):

2DKL = tr(Σ−11 Σ0)+(µ1−µ0)T Σ−1

1 (µ1−µ0)−k− ln(|Σ0||Σ1|

)I The entropy of the multivariate normal distribution is

proportional to |Σ|.I Transformation of coordinates.I Multivariate statistics.I Google: eigenvalues of A are the solutions of the

characteristic equation |A− λI| = 0.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

12

4*

Conclusion

We learned

I Geometrical interpretation of the determinant,I Used properties P1-P4 to derive properties P5-P11I Used properties P1-P11 to derive formulas for

determinantsI Applications

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

13

4*

Literature

1. youtube lecture N18, of Prof. Gilbert Strang, MIT2. The determinant. Essence of linear algebra, chapter 5,https://www.youtube.com/watch?v=Ip3X9LOh2dk&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab&index=73. Harville, D. A. (1997). Matrix Algebra From a Statistician’sPerspective. Springer-Verlag.4. Brookes, M. (2005). ”The Matrix Reference Manual (online)”.5. Ding, J., Zhou, A. (2007). ”Eigenvalues of rank-one updatedmatrices with some applications”. Applied Mathematics Letters. 20(12): 1223-1226.

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

14

4*

Advanced Properties. Part III.

Let A be n × n matrix

13. det(exp(A)) = exp(tr(A)) or tr(A) = log det(exp(A))

14. for positive A, have tr(I − A−1) ≤ log det(A) ≤ tr(A− I)

15. det(

A 0C D

)= det(A) det(D)

16. if A−1 exist det(

A BC D

)= det(A) det(D − CA−1B)

17. det(Im + abT ) = 1 + (a,b)

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

15

4*

Advanced Properties. Part IV.

Let A be n × n matrix

18. Matrix determinant Lemma

det(A + abT ) = det(A(I + bT A−1a)) = (1 + bT A−1a) det(A)

19. U,V is n ×m

det(A + UV T ) = det(A(I + V T A−1U)) det(A)

20. if W is m ×m invertible

det(A + UWV T ) = det(W−1 + V T A−1U)) det(W ) det(A).

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

16

4*

Advanced Properties. Part V.

21. det(A) = 0 if and only if rank(A) < n,22. A−1 exists if and only if det(A) 6= 0,23. adding to a row/column a linear combination of any other

rows/columns does not change det(A)

24. if two (or more) rows/colums are linear dependent, thendet(A) = 0

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

17

4*

Formula for 4× 4 and n × n

|A| =

∣∣∣∣∣∣∣∣a b c de f g hi j k `m n o p

∣∣∣∣∣∣∣∣ = a

∣∣∣∣∣∣f g hj k `n o p

∣∣∣∣∣∣− e

∣∣∣∣∣∣b c dj k `n o p

∣∣∣∣∣∣+ i

∣∣∣∣∣∣b c df g hn o p

∣∣∣∣∣∣−m

∣∣∣∣∣∣b c df g hj k `

∣∣∣∣∣∣

Center for UncertaintyQuantification

Center for UncertaintyQuantification

Center for Uncertainty Quantification Logo Lock-up

18