Eigenvalues, eigenvectors and applicationssuku/kozhikode.pdfMarkov Matrices Eigen value and eigen vector Problem Big Problem Getting a common opinion from individual opinion From individual

Linear transformations on planeEigen values

Markov Matrices

Eigenvalues, eigenvectors and applications

Dr. D. Sukumar

Department of MathematicsIndian Institute of Technology Hyderabad

Recent Trends in Applied Sciences with Engineering ApplicationsJune 27-29, 2013

Department of Applied ScienceGovernment Engineering College,Kozhikode, Kerala

Dr. D. Sukumar (IITH) Eigenvalues


Markov Matrices

Typical ExamplesProperties

Maps which preserve

Origin

lines passing through origin

parallelograms with one corner as origin



Markov Matrices


Outline

1 Linear transformations on planeTypical ExamplesProperties

2 Eigen valuesEigen value and eigen vector

3 Markov MatricesFormationInterpretationProperties



Markov Matrices


Rotation

(0 −11 0

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Rotation

(0 −11 0

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Reflection

(1 00 −1

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Reflection

(1 00 −1

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Expansion

(2 00 2

)Compression

(1/2 0

0 1/2

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2

x

y

-1 1 2-1

1

2



Markov Matrices


Expansion

(2 00 2

)Compression

(1/2 0

0 1/2

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2

x

y

-1 1 2-1

1

2



Markov Matrices


Expansion

(2 00 2

)Compression

(1/2 0

0 1/2

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2

x

y

-1 1 2-1

1

2



Markov Matrices


Multi-scaling or Stretching

(2 00 3

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Multi-scaling or Stretching

(2 00 3

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Projection

(1 00 0

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Projection

(1 00 0

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Shear transformation

(1 10 1

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Shear transformation

(1 10 1

)

-1 1 2-1

1

2

x

y

x

y

-1 1 2-1

1

2



Markov Matrices


Outline






Markov Matrices


Properties

Area

Eigen vectors

Eigen values

Determinant

Diagonalizable



Markov Matrices


Properties

Area

Eigen vectors

Eigen values

Determinant

Diagonalizable



Markov Matrices


Properties

Area

Eigen vectors

Eigen values

Determinant

Diagonalizable



Markov Matrices


Properties

Area

Eigen vectors

Eigen values

Determinant

Diagonalizable



Markov Matrices


Properties

Area

Eigen vectors

Eigen values

Determinant

Diagonalizable



Markov Matrices


Table of properties

Map Area Fixed Dir Scale in FD Det DiagonableEigenvector Eigenvalue

Rotation 1 NO NO 1 NO

Reflection 1 x-axis, y -axis 1,-1 -1 Yes

Expansion 4 x-axis, y -axis 2, 2 4 YesCompression 1/4 x-axis, y -axis 1/2,1/2 1/4 Yes

Multi-scaling 6 x-axis, y -axis 2,3 6 Yes

Projection 0 x-axis, y -axis 1,0 0 Yes

Shear 1 x-axis 1 1 NO

Table: Properties



Markov MatricesEigen value and eigen vector

Problem

Big Problem

Getting a common opinion from individual opinion

From individual preference to common preference

Purpose

Showing all steps of this process using linear algebra

Mainly using eigenvalues and eigenvectors




Problem

Big Problem



Purpose






Problem

Big Problem



Purpose






Problem

Big Problem



Purpose






Outline







Eigen value

Let A a be square matrix.

Eigen values of A are solutions or roots of

det(A− λI ) = 0.

IfAx = λx or (A− λI )x = 0,

for a non-zero vector x then

λ is an eigenvalue of A andx is an eigenvector corresponding to the eigenvalue λ.




Eigen value



det(A− λI ) = 0.







Eigen value



det(A− λI ) = 0.



λ is an eigenvalue of A and

x is an eigenvector corresponding to the eigenvalue λ.




Eigen value



det(A− λI ) = 0.







Example

Consider the matrix A =

[2 53 0

].

Example (Eigen value)

A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ

]det(A− λI ) = (2− λ)(−λ)− (3× 5) = λ2 − 2λ− 15The roots of the polynomial are the eigen values: -3 and 5.

Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11

)Example (When λ = 5)[

2− 5 53 −5

](x1x2

)=

(00

)Eigen vector

(53

)




Example


[2 53 0

].


A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ

]

det(A− λI ) = (2− λ)(−λ)− (3× 5) = λ2 − 2λ− 15The roots of the polynomial are the eigen values: -3 and 5.

Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11


2− 5 53 −5

](x1x2

)=

(00

)Eigen vector

(53

)




Example


[2 53 0

].


A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ

]det(A− λI ) = (2− λ)(−λ)− (3× 5) = λ2 − 2λ− 15

The roots of the polynomial are the eigen values: -3 and 5.

Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11


2− 5 53 −5

](x1x2

)=

(00

)Eigen vector

(53

)




Example


[2 53 0

].


A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ


Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11


2− 5 53 −5

](x1x2

)=

(00

)Eigen vector

(53

)




Example


[2 53 0

].


A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ


Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11

)

Example (When λ = 5)[2− 5 5

3 −5

](x1x2

)=

(00

)Eigen vector

(53

)




Example


[2 53 0

].


A− λI =

[2 53 0

]− λ

[1 00 1

]=

[2− λ 5

3 −λ


Example (When λ = −3)[2− (−3) 5

3 −(−3)

](x1x2

)=

(00

)Eigen vector

(−11


2− 5 53 −5

](x1x2

)=

(00

)Eigen vector

(53

)Dr. D. Sukumar (IITH) Eigenvalues



For the matrix A =

[2 53 0

]the eigenvalues are -3 and 5

the eigenvector corresponding to -3 is

[−11

][

2 53 0

] [−11

]= −3

[−11

]

the eigenvector corresponding to 5 is

[53

][

2 53 0

] [53

]=




For the matrix A =

[2 53 0



[−11

][

2 53 0

] [−11

]=

[3−3

]= −3

[−11

]


[53

][

2 53 0

] [53

]=




For the matrix A =

[2 53 0



[−11

][

2 53 0

] [−11

]= −3

[−11

]


[53

][

2 53 0

] [53

]=




For the matrix A =

[2 53 0



[−11

][

2 53 0

] [−11

]= −3

[−11

]


[53

][

2 53 0

] [53

]=




For the matrix A =

[2 53 0



[−11

][

2 53 0

] [−11

]= −3

[−11

]


[53

][

2 53 0

] [53

]= 5

[53

]



Markov Matrices

FormationInterpretationProperties

Outline






Markov Matrices


5 Volunteers

1 Data: Each one should give your marking for each one ofyou.

For example: Isha: (I)100, (J)20, (K)50, (L)30, (M)0

2 Normalizing or averaging: How much you have given from thetotal marking

Total mark: 100+20+50+30+0=200Normal mark: , , , ,

3 Arranging: Writing them in a matrix form0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers

1 Data: Each one should give your marking for each one ofyou. For example: Isha: (I)100, (J)20, (K)50, (L)30, (M)0




0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers


2 Normalizing or averaging:

How much you have given from thetotal marking



0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers



Total mark: 100+20+50+30+0=200

Normal mark: , , , ,


0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers



Total mark: 100+20+50+30+0=200Normal mark: 100/200, 20/200,50/200 , 30/200, 0/200


0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers



Total mark: 100+20+50+30+0=200Normal mark: 0.50, 0.1, 0.25, 0.15, 0.0


0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


5 Volunteers



Total mark: 100+20+50+30+0=200Normal mark: 0.50, 0.1, 0.25, 0.15, 0.0


0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2



Markov Matrices


Isha Jose Kumar Latha ManiIsha 0.5 0.8 0.0 0.4 0.2Jose 0.1 0.32 0.0 0.0 0.2

Kumar 0.25 0.24 1.0 0.0 0.2Latha 0.15 0.16 0.0 0.4 0.2Mani 0.0 0.2 0.0 0.2 0.2

↓ ↓ ↓ ↓ ↓sum is 1 1 1 1 1



Markov Matrices


Markov Matrix

M =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

Definition (Markov Matrix)

A real n × n matrix is called a Markov matrix if

1 Each entry is non-negative: mij ≥ 0 for all1 ≤ i ≤ n, 1 ≤ j ≤ n.

2 Each column sum is 1:∑n

i=1mij = 1 for each 1 ≤ j ≤ n.



Markov Matrices


Markov Matrix

M =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

Definition (Markov Matrix)

A real n × n matrix is called a Markov matrix if

1 Each entry is non-negative: mij ≥ 0 for all1 ≤ i ≤ n, 1 ≤ j ≤ n.

2 Each column sum is 1:∑n

i=1mij = 1 for each 1 ≤ j ≤ n.



Markov Matrices


Outline






Markov Matrices


Interpretation

We have formed a Markov matrix from individual opinions.

If Isha’s opinion is full value then Isha is better than others.

If Jose’s opinion is full value then Isha is better than others.

If Mani’s opinion is full value then all are equal.

If all opinions are equal value then Isha is better than others.

If opinion has different value then Kumar is better than others.

0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

When the opinion values matches with the conclusion value?.For which opinion value we will get the same conclusion value.



Markov Matrices


Interpretation







0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

10000

=

0.50.1

0.250.15

0




Markov Matrices


Interpretation







0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

01000

=

0.8

0.320.240.160.2




Markov Matrices


Interpretation







0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

00001

=

0.20.20.20.20.2




Markov Matrices


Interpretation







0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

0.20.20.20.20.2

=

0.3800.1240.3380.1820.120




Markov Matrices


Interpretation






If opinion has different value then Kumar is better than others.0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

0.20.40.10.20.1

=

0.3200.0720.3940.1860.100




Markov Matrices


Interpretation






If opinion has different value then Kumar is better than others.0.5 0.8 0.0 0.4 0.20.1 0.32 0.0 0.0 0.2

0.25 0.24 1.0 0.0 0.20.15 0.16 0.0 0.4 0.20.0 0.2 0.0 0.2 0.2

x1x2x3x3x4

=

x1x2x3x4x5




Markov Matrices


Big Problem



For a Markov matrix M, we need a stable opinion value x .

Find a vector x such that

Mx = x

For M, find an eigenvector x corresponding to eigenvalue 1.



Markov Matrices


Big Problem





Mx = x




Markov Matrices


Big Problem





Mx = x




Markov Matrices


Big Problem





Mx = x




Markov Matrices


Big Problem





Mx = x




Markov Matrices


Outline






Markov Matrices


Properties

Theorem

For every Markov matrix 1 is surely an eigenvalue.

Proof.

For a Markov matrix M every column sum is 1. Consider the

transpose matrix MT . Now its row sum is 1. Let e =

1...1

MT e = e

1 is an eigenvalue of MT with e as its eigenvector.

We know det(A) = det(AT ), so det(M − λI ) = det(MT − λI )Hence 1 is an eigenvalue of M

But eigen vectors of A and AT need not be same.



Markov Matrices


Properties

Theorem


Proof.



1...1

MT e = e






Markov Matrices


Properties

Theorem


Proof.



1...1

MT e = e






Markov Matrices


Properties

Theorem


Proof.



1...1

MT e = e


We know det(A) = det(AT ), so det(M − λI ) = det(MT − λI )

Hence 1 is an eigenvalue of M




Markov Matrices


Properties

Theorem


Proof.



1...1

MT e = e






Markov Matrices


Properties

Theorem


Proof.



1...1

MT e = e






Markov Matrices


Further Properties

The previous result guaranteed the eigenvalue 1, but there is aproblem if it has more eigenvectors corresponding to eigenvalue 1.

Theorem

For a Markov matrix M with all entries mij > 0,

1 1 is always an eigenvalue.

2 Other eigenvalues have magnitude less than 1 ( |λ| < 1 ).

3 There is an eigenvector corresponding to the eigenvalue 1.

4 This eigenvector has all positive entries with sum 1.

5 Such an eigenvector is unique.



Markov Matrices


Further Properties


Theorem









Markov Matrices


Further Properties


Theorem









Markov Matrices


Further Properties


Theorem









Markov Matrices


Further Properties


Theorem









Markov Matrices


Further Properties


Theorem









Markov Matrices


Finding the stable opinion

Now we know that every Markov matrix with positive entries has aunique eigenvector corresponding to the eigenvalue λ = 1.

M − λI =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

− λ

1 0 0 00 1 0 0...

.... . .

...0 0 0 1

n×n

(M − λI )x =

m11 − λ m12 · · · m1n

m21 m22 − λ m2n...

......

mn1 mn2 mnn − λ

x1x2...vn

=

00...0

Solve (M − λI )x = 0 by Gauss Elimination Process



Markov Matrices




M − λI =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

− λ

1 0 0 00 1 0 0...

.... . .

...0 0 0 1

n×n

(M − λI )x =

m11 − λ m12 · · · m1n

m21 m22 − λ m2n...

......

mn1 mn2 mnn − λ

x1x2...vn

=

00...0




Markov Matrices




M − λI =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

− λ

1 0 0 00 1 0 0...

.... . .

...0 0 0 1

n×n

(M − λI )x =

m11 − λ m12 · · · m1n

m21 m22 − λ m2n...

......

mn1 mn2 mnn − λ

x1x2...vn

=

00...0




Markov Matrices




M − λI =

m11 m12 · · · m1n

m21 m22 m2n...

......

mn1 mn2 mnn

n×n

− λ

1 0 0 00 1 0 0...

.... . .

...0 0 0 1

n×n

(M − λI )x =

m11 − λ m12 · · · m1n

m21 m22 − λ m2n...

......

mn1 mn2 mnn − λ

x1x2...vn

=

00...0




Markov Matrices


Other Views

It is a simpler version of page-rank.

It represents transition matrix and steady state.

Markov process in input-output business models.

· · ·



Markov Matrices


Other Views




· · ·



Markov Matrices


Other Views




· · ·



Markov Matrices


Other Views




· · ·



Markov Matrices


Questions

Thank you



Markov Matrices


Questions

Thank you



Markov Matrices


You know better

In aeronautical engineering eigenvalues may determine whetherthe flow over a wing is laminar or turbulent.

In electrical engineering they may determine the frequencyresponse of an amplifier or the reliability of a nationalpower system.

In structural mechanics eigenvalues may determine whether anautomobile is too noisy or whether a building willcollapse in an earth-quake.

In probability they may determine the rate of convergence of aMarkov process.

In ecology they may determine whether a food web will settleinto a steady equilibrium.

In numerical analysis they may determine whether a discretizationof a differential equation will get the right answer orhow fast a conjugate gradient iteration will converge.


Documents

Eigenvalues, eigenvectors and applicationssuku/kozhikode.pdfMarkov Matrices Eigen value and eigen vector Problem Big Problem Getting a common opinion from individual opinion From individual