Coding the Matrix! - Events Server Dept., CSA, IISc · Indian Institute of Science, Bangalore June 28, 2013 Divya Padmanabhan Coding the Matrix! June 28, 2013 1. How does the most

Coding the Matrix!

Divya Padmanabhan

Intelligent Systems Lab,Dept. of CSA,

Indian Institute of Science,Bangalore

June 28, 2013

Divya Padmanabhan Coding the Matrix! June 28, 2013 1

How does the most popular search engine work?

Google it!

Search more than 8 billion pages ..

Results in less than a second..

How is it possible???

What is Google doing in that 1 second?


How does Facebook manage information about you?

Friend network matrixU1 · · · Un

U1 · · ·U2 · · ·...Un · · ·


How does Flipkart recommend products for you?

How does Flipkart know what you require?

How does Flipkart keep track of so many users and find theirinterests?


Some BASICS!!!


Preliminaries

Vectors

eg. How well can you sing? paint? play? ..

Professional singer :(1 · · · )

. Amateur :(0.1 · · · )

Norm |~x | =√x21 + x22 + · · ·+ x2n

Dot product ~x .~y = |~x ||~y | cos θ = x1y1 + x2y2 + · · ·+ xnyn


Preliminaries

Vectors

eg. How well can you sing? paint? play? ..

Professional singer :(1 · · · )

. Amateur :(0.1 · · · )

Norm |~x | =√x21 + x22 + · · ·+ x2n

Dot product ~x .~y = |~x ||~y | cos θ = x1y1 + x2y2 + · · ·+ xnyn


Orthogonality and Projections

Figure: Orthogonal vectors ~xT~y = ||x ||||y || cos θ = 0

~x

~y

θ

Figure: Projection of ~x on ~y = ~x.~y||~y ||

O

Z

~x

~y

~x.~y||y||


Orthogonality and Projections

Figure: Orthogonal vectors ~xT~y = ||x ||||y || cos θ = 0

~x

~y

θ

Figure: Projection of ~x on ~y = ~x.~y||~y ||

O

Z

~x

~y

~x.~y||y||


Linear Combination and Independence

Linear Combination

c1~x + c2~y , c1 ∈ R, c2 ∈ R

Linear Independence of vectors ~x and ~y

c1~x + c2~y = 0⇒ c1 = 0, c2 = 0

Can’t express ~x in terms of ~y .

Eg.

[10

],

[23

], c1 = 0, c2 = 0

Linearly dependent :[12

],

[24

], c1 = 2, c2 = −1



Linear Combination

c1~x + c2~y , c1 ∈ R, c2 ∈ R


c1~x + c2~y = 0⇒ c1 = 0, c2 = 0


Eg.

[10

],

[23

], c1 = 0, c2 = 0


],

[24

], c1 = 2, c2 = −1



Linear Combination

c1~x + c2~y , c1 ∈ R, c2 ∈ R


c1~x + c2~y = 0⇒ c1 = 0, c2 = 0


Eg.

[10

],

[23

], c1 = 0, c2 = 0


],

[24

], c1 = 2, c2 = −1


Matrix Multiplication

A︷︸︸︷[2 34 5

] B︷︸︸︷[12

]=

C︷︸︸︷[8

14

]

1

[24

]+ 2

[35

]=

[8

14

]


Matrix Multiplication

A︷︸︸︷[2 34 5

] B︷︸︸︷[12

]=

C︷︸︸︷[8

14

]

1

[24

]+ 2

[35

]=

[8

14

]


Row and column vectors

A =

[2 1 03 −1 4

]

Row vectors =

r1 =

210

, r2 =

3−14

Column vectors =

{c1 =

[23

], c2 =

[1−1

], c3 =

[04

]}



A =

[2 1 03 −1 4

]Row vectors =

r1 =

210

, r2 =

3−14

Column vectors =

{c1 =

[23

], c2 =

[1−1

], c3 =

[04

]}



A =

[2 1 03 −1 4

]Row vectors =

r1 =

210

, r2 =

3−14

Column vectors =

{c1 =

[23

], c2 =

[1−1

], c3 =

[04

]}


Fundamental spaces

Row space and column space of A

Row Space: Linear combination of r1, r2.

Column space : Linear combination of c1, c2, c3

Null space: Vectors orthogonal to row space of A.

Row Space (A) = Col space (AT )


Representatives of a space

Basis and dimensions of a space

Basis : Smallest no. of linearly independent vectors whose linearcombination gives the entire space.

[−21

]= −2

[10

]+ 1

[01

]Dimension : No. of vectors in the basis.


Redundancy elimination

Rank and nullity of matrix A

Rank(A)= no. of linearly independent column vectors of A.

Nullity(A) = dimension of nullspace of A.

Rank measures redundancy in the matrix.

[1 0 −4 −28 −150 1 −1 −2 −6

]Rank-Nullity Theorem: Rank(A) + Nullity(A) = n






Rank measures redundancy in the matrix.[1 0 −4 −28 −150 1 −1 −2 −6

]

Rank-Nullity Theorem: Rank(A) + Nullity(A) = n






Rank measures redundancy in the matrix.[1 0 −4 −28 −150 1 −1 −2 −6

]Rank-Nullity Theorem: Rank(A) + Nullity(A) = n


Eigen values and eigen vectors

x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6

How do you determine the x and λ given a matrix A?

Ax = λx , x 6= ~0 ( or xTAT = λxT , x 6= ~0 )(A− λI )x = 0det(A− λI ) = 0



x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→

x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6





x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6





x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6


Ax = λx , x 6= ~0 ( or xTAT = λxT , x 6= ~0 )

(A− λI )x = 0det(A− λI ) = 0



x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6


Ax = λx , x 6= ~0 ( or xTAT = λxT , x 6= ~0 )(A− λI )x = 0

det(A− λI ) = 0



x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6





x

Resear h

1

0.8

Publi

Speaking

A2×2−−−→x

Ax = λx

Resear h

Publi

Speaking

1

0.8

2

1.6




Now for the FUN stuff!!!


A Typical Search Activity : Random surfer

Jump−−−→


A Typical Search Activity : Random surfer

Jump−−−→


Importance of pages


Importance of pages


Page Rank Algorithm

Transition Matrix HA B C D

A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

Anything particular about H?

H : row stochastic

Vt(A) : Probability of being at page A at time tVt : row vector. eg.

[0.1 0.3 · · ·

]Vt(A) = Vt−1(A)HAA + Vt−1(B)HBA + . . .

Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·


Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·


Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·


Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·

]

Vt(A) = Vt−1(A)HAA + Vt−1(B)HBA + . . .

Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·


Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


H : row stochastic


[0.1 0.3 · · ·


Vt = Vt−1H


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

H : row stochastic, guaranteed to have eigen value 1, under someconditions.

V0 =

1/41/41/41/4

,

V1 = V0H =

9/245/245/245/24

, V2 = V1H =

15/4811/4811/4811/48


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


V0 =

1/41/41/41/4

, V1 = V0H =

9/245/245/245/24

,

V2 = V1H =

15/4811/4811/4811/48


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


V0 =

1/41/41/41/4

, V1 = V0H =

9/245/245/245/24

, V2 = V1H =

15/4811/4811/4811/48


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


...Vn = Vn−1H

...

vH = 1v

v is the eigen vector of H corresponding to eigen value 1 !!!


Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


...Vn = Vn−1H

...

vH = 1v



Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


...Vn = Vn−1H

...

vH = 1v



Page Rank Algorithm


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0


...Vn = Vn−1H

...

vH = 1v

v is the eigen vector of H corresponding to eigen value 1 !!!Divya Padmanabhan Coding the Matrix! June 28, 2013 20

Page Rank Algorithm - conditions on H


A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

Row stochastic

Irreducible :Possible to reach a page from any other page.

Aperiodic : Not periodic




A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

Row stochastic






A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

Row stochastic






A 0 13

13

13

B 12 0 0 1

2C 1 0 0 0D 0 1

212 0

Row stochastic




Page Rank Algorithm - Dangling nodes


A 0 13

13

13

B 12 0 0 1

2C 1

2 0 0 12

D 0 0 0 0

S = H + Y

=

A B C D

A 0 13

13

13

B 12 0 0 1

2

C 12 0 0 1

2

D 14

14

14

14


Page Rank Algorithm - Dangling nodes


A 0 13

13

13

B 12 0 0 1

2C 1

2 0 0 12

D 0 0 0 0

S = H + Y

=

A B C D

A 0 13

13

13

B 12 0 0 1

2

C 12 0 0 1

2

D 14

14

14

14


Page Rank Algorithm - Entering a new destination

S =

A B C D

A 0 13

13

13

B 12 0 0 1

2C 1

2 0 0 12

D 14

14

14

14

B :

1/n · · · 1/n· · ·

1/n · · · 1/n

n×n

Google matrix!

Surfer follows the link structure 60 % of timeOtherwise to a random page!

G = αS + (1− α)B


Page Rank Algorithm

Search Engines

Get pages containing query words.

Use page rank to determine the order of pagesto be displayed.

Ford 0.1Maruti 0.7Volkswagen 0.05Hyundai 0.15


Page Rank Algorithm

Search Engines

Get pages containing query words.

Use page rank to determine the order of pagesto be displayed.

Ford 0.1Maruti 0.7Volkswagen 0.05Hyundai 0.15


Recommender Systems


Recommender Systems

Matrix Factorization

Original matrix Xd×n (d users, n movies)

X ≈ X̂ = Md×k × Hk×n

Find M,H.


References

1 Gilbert Strang, “Linear Algebra and its Applications”, Fourth Edition.

2 Langville and Meyer,“Google’s PageRank and Beyond”.

3 Gene H. Golub, Charles F. Van Loan, “Matrix Computations”.

4 Howard Anton, Chris Rorres, “Elementary Linear Algebra”.


THANK YOU!


Documents

Coding the Matrix! - Events Server Dept., CSA, IISc · Indian Institute of Science, Bangalore June 28, 2013 Divya Padmanabhan Coding the Matrix! June 28, 2013 1. How does the most