Coding the Matrix

8/18/2019 Coding the Matrix

1/61

Coding the Matrix!

Divya Padmanabhan

Intelligent Systems Lab,

Dept. of CSA,

Indian Institute of Science,

Bangalore

June 28, 2013

Divya Padmanabhan Coding the Matrix! June 28, 2013 1

http://find/


2/61

How does the most popular search engine work?

Google it!

Search more than 8 billion pages ..

Results in less than a second..How is it possible???

What is Google doing in that 1 second?


http://find/


3/61

How does Facebook manage information about you?

Friend network matrix

U 1 · · · U n

U 1 · · ·U 2 · · ·...U n · · ·


http://find/http://goback/


4/61

How does Flipkart recommend products for you?

How does Flipkart know what you require?How does Flipkart keep track of so many users and find theirinterests?


http://goforward/http://find/http://goback/


5/61

Some BASICS!!!


http://find/


6/61

Preliminaries

Vectors

eg. How well can you sing? paint? play? ..

Professional singer :1 · · · . Amateur :0.1 · · ·


http://find/


7/61

Preliminaries

Vectors

eg. How well can you sing? paint? play? ..

Professional singer :1 · · · . Amateur :0.1 · · · Norm | x | =

x 21 + x

22 + · · · + x

2n

Dot product x . y = | x || y | cos θ = x 1y 1 + x 2y 2 + · · · + x ny n


http://find/


8/61

Orthogonality and Projections

Figure: Orthogonal vectors x T y = ||x ||||y || cos θ = 0

x

y

θ


http://find/


9/61

Orthogonality and Projections

Figure: Orthogonal vectors x T y = ||x ||||y || cos θ = 0

x

y

θ

Figure: Projection of x on y = x . y || y ||

x

y

x. y||y||


http://find/


10/61

Linear Combination and Independence

Linear Combinationc 1 x + c 2 y , c 1 ∈ R , c 2 ∈ R

Linear Independence of vectors x and y

c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0


http://find/


11/61




c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0Can’t express x in terms of y .


http://find/


12/61




c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0Can’t express x in terms of y .

Eg.

10

,

23

, c 1 = 0, c 2 = 0

Linearly dependent :12

,

24

, c 1 = 2, c 2 = −1




13/61

Matrix Multiplication


http://find/


14/61

Matrix Multiplication

A

2 34 5B

12 =C

814

1

24

+ 2

35

=

8

14




15/61

Row and column vectors

A =

2 1 03 −1 4


http://find/


16/61


A =

2 1 03 −1 4

Row vectors = r 1 = 2

10

, r 2 = 3

−14


R d l

http://find/


17/61


A =

2 1 03 −1 4

Row vectors = r 1 = 2

10

, r 2 = 3

−14

Column vectors =

c 1 =

23

, c 2 =

1−1

, c 3 =

04


F d l

http://find/


18/61

Fundamental spaces

Row space and column space of A

Row Space: Linear combination of r 1, r 2.

Column space : Linear combination of c 1, c 2, c 3

Null space: Vectors orthogonal to row space of A.

Row Space (A) = Col space (AT )


R i f

http://find/


19/61

Representatives of a space

Basis and dimensions of a space

Basis : Smallest no. of linearly independent vectors whose linearcombination gives the entire space.

−21

= −2

10

+ 1

01

Dimension : No. of vectors in the basis.


R d d li i ti



20/61

Redundancy elimination

Rank and nullity of matrix A

Rank(A)= no. of linearly independent column vectors of A.

Nullity(A) = dimension of nullspace of A.

Rank measures redundancy in the matrix.


R d d li i ti



21/61





Rank measures redundancy in the matrix.1 0 −4 −28 −150 1 −1 −2 −6





22/61





Rank measures redundancy in the matrix.1 0 −4 −28 −150 1 −1 −2 −6

Rank-Nullity Theorem: Rank(A) + Nullity(A) = n


Eigen values and eigen vectors

http://find/


23/61


x



http://find/


24/61


x

A2×2−−−→



http://find/


25/61


x

A2×2−−−→

x

Ax = λx





26/61


x

A2×2−−−→

x

Ax = λx

How do you determine the x and λ given a matrix A?

Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )



http://find/


27/61


x

A2×2−−−→

x

Ax = λx


Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0





28/61


x

A2×2−−−→

x

Ax = λx


Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0det(A − λI ) = 0





29/61


x

A2×2−−−→

x

Ax = λx


Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0det(A − λI ) = 0




30/61

Now for the FUN stuff!!!


A Typical Search Activity : Random surfer

http://find/


31/61




http://find/


32/61

yp y

Jump−−−→


Importance of pages

http://find/


33/61

p p g


Importance of pages

http://find/


34/61

p p g


Page Rank Algorithm

http://find/


35/61

g g


Page Rank Algorithm

http://find/


36/61

Transition Matrix H

A B C D

A 0

1

3

1

3

1

3B 1

2 0 0 1

2

C 1 0 0 0D 0 1

212

0


Page Rank Algorithm

http://find/


37/61

Transition Matrix H

A B C D

A 0

1

3

1

3

1

3B 1

2 0 0 1

2

C 1 0 0 0D 0 1

212

0

Anything particular about H ?


Page Rank Algorithm

http://find/


38/61

Transition Matrix H

A B C D

A 0

1

3

1

3

1

3B 1

2 0 0 1

2

C 1 0 0 0D 0 1

212

0


H : row stochastic


Page Rank Algorithm

http://find/


39/61

Transition Matrix H

A B C D

A 0

1

3

1

3

1

3B 1

2 0 0 1

2

C 1 0 0 0D 0 1

212

0


H : row stochastic

V t (A) : Probability of being at page A at time t V t : row vector. eg.

0.1 0.3 · · ·


Page Rank Algorithm

http://find/


40/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 1

2 0 0 1

2

C 1 0 0 0D 0 1

212

0


H : row stochastic


0.1 0.3 · · ·

V t (A) = V t −1(A)H AA + V t −1(B )H BA + . . .


Page Rank Algorithm

http://find/


41/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 12

0 0 12

C 1 0 0 0D 0 1

212

0


H : row stochastic


0.1 0.3 · · ·

V t (A) = V t −1(A)H AA + V t −1(B )H BA + . . .

V t = V t −1H


Page Rank Algorithm

http://find/


42/61

Transition Matrix H

A B C D A 0 1

313

13

B 12

0 0 12

C 1 0 0 0D 0 1

212

0

H : row stochastic, guaranteed to have eigen value 1, under someconditions.

V 0 =

1/41/4

1/41/4

,


Page Rank Algorithm

http://find/


43/61

Transition Matrix H

A B C D A 0 1

313

13

B 12

0 0 12

C 1 0 0 0D 0 1

212

0


V 0 =

1/41/4

1/41/4

, V 1 = V 0H =

9/245/24

5/245/24

,


http://find/


44/61

Page Rank Algorithm


45/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 12

0 0 12

C 1 0 0 0D 0 1

212

0


...V n = V n−1H


Page Rank Algorithm

http://find/


46/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 12

0 0 12

C 1 0 0 0D 0 1

212

0


...V n = V n−1H


Page Rank Algorithm

http://find/


47/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 12

0 0 12

C 1 0 0 0D 0 1

212

0


...V n = V n−1H

...

vH = 1v


Page Rank Algorithm



48/61

Transition Matrix H

A B C D

A 0 13

1

3

1

3B 12

0 0 12

C 1 0 0 0D 0 1

212

0


...V n = V n−1H

...

vH = 1v

v is the eigen vector of H corresponding to eigen value 1 !!!Divya Padmanabhan Coding the Matrix! June 28, 2013 20

Page Rank Algorithm - conditions on H

http://find/


49/61

Transition Matrix H

A B C D A 0 1

313

13

B 12

0 0 12

C 1 0 0 0D 0 1

212

0



http://find/


50/61

Transition Matrix H

A B C D A 0 1

313

13

B 12

0 0 12

C 1 0 0 0D 0 1

212

0

Row stochastic



http://find/


51/61

Transition Matrix H

A B C D A 0 1

313

13

B 12

0 0 12

C 1 0 0 0D 0 1

212

0

Row stochastic

Irreducible :Possible to reach a page from any other page.


http://find/


52/61

Page Rank Algorithm - Dangling nodes


53/61

Transition Matrix H

A B C D

A 0 13 13 13B 1

2 0 0 1

2

C 12

0 0 12

D 0 0 0 0


Page Rank Algorithm - Dangling nodes

http://find/


54/61

Transition Matrix H

A B C D

A 0 13 13 13B 1

2 0 0 1

2

C 12

0 0 12

D 0 0 0 0

S = H + Y

=

A B C D

A 0 13

13

13

B 1

2

0 0 1

2C 1

2 0 0 1

2

D 14

14

14

14


Page Rank Algorithm - Entering a new destination



55/61

S =

A B C D

A 0 13

13

13

B 12

0 0 12

C 12

0 0 12

D 14

14

14

14

B :

1/n · · · 1/n· · ·

1/n · · · 1/n

n×n

Google matrix!

Surfer follows the link structure 60 % of timeOtherwise to a random page!

G = αS + (1 − α)B


Page Rank Algorithm

http://find/


56/61

Search Engines

Get pages containing query words.

Use page rank to determine the order of pagesto be displayed.


Page Rank Algorithm

http://find/


57/61

Search Engines

Get pages containing query words.

Use page rank to determine the order of pagesto be displayed.

Ford 0.1Maruti 0.7Volkswagen 0.05

Hyundai 0.15


Recommender Systems



58/61


Recommender Systems

http://find/


59/61

Matrix Factorization

Original matrix X d ×n (d users, n movies)

X ≈ X̂ = M d ×k × H k ×n

Find M , H .


References

http://find/


60/61

1 Gilbert Strang, “Linear Algebra and its Applications”, Fourth Edition.

2 Langville and Meyer,“Google’s PageRank and Beyond”.

3 Gene H. Golub, Charles F. Van Loan, “Matrix Computations”.

4 Howard Anton, Chris Rorres, “Elementary Linear Algebra”.


http://find/


61/61

THANK YOU!

http://find/

Documents

Coding the Matrix