View
236
Download
0
Category
Preview:
Citation preview
8/18/2019 Coding the Matrix
1/61
Coding the Matrix!
Divya Padmanabhan
Intelligent Systems Lab,
Dept. of CSA,
Indian Institute of Science,
Bangalore
June 28, 2013
Divya Padmanabhan Coding the Matrix! June 28, 2013 1
http://find/
8/18/2019 Coding the Matrix
2/61
How does the most popular search engine work?
Google it!
Search more than 8 billion pages ..
Results in less than a second..How is it possible???
What is Google doing in that 1 second?
Divya Padmanabhan Coding the Matrix! June 28, 2013 2
http://find/
8/18/2019 Coding the Matrix
3/61
How does Facebook manage information about you?
Friend network matrix
U 1 · · · U n
U 1 · · ·U 2 · · ·...U n · · ·
Divya Padmanabhan Coding the Matrix! June 28, 2013 3
http://find/http://goback/
8/18/2019 Coding the Matrix
4/61
How does Flipkart recommend products for you?
How does Flipkart know what you require?How does Flipkart keep track of so many users and find theirinterests?
Divya Padmanabhan Coding the Matrix! June 28, 2013 4
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
5/61
Some BASICS!!!
Divya Padmanabhan Coding the Matrix! June 28, 2013 5
http://find/
8/18/2019 Coding the Matrix
6/61
Preliminaries
Vectors
eg. How well can you sing? paint? play? ..
Professional singer :1 · · · . Amateur :0.1 · · ·
Divya Padmanabhan Coding the Matrix! June 28, 2013 6
http://find/
8/18/2019 Coding the Matrix
7/61
Preliminaries
Vectors
eg. How well can you sing? paint? play? ..
Professional singer :1 · · · . Amateur :0.1 · · · Norm | x | =
x 21 + x
22 + · · · + x
2n
Dot product x . y = | x || y | cos θ = x 1y 1 + x 2y 2 + · · · + x ny n
Divya Padmanabhan Coding the Matrix! June 28, 2013 6
http://find/
8/18/2019 Coding the Matrix
8/61
Orthogonality and Projections
Figure: Orthogonal vectors x T y = ||x ||||y || cos θ = 0
x
y
θ
Divya Padmanabhan Coding the Matrix! June 28, 2013 7
http://find/
8/18/2019 Coding the Matrix
9/61
Orthogonality and Projections
Figure: Orthogonal vectors x T y = ||x ||||y || cos θ = 0
x
y
θ
Figure: Projection of x on y = x . y || y ||
x
y
x. y||y||
Divya Padmanabhan Coding the Matrix! June 28, 2013 7
http://find/
8/18/2019 Coding the Matrix
10/61
Linear Combination and Independence
Linear Combinationc 1 x + c 2 y , c 1 ∈ R , c 2 ∈ R
Linear Independence of vectors x and y
c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0
Divya Padmanabhan Coding the Matrix! June 28, 2013 8
http://find/
8/18/2019 Coding the Matrix
11/61
Linear Combination and Independence
Linear Combinationc 1 x + c 2 y , c 1 ∈ R , c 2 ∈ R
Linear Independence of vectors x and y
c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0Can’t express x in terms of y .
Divya Padmanabhan Coding the Matrix! June 28, 2013 8
http://find/
8/18/2019 Coding the Matrix
12/61
Linear Combination and Independence
Linear Combinationc 1 x + c 2 y , c 1 ∈ R , c 2 ∈ R
Linear Independence of vectors x and y
c 1 x + c 2 y = 0 ⇒ c 1 = 0, c 2 = 0Can’t express x in terms of y .
Eg.
10
,
23
, c 1 = 0, c 2 = 0
Linearly dependent :12
,
24
, c 1 = 2, c 2 = −1
Divya Padmanabhan Coding the Matrix! June 28, 2013 8
http://find/http://goback/
8/18/2019 Coding the Matrix
13/61
Matrix Multiplication
Divya Padmanabhan Coding the Matrix! June 28, 2013 9
http://find/
8/18/2019 Coding the Matrix
14/61
Matrix Multiplication
A
2 34 5B
12 =C
814
1
24
+ 2
35
=
8
14
Divya Padmanabhan Coding the Matrix! June 28, 2013 9
http://find/http://goback/
8/18/2019 Coding the Matrix
15/61
Row and column vectors
A =
2 1 03 −1 4
Divya Padmanabhan Coding the Matrix! June 28, 2013 10
http://find/
8/18/2019 Coding the Matrix
16/61
Row and column vectors
A =
2 1 03 −1 4
Row vectors = r 1 = 2
10
, r 2 = 3
−14
Divya Padmanabhan Coding the Matrix! June 28, 2013 10
R d l
http://find/
8/18/2019 Coding the Matrix
17/61
Row and column vectors
A =
2 1 03 −1 4
Row vectors = r 1 = 2
10
, r 2 = 3
−14
Column vectors =
c 1 =
23
, c 2 =
1−1
, c 3 =
04
Divya Padmanabhan Coding the Matrix! June 28, 2013 10
F d l
http://find/
8/18/2019 Coding the Matrix
18/61
Fundamental spaces
Row space and column space of A
Row Space: Linear combination of r 1, r 2.
Column space : Linear combination of c 1, c 2, c 3
Null space: Vectors orthogonal to row space of A.
Row Space (A) = Col space (AT )
Divya Padmanabhan Coding the Matrix! June 28, 2013 11
R i f
http://find/
8/18/2019 Coding the Matrix
19/61
Representatives of a space
Basis and dimensions of a space
Basis : Smallest no. of linearly independent vectors whose linearcombination gives the entire space.
−21
= −2
10
+ 1
01
Dimension : No. of vectors in the basis.
Divya Padmanabhan Coding the Matrix! June 28, 2013 12
R d d li i ti
http://find/http://goback/
8/18/2019 Coding the Matrix
20/61
Redundancy elimination
Rank and nullity of matrix A
Rank(A)= no. of linearly independent column vectors of A.
Nullity(A) = dimension of nullspace of A.
Rank measures redundancy in the matrix.
Divya Padmanabhan Coding the Matrix! June 28, 2013 13
R d d li i ti
http://find/http://goback/
8/18/2019 Coding the Matrix
21/61
Redundancy elimination
Rank and nullity of matrix A
Rank(A)= no. of linearly independent column vectors of A.
Nullity(A) = dimension of nullspace of A.
Rank measures redundancy in the matrix.1 0 −4 −28 −150 1 −1 −2 −6
Divya Padmanabhan Coding the Matrix! June 28, 2013 13
Redundancy elimination
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
22/61
Redundancy elimination
Rank and nullity of matrix A
Rank(A)= no. of linearly independent column vectors of A.
Nullity(A) = dimension of nullspace of A.
Rank measures redundancy in the matrix.1 0 −4 −28 −150 1 −1 −2 −6
Rank-Nullity Theorem: Rank(A) + Nullity(A) = n
Divya Padmanabhan Coding the Matrix! June 28, 2013 13
Eigen values and eigen vectors
http://find/
8/18/2019 Coding the Matrix
23/61
Eigen values and eigen vectors
x
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://find/
8/18/2019 Coding the Matrix
24/61
Eigen values and eigen vectors
x
A2×2−−−→
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://find/
8/18/2019 Coding the Matrix
25/61
Eigen values and eigen vectors
x
A2×2−−−→
x
Ax = λx
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
26/61
Eigen values and eigen vectors
x
A2×2−−−→
x
Ax = λx
How do you determine the x and λ given a matrix A?
Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://find/
8/18/2019 Coding the Matrix
27/61
Eigen values and eigen vectors
x
A2×2−−−→
x
Ax = λx
How do you determine the x and λ given a matrix A?
Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
28/61
Eigen values and eigen vectors
x
A2×2−−−→
x
Ax = λx
How do you determine the x and λ given a matrix A?
Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0det(A − λI ) = 0
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
Eigen values and eigen vectors
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
29/61
Eigen values and eigen vectors
x
A2×2−−−→
x
Ax = λx
How do you determine the x and λ given a matrix A?
Ax = λx , x = 0 ( or x T AT = λx T , x = 0 )(A − λI )x = 0det(A − λI ) = 0
Divya Padmanabhan Coding the Matrix! June 28, 2013 14
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
30/61
Now for the FUN stuff!!!
Divya Padmanabhan Coding the Matrix! June 28, 2013 15
A Typical Search Activity : Random surfer
http://find/
8/18/2019 Coding the Matrix
31/61
A Typical Search Activity : Random surfer
Divya Padmanabhan Coding the Matrix! June 28, 2013 16
A Typical Search Activity : Random surfer
http://find/
8/18/2019 Coding the Matrix
32/61
yp y
Jump−−−→
Divya Padmanabhan Coding the Matrix! June 28, 2013 16
Importance of pages
http://find/
8/18/2019 Coding the Matrix
33/61
p p g
Divya Padmanabhan Coding the Matrix! June 28, 2013 17
Importance of pages
http://find/
8/18/2019 Coding the Matrix
34/61
p p g
Divya Padmanabhan Coding the Matrix! June 28, 2013 17
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
35/61
g g
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
36/61
Transition Matrix H
A B C D
A 0
1
3
1
3
1
3B 1
2 0 0 1
2
C 1 0 0 0D 0 1
212
0
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
37/61
Transition Matrix H
A B C D
A 0
1
3
1
3
1
3B 1
2 0 0 1
2
C 1 0 0 0D 0 1
212
0
Anything particular about H ?
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
38/61
Transition Matrix H
A B C D
A 0
1
3
1
3
1
3B 1
2 0 0 1
2
C 1 0 0 0D 0 1
212
0
Anything particular about H ?
H : row stochastic
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
39/61
Transition Matrix H
A B C D
A 0
1
3
1
3
1
3B 1
2 0 0 1
2
C 1 0 0 0D 0 1
212
0
Anything particular about H ?
H : row stochastic
V t (A) : Probability of being at page A at time t V t : row vector. eg.
0.1 0.3 · · ·
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
40/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 1
2 0 0 1
2
C 1 0 0 0D 0 1
212
0
Anything particular about H ?
H : row stochastic
V t (A) : Probability of being at page A at time t V t : row vector. eg.
0.1 0.3 · · ·
V t (A) = V t −1(A)H AA + V t −1(B )H BA + . . .
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
41/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 12
0 0 12
C 1 0 0 0D 0 1
212
0
Anything particular about H ?
H : row stochastic
V t (A) : Probability of being at page A at time t V t : row vector. eg.
0.1 0.3 · · ·
V t (A) = V t −1(A)H AA + V t −1(B )H BA + . . .
V t = V t −1H
Divya Padmanabhan Coding the Matrix! June 28, 2013 18
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
42/61
Transition Matrix H
A B C D A 0 1
313
13
B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
V 0 =
1/41/4
1/41/4
,
Divya Padmanabhan Coding the Matrix! June 28, 2013 19
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
43/61
Transition Matrix H
A B C D A 0 1
313
13
B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
V 0 =
1/41/4
1/41/4
, V 1 = V 0H =
9/245/24
5/245/24
,
Divya Padmanabhan Coding the Matrix! June 28, 2013 19
http://find/
8/18/2019 Coding the Matrix
44/61
Page Rank Algorithm
8/18/2019 Coding the Matrix
45/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
...V n = V n−1H
Divya Padmanabhan Coding the Matrix! June 28, 2013 20
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
46/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
...V n = V n−1H
Divya Padmanabhan Coding the Matrix! June 28, 2013 20
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
47/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
...V n = V n−1H
...
vH = 1v
Divya Padmanabhan Coding the Matrix! June 28, 2013 20
Page Rank Algorithm
http://find/http://goback/
8/18/2019 Coding the Matrix
48/61
Transition Matrix H
A B C D
A 0 13
1
3
1
3B 12
0 0 12
C 1 0 0 0D 0 1
212
0
H : row stochastic, guaranteed to have eigen value 1, under someconditions.
...V n = V n−1H
...
vH = 1v
v is the eigen vector of H corresponding to eigen value 1 !!!Divya Padmanabhan Coding the Matrix! June 28, 2013 20
Page Rank Algorithm - conditions on H
http://find/
8/18/2019 Coding the Matrix
49/61
Transition Matrix H
A B C D A 0 1
313
13
B 12
0 0 12
C 1 0 0 0D 0 1
212
0
Divya Padmanabhan Coding the Matrix! June 28, 2013 21
Page Rank Algorithm - conditions on H
http://find/
8/18/2019 Coding the Matrix
50/61
Transition Matrix H
A B C D A 0 1
313
13
B 12
0 0 12
C 1 0 0 0D 0 1
212
0
Row stochastic
Divya Padmanabhan Coding the Matrix! June 28, 2013 21
Page Rank Algorithm - conditions on H
http://find/
8/18/2019 Coding the Matrix
51/61
Transition Matrix H
A B C D A 0 1
313
13
B 12
0 0 12
C 1 0 0 0D 0 1
212
0
Row stochastic
Irreducible :Possible to reach a page from any other page.
Divya Padmanabhan Coding the Matrix! June 28, 2013 21
http://find/
8/18/2019 Coding the Matrix
52/61
Page Rank Algorithm - Dangling nodes
8/18/2019 Coding the Matrix
53/61
Transition Matrix H
A B C D
A 0 13 13 13B 1
2 0 0 1
2
C 12
0 0 12
D 0 0 0 0
Divya Padmanabhan Coding the Matrix! June 28, 2013 22
Page Rank Algorithm - Dangling nodes
http://find/
8/18/2019 Coding the Matrix
54/61
Transition Matrix H
A B C D
A 0 13 13 13B 1
2 0 0 1
2
C 12
0 0 12
D 0 0 0 0
S = H + Y
=
A B C D
A 0 13
13
13
B 1
2
0 0 1
2C 1
2 0 0 1
2
D 14
14
14
14
Divya Padmanabhan Coding the Matrix! June 28, 2013 22
Page Rank Algorithm - Entering a new destination
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
55/61
S =
A B C D
A 0 13
13
13
B 12
0 0 12
C 12
0 0 12
D 14
14
14
14
B :
1/n · · · 1/n· · ·
1/n · · · 1/n
n×n
Google matrix!
Surfer follows the link structure 60 % of timeOtherwise to a random page!
G = αS + (1 − α)B
Divya Padmanabhan Coding the Matrix! June 28, 2013 23
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
56/61
Search Engines
Get pages containing query words.
Use page rank to determine the order of pagesto be displayed.
Divya Padmanabhan Coding the Matrix! June 28, 2013 24
Page Rank Algorithm
http://find/
8/18/2019 Coding the Matrix
57/61
Search Engines
Get pages containing query words.
Use page rank to determine the order of pagesto be displayed.
Ford 0.1Maruti 0.7Volkswagen 0.05
Hyundai 0.15
Divya Padmanabhan Coding the Matrix! June 28, 2013 24
Recommender Systems
http://goforward/http://find/http://goback/
8/18/2019 Coding the Matrix
58/61
Divya Padmanabhan Coding the Matrix! June 28, 2013 25
Recommender Systems
http://find/
8/18/2019 Coding the Matrix
59/61
Matrix Factorization
Original matrix X d ×n (d users, n movies)
X ≈ X̂ = M d ×k × H k ×n
Find M , H .
Divya Padmanabhan Coding the Matrix! June 28, 2013 26
References
http://find/
8/18/2019 Coding the Matrix
60/61
1 Gilbert Strang, “Linear Algebra and its Applications”, Fourth Edition.
2 Langville and Meyer,“Google’s PageRank and Beyond”.
3 Gene H. Golub, Charles F. Van Loan, “Matrix Computations”.
4 Howard Anton, Chris Rorres, “Elementary Linear Algebra”.
Divya Padmanabhan Coding the Matrix! June 28, 2013 27
http://find/
8/18/2019 Coding the Matrix
61/61
THANK YOU!
Divya Padmanabhan Coding the Matrix! June 28, 2013 28
http://find/Recommended