Network Limits and Graphons

Network Limits and Graphons

Nikolaj Takata Mücke

TUM

7/2 - 2018

Nikolaj Takata Mücke (TUM) Network Limits and Graphons 7/2 - 2018 1 / 58

Overview

1 Introduction2 A Little bit About Graphs and Networks

K-Nearest-Neighbour GraphsSmall-World GraphsNetwork Limits and Graphons

3 Dynamics on NetworksNetwork Limits and GraphonsApproximation Properties

4 The Kuramoto Modelq-Twisted StatesContinuum Limit of The Kuramoto ModelStability AnalysisSynchronizationContinuation


A Little bit About Graphs and Networks


What is a graph?An ordered pair, G = (V ,E )

V : The set of vertices (nodes)E : The of edges



Graph RepresentationGraph picture

Works well to get an overview of the structureBecomes very messy for large graphs!

Adjacency MatrixGood when doing computationsDifficult to get an intuitive understanding of the graph

Pixel PictureGood to get an overview of large graphs

Figure 1: The Petersen graph, its adjacency matrix, and its pixel pictureNikolaj Takata Mücke (TUM) Network Limits and Graphons 7/2 - 2018 4 / 58

A Little bit About Graphs and Networks K-Nearest-Neighbour Graphs

K-Nearest-Neighbour Graphs

Definition (k-Nearest-Neighbour Graph)Let Cn,k be a graph. If

V (Cn,k) = [n] and E (Cn,k) = {(i , j) ∈ [n]× [n] | 0 < dn(i , j) ≤ k}

for sufficiently large n ∈ N and k ∈ N such that 2k < n, wheredn(i , j) = min{|i − j |, n − |i − j |}. Then we say that Cn,k is a k-NN graph.

Intuition: A graph where every node is only connected to the k nearestnodes. Where nearest is defined by some metric.


A Little bit About Graphs and Networks K-Nearest-Neighbour Graphs

(a) Network representation

0 10 20 30 40 50 60 70 80 90 100

nz = 5000

0

10

20

30

40

50

60

70

80

90

100

(b) Pixel picture

Figure 2: k-NN graph with n = 100 and k = 25


A Little bit About Graphs and Networks Small-World Graphs

Small-World Graphs

Definition (Small-World Graph)Let L be the distance between two arbitrary nodes in a graph Gn, i.e. thenumber of steps required to go from one node to the other. Then we saythat Gn is a Small-World graph if L ∝ log n.

Intuition: A graph where you can come from an arbitrary chosen node toany node by a small number of steps.



W-Graphs

Definition (W-Graphs)Let Xn = {x1, x2, . . . , xn} ⊂ I = [0, 1], W : I2 → I be a symmetricmeasurable function and Gn = 〈[n],E (Gn)〉. Then we call Gn a W-randomgraph if

P ((i , j) ∈ E (Gn)) ={

W (xi , xj), i 6= j0, Otherwise

and we denote it Gn = G(W ,Xn).

Intuition: A graph where the probability of two nodes are connected isgiven by some probability function, W .



SW-Graphs

Definition (SW-Graphs)

Let Xn ={0, 1

n ,2n , . . . ,

n−1n

}and W be defined as

W (x , y) ={

1, d(x , y) ≤ r0, Otherwise , (1)

and

Wp = (1− p)W + p(1−W ), p ∈ [0, 0.5], (2)

then Gn,p = G(Wp,Xn) is an SW-graph.

Intution: A brnc-NN graph with certain edges made into randomconnections with any other node.



W (x , y) ={

1, d(x , y) ≤ r0, Otherwise

Wp = (1− p)W + p(1−W ), p ∈ [0, 0.5]

p = 0: W0 = Wp = 0.5: W0.5 = 0.5p = 1: W1 = 1−W



K-NN Random Graphs



K-NN Random Graphs

0 10 20 30 40 50 60 70 80 90 100

nz = 5000

0

10

20

30

40

50

60

70

80

90

100

(a) p = 0

0 10 20 30 40 50 60 70 80 90 100

nz = 5000

0

10

20

30

40

50

60

70

80

90

100

(b) p = 0.25

0 10 20 30 40 50 60 70 80 90 100

nz = 5000

0

10

20

30

40

50

60

70

80

90

100

(c) p = 0.5

Figure 3: SW-graphs n = 100, k = 25 and varying randomness parameter p.


A Little bit About Graphs and Networks Network Limits and Graphons


What happens when we increase the number of nodes and edges?What happens at the limit, n→∞?

Definition (Graphon)A graphon is a measurable function W : I2 → I.



a) Adjacency matrix of a k-NN graphb) The support of the corresponding graphon



Graph Limit for W-graphs

TheoremLet {Gn,p}n∈N be a sequnce of W-random graphs. Then the seqeunce isconverging with probability one and it’s limit is the graphon W .


Dynamics on Networks

The General Form

Every node in a graph is in some state. This state evolves with timeaccording to some dynamicsIn a network with n node we have a system of n ODE’s:

ddt u(n)

i =n∑

j=1a(n)

ij K (u(n)i , u(n)

j ), (3)

a(n)ij is the entries of the adjacency matrix.

K : R2 → R, is some function.


Dynamics on Networks

Non-Linear Heat Equation

The non-linear heat equation on graphs Gn. This dynamical system isgiven by

ddt u(n)

i = 1n

n∑j=1

w (n)ij D(u(n)

i − u(n)j ), (4)

where w (n)ij is only non-zero if (i , j) ∈ E (Gn). we assume that D : R→ R

is Lipschitz continuous.


Dynamics on Networks Network Limits and Graphons


Why consider limits?If n gets large we run into troubles

Very difficult to assess behaviour analytically (fixpoints, etc.)Very time consuming to compute

We get an infinite dimensional dynamical system, i.e. a PDEThese are (sometimes) easier to analyse



How does the limit PDE look?

We define the n-dimensional vector

u(n)(t) = (u(n)1 (t), u(n)

2 (t), . . . , u(n)n (t))

If we let n→∞ one will obtain an infinite dimensional vector or simply afunction u(x , t) where x ∈ I.

u(n)(t)→ u(x , t), n→∞

This will be denoted the continuum limit of u(n)(t).




The heat equation:

ddt u(n)

i = 1n

n∑j=1

w (n)ij D(u(n)

i − u(n)j ) (5)

Riemann sum:n∑

j=1f (ti)(xi − xi−1), ti ∈ [xi , xi−1]

Riemann integral:

limn→∞

n∑j=1

f (ti)(xi − xi−1) =∫ b

af (x) dx




The heat equation for n→∞:

u(n)(t)→ u(x , t)

1n

n∑j=1

w (n)ij D(u(n)

i − u(n)j )→

∫ 1

0W (x , y)D(u(x , t)− u(y , t)) dy

where W (x , y) is the limit graphon.



The Continuum Limit PDE

ddt u(x , t) =

∫ 1

0W (x , y)D(u(x , t)− u(y , t)) dy

u(x , 0) = g(x)


Dynamics on Networks Approximation Properties

How Good is This Approximation?

So far we have only provided the intuitive arguments:The right hand side of the Dynamical System resembles a RiemannsumWe therefore "guess" that a Riemann integral approximates the sumin the limit n→∞

Can we provide rigourous arguments for that?



YES WE CAN!



Approximation on Deterministic Network

TheoremSuppose g ∈ L∞(I), W : I2 → {0, 1} is a symmetric measurable functionand

γ := dimB∂W + ∈ [0, 2).

Let u and un denote the vector-valued functions corresponding to thesolutions of the continuum limit PDE and the original system of ODE’s,respectively. Then for any ε > 0 there exists N(ε) ∈ N such that forn ≥ N(ε) :

||u − u(n)||C(0,T ;L2(I)) ≤ C1(||g − gn||L2(I) + nγ/2+ε−1

)where constant C1 is independent of n.



||u − u(n)||C(0,T ;L2(I)) ≤ C1(||g − gn||L2(I) + nγ/2+ε−1

)

Small box dimension of the support of ∂W =⇒ fast convergencegn → g fast =⇒ fast convergence

Note: This is only for W a binary function!



Approximation of Random Network

TheoremSuppose W is almost everywhere continuous on I2, D : R→ R is Lipschitzcontinuous, and g ∈ L∞(I). Let u(x , t) denote the solution of thecontinuum limit PDE. Suppose further

mint∈[0,T ]

∫I2

D(u(y , t)− u(x , t))W (x , y)(1−W (x , y)) dx dy > 0

for some T > 0. Then

||u(n) − u||C(0,T ;L2(I)) → 0

in probability.



Convergence in probability means

limn→∞

P(||un − u||C(0,T ;L2(I)) > ε

)= 0



Approximation Properties

With only the following assumptionsD is Lipschitz continuousThe initial condition g ∈ L∞

W is measurable and binary... we can analyse dynamics on the following graphs

k-NN graphsSmall-worls graphsRandom graphs

W-graphsSW-graphs

... by their continuum limit!


The Kuramoto Model

The Kuramoto Model

Models a network of oscillators on some graph G , by the set of ODE’s;

ddt u(n)

i = ωi + σ

n∑

j:(i ,j)∈E(Gn)sin(2π(u(n)

i − u(n)j )

), i ∈ [n]. (6)

Models coupled oscillators such asJosephson JunctionNeural NetworksCoupled lasersMuch more...!

Proposed by Japanese mathematician Yoshiki Kuramoto.


The Kuramoto Model

The Kuramoto Model

ddt u(n)

i = ωi + σ

n∑


i − u(n)j )

), i ∈ [n]. (7)

i denotes the oscillatorui denotes the phase of the ith oscillatorωi is the natural frequency of oscillator iσ is the coupling between oscillatorsn is the number of oscillators


The Kuramoto Model

The Kuramoto Model

We will only study the case withAll intrinsic frequencies are the same, ωi = ωj for all i and j .Attractive coupling, σ = 1

This gives the system of ODE’s:

ddt u(n)

i = 1n

∑j:(i ,j)∈E(Gn)

sin(2π(u(n)

i − u(n)j )

), i ∈ [n]. (8)

These restrictions simplify the problem quite a lot, but makes it easier forus to convey the important points of this talk.


The Kuramoto Model

The Kuramoto Model

ddt u(5)

1 = 15(sin(u(5)

1 − u(5)4

)+ sin

(u(5)

1 − u(5)5

))ddt u(5)

2 = 15(sin(u(5)

2 − u(5)5

))ddt u(5)

3 = 15(sin(u(5)

3 − u(5)5

))ddt u(5)

4 = 15(sin(u(5)

4 − u(5)1

)+ sin

(u(5)

4 − u(5)5

))ddt u(5)

5 = 15(sin(u(5)

5 − u(5)1

)+ sin

(u(5)

5 − u(5)2

)+ sin

(u(5)

5 − u(5)3

)+ sin

(u(5)

5 − u(5)4

))


The Kuramoto Model

The Kuramoto Model on SW-graphs

We will study the Kuramoto model on small world SW-graph, Gn,p, on acircle.

0 10 20 30 40 50 60 70 80 90 100

nz = 5000

0

10

20

30

40

50

60

70

80

90

100


The Kuramoto Model

The Kuramoto Model on SW-graphs

Remember, connections between nodes are given by

Wp = (1− p)W + p(1−W ), p ∈ [0, 0.5], (9)

with

W (x , y) ={

1, d(x , y) ≤ r0, Otherwise . (10)

Then the Kuramoto model can be written as

ddt u(n)

i = 1n

n∑j=1

Wp(i , j) sin(2π(u(n)

i − u(n)j )

), i ∈ [n]. (11)


The Kuramoto Model q-Twisted States

q-Twisted States

An important class of steady state solutions on k-NN graphs is theso-called q-Twisted States:

u(n)i ,q = q(i − 1)

n + c mod 1, c ∈ [0, 1), i ∈ [n], (12)


The Kuramoto Model q-Twisted States

q-Twisted States

0 10 20 30 40 50 60 70 80 90 100

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 80 90 100

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 80 90 100

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 80 90 100

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 5: q = 1, q = 2, q = 3, q = 4


The Kuramoto Model Continuum Limit of The Kuramoto Model

Continuum Limit of The Kuramoto Model

We want to derive the continuum limit PDE of the Kuramoto system sowe can study

Stability of the q-twisted statesSynchronization

Both in terms of r and the randomness parameter, p.



Continuum Limit of The Kuramoto Model

As a reminder, the discrete Kuramoto model on SW-graphs is given by

ddt u(n)

i = 1n

n∑j=1

Wp(i , j) sin(2π(u(n)

i − u(n)j )

), i ∈ [n]. (13)

From our theorems earlier we have, in the limit n→∞, the continuumlimit PDE:

∂

∂t u(x , t) =∫

IWp(x , y) sin (2π(u(x , t)− u(y , t))) dy . (14)

Note: We assume that the assumptions are fulfilled.



Continuous q-twisted state

uq(x) = qx + c mod 1, c ∈ [0, 1), q ∈ Z. (15)



LemmaLet

R(n)i (u(n)) = 1

n∑


i − u(n)j )

). (16)

Then for any q ∈ N ∪ {0} and i ∈ N

limn→∞

R(n)i (u(n)

q ) = 0 (17)

almost surely.

Basically: The continuous q-twisted state is also a steady state solutionfor the discrete system in the limit n→∞.


The Kuramoto Model Stability Analysis

Rewriting the Continuum Limit

∂

∂t u(x , t) =∫

IWp(x , y) sin (2π(u(y , t)− u(x , t))) dy

Becomes

∂

∂t u(x , t) =∫

IKp(y − x) sin (2π(u(y , t)− u(x , t))) dy

where

Kp(x) = pG1/2(x) + (1− 2p)Gr (x), Gr ={

1, d(x) ≤ r0, Otherwise .



TheoremFor q ∈ Z and p ∈ [0, 0.5], the q-twisted state is a steady state of (14).Moreover, it is linearly stable with respect to perturbations from L∞(I)provided

λp(q,m) := K̃p(m + q)− 2K̃p(q) + K̃ (q −m) < 0, ∀m ∈ N, (18)

where

K̃p(m) =∫

IKp(x) cos(2πmx) dx , m ∈ Z. (19)



Claim:

K̃p(m) ={

p 1πm sin(2πmr) + (1− 2p) 1

πm sin(2πmr), m 6= 0p + (1− 2p)2r , m = 0 .



... Which gives us the following criteria for stability of the q-twisted state

λp(q,m) = K̃p(m + q)− 2K̃p(q) + K̃ (q −m)= p(−2δq0 + δqm) + (1− 2p)λ0(q,m) < 0

where

λ0(q,m) = 1π

[ 1q + m sin(2πr(q + m))− 2

q sin(2πrq)

+ 1q −m sin(2πr(q −m))

]for q 6= 0.



Solving

λp(q,m) = p(−2δq0 + δqm) + (1− 2p)λ0(q,m) < 0

for p or r is quite dificult! But it is done in [9] for r in the case with p = 0:

0 ≤ qr ≤ µ ≈ 0.66 (20)

and in [7] for p

0 < p < −λ0(q, q)1− 2λ0(q, q) (21)


The Kuramoto Model Synchronization

Synchronization

What is Synchronization?When all phases are the same!u1 = u2 = . . . = un

Corresponds to q = 0 in q-twisted states



Synchronization

Using that q = 0 corresponds to synchronization, we can use the conditionfor stability of the q-twisted states;

λp(0,m) = −2p + (1− 2p)λ0(0,m) < 0

λ0(0,m) = 2πm sin(2πmr)− 4r

It is stable for all r !



Synchronization Rate

The rate of the synchronization is the time it takes for the system toreach the synchronized state. The rate is determined by

supm∈N

λp(0,m) = supm∈N

(−2p + (1− 2p)λ0(0,m)) = λp(0, 1) (22)

Larger |λp(0, 1)| → faster synchronization!



Synchronization Rate in Terms of r

How does r affect the rate of synchronization? We set p = 0 and take acloser look at λ(0, 1)

λ0(0, 1) = λ0(0, 1) = 2πsin(2πr)− 4r (23)

By Taylor expansion in r

λ0(0, 1) = −8π2

3 r3 +O(r5) (24)

Hence, larger r gives faster synchronization!



Synchronization Rate in Terms of rN = 100, p = 0.2, initial condition q = 1.

5 10 15 20 25 30 35 40

k

0

50

100

150

200

250

Tim

e it ta

kes to s

ynchro

niz

e



Synchronization Rate in Terms of p

How does p affect the rate of synchronization?

λp(0, 1) = −2p + (1− 2p)λ0(0,m) (25)

ddpλp(0, 1) = −2− 2λ0(0, 1) < 0 (26)

Thus, increse in p → larger |λp(0, 1)| → faster synchronization!



N = 100, k = 10, initial condition q = 1.

0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

p-value

40

50

60

70

80

90

100

110

120

130

Tim

e it ta

kes to s

ynchro

niz

e


The Kuramoto Model Continuation

Continuation

What is continuation?Changing a parameter (or more parameters) continuously to seehow the solution changes



Continuation of 1-twisted stateN = 100, and p = 0 → p = 3.5 · 10−3



Continuation of 1-twisted stateN = 100, and p = 4.9 · 10−3 → p = 3.9 · 10−3



Continuation of 2-twisted stateN = 100, and p = 0 → p = 6 · 10−4



Continuation of 2-twisted stateN = 100, and p = 1.1 · 10−3 → p = 1.63 · 10−3



Christian Borgs, Jennifer T Chayes, László Lovász, Vera T Sós, andKatalin Vesztergombi.Convergent sequences of dense graphs i: Subgraph frequencies, metricproperties and testing.Advances in Mathematics, 219(6):1801–1851, 2008.

Daniel Glasscock.a graphon?Notices of the AMS, 62(1), 2015.

László Lovász.Large networks and graph limits, volume 60.American Mathematical Society Providence, 2012.

László Lovász and Balázs Szegedy.Limits of dense graph sequences.Journal of Combinatorial Theory, Series B, 96(6):933–957, 2006.

Georgi S Medvedev.The nonlinear heat equation on dense graphs and graph limits.



SIAM Journal on Mathematical Analysis, 46(4):2743–2766, 2014.

Georgi S Medvedev.The nonlinear heat equation on w-random graphs.Archive for Rational Mechanics and Analysis, 212(3):781–803, 2014.

Georgi S Medvedev.Small-world networks of kuramoto oscillators.Physica D: Nonlinear Phenomena, 266:13–22, 2014.

Duncan J Watts and Steven H Strogatz.Collective dynamics of ‘small-world’networks.nature, 393(6684):440–442, 1998.

Daniel A Wiley, Steven H Strogatz, and Michelle Girvan.The size of the sync basin.Chaos: An Interdisciplinary Journal of Nonlinear Science,16(1):015103, 2006.


Documents

Network Limits and Graphons