PoCS|@pocsvox Mechanisms for Generating Power-Law Power ...pdodds/teaching/courses/2013-08UVM... · PoCS|@pocsvox Power-Law Mechanisms, Pt. 1 Random Walks The First Return Problem

PoCS | @pocsvox

Power-LawMechanisms, Pt. 1

Random WalksThe First Return Problem

Examples

VariabletransformationBasics

Holtsmark’s Distribution

PLIPLO

References

Story?CoP What’s

The

S

1 of 46

Mechanisms for GeneratingPower-Law Size Distributions, Part 1

Principles of Complex Systems | @pocsvoxCSYS/MATH 300, Fall, 2013 | #FallPoCS2013

Prof. Peter Dodds | @peterdodds

Dept. of Mathematics & Statistics | Vermont Complex Systems CenterVermont Advanced Computing Core | University of Vermont

Story?CoP What’s

The

S

Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.

http://www.uvm.edu/~pdodds/teaching/courses/2013-08UVM-300

http://www.twitter.com/@pocsvox

http://www.uvm.edu

http://www.uvm.edu/~pdodds




http://www.uvm.edu/~cems/mathstat/

http://www.uvm.edu/~cems/complexsystems/

http://www.uvm.edu/~vacc/

http://www.uvm.edu



PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

2 of 46

These slides are brought to you by:



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

3 of 46

Outline

Random WalksThe First Return ProblemExamples

Variable transformationBasicsHoltsmark’s DistributionPLIPLO

References



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

4 of 46

Mechanisms:

A powerful story in the rise of complexity:I structure arises out of randomness.I Exhibit A: Random walks. ()

The essential random walk:I One spatial dimension.I Time and space are discreteI Random walker (e.g., a drunk) starts at origin x = 0.I Step at time t is εt:

εt =

+1 with probability 1/2−1 with probability 1/2



http://www.uvm.edu


http://en.wikipedia.org/wiki/Random_walk

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

5 of 46

A few random random walks:

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−100

−50

0

50

t

x

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−100

−50

0

50

t

x

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−100

0

100

200

t

x

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−100

0

100

200

t

x



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

6 of 46

Random walks:

Displacement after t steps:

xt =

t∑i=1

εi

Expected displacement:

〈xt〉 =

⟨t∑

i=1

εi

⟩=

t∑i=1

〈εi〉 = 0

I At any time step, we ‘expect’ our drunkard to be backat the pub.

I Obviously fails for odd number of steps...I But as time goes on, the chance of our drunkard

lurching back to the pub must diminish, right?



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

7 of 46

Variances sum: ()∗

Var(xt) = Var

(t∑

i=1

εi

)

=

t∑i=1

Var (εi) =

t∑i=1

1 = t

∗ Sum rule = a good reason for using the variance to measurespread; only works for independent distributions.

So typical displacement from the origin scales as:

σ = t1/2

I A non-trivial scaling law arises out ofadditive aggregation or accumulation.



http://www.uvm.edu


http://en.wikipedia.org/wiki/Variance#Variance_of_the_sum_of_uncorrelated_variables

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

8 of 46

Stock Market randomness:

Also known as the bean machine (), the quincunx(simulation) (), and the Galton box.



http://www.uvm.edu


http://en.wikipedia.org/wiki/Bean_machine

http://www.mathsisfun.com/data/quincunx.html

http://www.mathsisfun.com/data/quincunx.html

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

9 of 46

Great moments in Televised Random Walks:

Plinko! () from the Price is Right.



http://www.uvm.edu


http://en.wikipedia.org/wiki/Plinko

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

10 of 46

Random walk basics:

Counting random walks:I Each specific random walk of length t appears with a

chance 1/2t.I We’ll be more interested in how many random walks

end up at the same place.I Define N(i, j, t) as # distinct walks that start at x = i

and end at x = j after t time steps.I Random walk must displace by +(j− i) after t steps.I Insert question from assignment 2 ()

N(i, j, t) =(

t(t + j− i)/2

)



http://www.uvm.edu


http://www.uvm.edu/~pdodds/teaching/courses/2013-08UVM-300/docs/2013-08UVM-300assignment2.pdf

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

11 of 46

How does P(xt) behave for large t?I Take time t = 2n to help ourselves.I x2n ∈ 0,±2,±4, . . . ,±2nI x2n is even so set x2n = 2k.I Using our expression N(i, j, t) with i = 0, j = 2k, and

t = 2n, we have

Pr(x2n ≡ 2k) ∝(

2nn + k

)I For large n, the binomial deliciously approaches the

Normal Distribution of Snoredom:

Pr(xt ≡ x) ' 1√2πt

e−x22t .

Insert question from assignment 2 ()I The whole is different from the parts. #nutritiousI See also: Stable Distributions ()



http://www.uvm.edu



http://en.wikipedia.org/wiki/Stable_distribution

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

12 of 46

Universality () is also not left-handed:

I This is Diffusion (): the most essential kind ofspreading (more later).

I View as Random Additive Growth Mechanism.



http://www.uvm.edu


http://en.wikipedia.org/wiki/Universality_(dynamical_systems)

http://en.wikipedia.org/wiki/Diffusion

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

13 of 46

Random walks are even weirder than you mightthink...I ξr,t = the probability that by time step t, a random

walk has crossed the origin r times.I Think of a coin flip game with ten thousand tosses.I If you are behind early on, what are the chances you

will make a comeback?I The most likely number of lead changes is... 0.I In fact: ξ0,t > ξ1,t > ξ2,t > · · ·I Even crazier:

The expected time between tied scores =∞!

See Feller, Intro to Probability Theory, Volume I [3]



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

15 of 46

Random walks #crazytownbananapants

The problem of first return:I What is the probability that a random walker in one

dimension returns to the origin for the first time after tsteps?

I Will our drunkard always return to the origin?I What about higher dimensions?

Reasons for caring:1. We will find a power-law size distribution with an

interesting exponent.2. Some physical structures may result from random

walks.3. We’ll start to see how different scalings relate to

each other.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

16 of 46

For random walks in 1-d:

0 5 10 15 20−4

−2

0

2

4

t

x

I A return to origin can only happen when t = 2n.I In example above, returns occur at t = 8, 10, and 14.I Call Pfr(2n) the probability of first return at t = 2n.I Probability calculation ≡ Counting problem

(combinatorics/statistical mechanics).I Idea: Transform first return problem into an easier

return problem.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

17 of 46

0 2 4 6 8 10 12 14 160

1

2

3

4

t

x

I Can assume drunkard first lurches to x = 1.I Observe walk first returning at t = 16 stays at or

above x = 1 for 1 ≤ t ≤ 15 (dashed red line).I Now want walks that can return many times to x = 1.I Pfr(2n) =

2 · 12 Pr(xt ≥ 1, 1 ≤ t ≤ 2n− 1, and x1 = x2n−1 = 1)

I The 12 accounts for x2n = 2 instead of 0.

I The 2 accounts for drunkards that first lurch tox = −1.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

18 of 46

Counting first returns:

Approach:I Move to counting numbers of walks.I Return to probability at end.I Again, N(i, j, t) is the # of possible walks between

x = i and x = j taking t steps.I Consider all paths starting at x = 1 and ending at

x = 1 after t = 2n− 2 steps.I Idea: If we can compute the number of walks that hit

x = 0 at least once, then we can subtract this fromthe total number to find the ones that maintain x ≥ 1.

I Call walks that drop below x = 1 excluded walks.I We’ll use a method of images to identify these

excluded walks.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

19 of 46

Examples of excluded walks:

0 2 4 6 8 10 12 14 16−4

−3

−2

−1

0

1

2

3

4

t

x

0 2 4 6 8 10 12 14 16−4

−3

−2

−1

0

1

2

3

4

t

x

Key observation for excluded walks:I For any path starting at x=1 that hits 0, there is a

unique matching path starting at x=−1.I Matching path first mirrors and then tracks after first

reaching x=0.I # of t-step paths starting and ending at x=1 and

hitting x=0 at least once= # of t-step paths starting at x=−1 and ending atx=1 = N(−1, 1, t)

I So Nfirst return(2n) = N(1, 1, 2n− 2)− N(−1, 1, 2n− 2)



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

20 of 46

Probability of first return:

Insert question from assignment 2 () :I Find

Nfr(2n) ∼ 22n−3/2√

2πn3/2.

I Normalized number of paths gives probability.I Total number of possible paths = 22n.I

Pfr(2n) =1

22n Nfr(2n)

' 122n

22n−3/2√

2πn3/2

=1√2π

(2n)−3/2 ∝ t−3/2.



http://www.uvm.edu



PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

21 of 46

First Returns

I

P(t) ∝ t−3/2, γ = 3/2

I Same scaling holds for continuous space/time walks.I P(t) is normalizable.I Recurrence: Random walker always returns to originI But mean, variance, and all higher moments are

infinite. #totalmadnessI Even though walker must return, expect a long wait...I One moral: Repeated gambling against an infinitely

wealthy opponent must lead to ruin.

Higher dimensions ():I Walker in d = 2 dimensions must also returnI Walker may not return in d ≥ 3 dimensions



http://www.uvm.edu


http://mathworld.wolfram.com/PolyasRandomWalkConstants.html

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

22 of 46

Random walks

On finite spaces:I In any finite homogeneous space, a random walker

will visit every site with equal probabilityI Call this probability the Invariant Density of a

dynamical systemI Non-trivial Invariant Densities arise in chaotic

systems.

On networks:I On networks, a random walker visits each node with

frequency ∝ node degree #groovyI Equal probability still present:

walkers traverse edges with equal frequency.#totallygroovy



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

24 of 46

Scheidegger Networks [8, 2]

I Random directed network on triangular lattice.I Toy model of real networks.I ‘Flow’ is southeast or southwest with equal

probability.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

25 of 46

Scheidegger networks

I Creates basins with random walk boundaries.I Observe that subtracting one random walk from

another gives random walk with increments:

εt =

+1 with probability 1/40 with probability 1/2−1 with probability 1/4

I Random walk with probabilistic pauses.I Basin termination = first return random walk problem.I Basin length ` distribution: P(`) ∝ `−3/2

I For real river networks, generalize to P(`) ∝ `−γ .



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

26 of 46

Connections between exponents:

I For a basin of length `, width ∝ `1/2

I Basin area a ∝ ` · `1/2 = `3/2

I Invert: ` ∝ a 2/3

I d` ∝ d(a2/3) = 2/3a−1/3daI Pr(basin area = a)da

= Pr(basin length = `)d`∝ `−3/2d`∝ (a2/3)−3/2a−1/3da= a−4/3da= a−τda



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

27 of 46


I Both basin area and length obey power lawdistributions

I Observed for real river networksI Reportedly: 1.3 < τ < 1.5 and 1.5 < γ < 2

Generalize relationship between area and length:I Hack’s law [4]:

` ∝ ah.

I For real, large networks h ' 0.5I Smaller basins possibly h > 1/2 (later: allometry).I Models exist with interesting values of h.I Plan: Redo calc with γ, τ , and h.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

28 of 46

Connections between exponents:I Given

` ∝ ah, P(a) ∝ a−τ , and P(`) ∝ `−γ

I d` ∝ d(ah) = hah−1daI Find τ in terms of γ and h.I Pr(basin area = a)da

= Pr(basin length = `)d`∝ `−γd`∝ (ah)−γah−1da= a−(1+h (γ−1))da

I

τ = 1 + h(γ − 1)

I Excellent example of the Scaling Relations foundbetween exponents describing power laws for manysystems.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

29 of 46


With more detailed description of network structure,τ = 1 + h(γ − 1) simplifies to: [1]

τ = 2− h

andγ = 1/h

I Only one exponent is independent (take h).I Simplifies system description.I Expect Scaling Relations where power laws are

found.I Need only characterize Universality () class with

independent exponents.



http://www.uvm.edu


http://en.wikipedia.org/wiki/Universality_(dynamical_systems)

PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

30 of 46

Other First Returns or First Passage Times:

Failure:I A very simple model of failure/death: [10]

I xt = entity’s ‘health’ at time tI Start with x0 > 0.I Entity fails when x hits 0.

StreamsI Dispersion of suspended sediments in streams.I Long times for clearing.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

31 of 46

More than randomness

I Can generalize to Fractional Random Walks [6, 7, 5]

I Levy flights, Fractional Brownian MotionI See Montroll and Shlesinger for example: [5]

“On 1/f noise and other distributions with long tails.”Proc. Natl. Acad. Sci., 1982.

I In 1-d, standard deviation σ scales as

σ ∼ tα

α = 1/2 — diffusiveα > 1/2 — superdiffusiveα < 1/2 — subdiffusive

I Extensive memory of path now matters...



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

33 of 46

Variable Transformation

Understand power laws as arising from1. Elementary distributions (e.g., exponentials).2. Variables connected by power relationships.

I Random variable X with known distribution Px

I Second random variable Y with y = f (x).

I Py(y)dy = Px(x)dx=∑

y|f (x)=y Px(f−1(y)) dy|f ′(f−1(y))|

I Often easier to do byhand...



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

34 of 46

General ExampleI Assume relationship between x and y is 1-1.I Power-law relationship between variables:

y = cx−α, α > 0I Look at y large and x smallI

dy = d(cx−α

)= c(−α)x−α−1dx

invert: dx =−1cα

xα+1dy

dx =−1cα

(yc

)−(α+1)/αdy

dx =−c1/α

αy−1−1/αdy



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

35 of 46

Now make transformation:

Py(y)dy = Px(x)dx

Py(y)dy = Px

(x)︷︸︸︷((yc

)−1/α) dx︷︸︸︷

c1/α

αy−1−1/αdy

I If Px(x)→ non-zero constant as x→ 0 then

Py(y) ∝ y−1−1/α as y→∞.

I If Px(x)→ xβ as x→ 0 then

Py(y) ∝ y−1−1/α−β/α as y→∞.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

Story?CoP What’s

The

S

36 of 46

Example

Exponential distribution

Given Px(x) = 1λe−x/λ and y = cx−α, then

P(y) ∝ y−1−1/α + O(

y−1−2/α)

I Exponentials arise from randomness (easy)...I More later when we cover robustness.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

38 of 46

Gravity

I Select a random point in theuniverse ~x

I Measure the force of gravityF(~x)

I Observe that PF(F) ∼ F−5/2.



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

39 of 46

Matter is concentrated in stars: [9]

I F is distributed unevenlyI Probability of being a distance r from a single star at~x = ~0:

Pr(r)dr ∝ r2dr

I Assume stars are distributed randomly in space(oops?)

I Assume only one star has significant effect at ~x.I Law of gravity:

F ∝ r−2

I invert:r ∝ F−1/2

I Also invert:dF ∝ d(r−2) ∝ r−3dr → dr ∝ r3dF ∝ F−3/2dF .



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

40 of 46

Transformation:

Using r ∝ F−1/2 , dr ∝ F−3/2dF , and Pr(r) ∝ r2

I

PF(F)dF = Pr(r)dr

I

∝ Pr(F−1/2)F−3/2dF

I

∝(

F−1/2)2

F−3/2dF

I

= F−1−3/2dF

I

= F−5/2dF .



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

41 of 46

Gravity:

PF(F) = F−5/2dFI

γ = 5/2

I Mean is finite.I Variance =∞.I A wild distribution.I Upshot: Random sampling of space usually safe

but can end badly...



http://www.uvm.edu


PoCS | @pocsvox



Examples



PLIPLO

References

43 of 46

Extreme Caution!

I PLIPLO = Power law in, power law outI Explain a power law as resulting from another

unexplained power law.I Yet another homunculus argument ()...I Don’t do this!!! (slap, slap)I We need mechanisms!



http://www.uvm.edu


http://en.wikipedia.org/wiki/Homunculus_argument

PoCS | @pocsvox



Examples



PLIPLO

References

@pocsvox

of

Complex

Systems

What’s

the

Story?

Principles

44 of 46

References I

[1] P. S. Dodds and D. H. Rothman.Unified view of scaling laws for river networks.Physical Review E, 59(5):4865–4877, 1999. pdf ()

[2] P. S. Dodds and D. H. Rothman.Scaling, universality, and geomorphology.Annu. Rev. Earth Planet. Sci., 28:571–610, 2000.pdf ()

[3] W. Feller.An Introduction to Probability Theory and ItsApplications, volume I.John Wiley & Sons, New York, third edition, 1968.



http://www.uvm.edu


http://www.uvm.edu/~pdodds/research/papers/others/1999/dodds1999a.pdf

http://www.uvm.edu/~pdodds/research/papers/others/2000/dodds2000a.pdf

PoCS | @pocsvox



Examples



PLIPLO

References

@pocsvox

of

Complex

Systems

What’s

the

Story?

Principles

45 of 46

References II

[4] J. T. Hack.Studies of longitudinal stream profiles in Virginia andMaryland.United States Geological Survey Professional Paper,294-B:45–97, 1957. pdf ()

[5] E. W. Montroll and M. F. Shlesinger.On the wonderful world of random walks, volume XIof Studies in statistical mechanics, chapter 1, pages1–121.New-Holland, New York, 1984.

[6] E. W. Montroll and M. W. Shlesinger.On 1/f noise aned other distributions with long tails.Proc. Natl. Acad. Sci., 79:3380–3383, 1982. pdf ()



http://www.uvm.edu


http://www.uvm.edu/~pdodds/research/papers/others/1957/hack1957a.pdf

http://www.uvm.edu/~pdodds/research/papers/others/1982/montroll1982a.pdf

PoCS | @pocsvox



Examples



PLIPLO

References

@pocsvox

of

Complex

Systems

What’s

the

Story?

Principles

46 of 46

References III

[7] E. W. Montroll and M. W. Shlesinger.Maximum entropy formalism, fractals, scalingphenomena, and 1/f noise: a tale of tails.J. Stat. Phys., 32:209–230, 1983.

[8] A. E. Scheidegger.The algebra of stream-order numbers.United States Geological Survey Professional Paper,525-B:B187–B189, 1967. pdf ()

[9] D. Sornette.Critical Phenomena in Natural Sciences.Springer-Verlag, Berlin, 1st edition, 2003.

[10] J. S. Weitz and H. B. Fraser.Explaining mortality rate plateaus.Proc. Natl. Acad. Sci., 98:15383–15386, 2001.pdf ()



http://www.uvm.edu


http://www.uvm.edu/~pdodds/research/papers/others/1967/scheidegger1967b.pdf

http://www.uvm.edu/~pdodds/research/papers/others/2001/weitz2001a.pdf

Documents

PoCS|@pocsvox Mechanisms for Generating Power-Law Power ...pdodds/teaching/courses/2013-08UVM... · PoCS|@pocsvox Power-Law Mechanisms, Pt. 1 Random Walks The First Return Problem