73
Are Power Laws Useful? Michael P.H. Stumpf Theoretical Systems Biology Group 07/03/2013 Are Power Laws Useful? Michael P.H. Stumpf 1 of 28

Are Powerlaws Useful?

Embed Size (px)

DESCRIPTION

Slides for the Talk at the "Seminar on Power Laws" in Lyon: http://www.ixxi.fr/?p=1943

Citation preview

Page 1: Are Powerlaws Useful?

Are Power Laws Useful?

Michael P.H. Stumpf

Theoretical Systems Biology Group

07/03/2013

Are Power Laws Useful? Michael P.H. Stumpf 1 of 28

Page 2: Are Powerlaws Useful?

Outline

Power Laws: Ubiquituous, Universal, Useful?

Critical Phenomena and Scaling Behaviour

Empirical Power Laws

Simple Models of Network Evolution

More Normal Than Normal

Summary

Are Power Laws Useful? Michael P.H. Stumpf 2 of 28

Page 3: Are Powerlaws Useful?

What Do We Mean By A Powerlaw?

Power Law Relationships

Y ∝ Xλ

log(X )

log(Y )

log(X )

log(p(X ))

Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28

Page 4: Are Powerlaws Useful?

What Do We Mean By A Powerlaw?

Power Law Relationships

Y ∝ Xλ

log(X )

log(Y )

log(X )

log(p(X ))

Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28

Page 5: Are Powerlaws Useful?

What Do We Mean By A Powerlaw?

Power Law Relationships

Y ∝ Xλ

log(X )

log(Y )

log(X )

log(p(X ))

Are Power Laws Useful? Michael P.H. Stumpf Power Laws: Ubiquituous, Universal, Useful? 3 of 28

Page 6: Are Powerlaws Useful?

Power Laws as Physical Scaling Relationships

Laminar Flow

R

x

r

Universality in FlowFor sufficiently low Reynolds number the velocity profile is universalfor all pipes and fluids:

v(r)v(0)

=

(1 −

r2

R2

)

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28

Page 7: Are Powerlaws Useful?

Power Laws as Physical Scaling Relationships

Laminar Flow

R

x

r

Universality in FlowFor sufficiently low Reynolds number the velocity profile is universalfor all pipes and fluids:

v(r)v(0)

=

(1 −

r2

R2

)Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28

Page 8: Are Powerlaws Useful?

Power Laws as Physical Scaling Relationships

1D Josephson-Junction Array in insulating and super conducting phaseSondhi et al., Rev.Mod.Phys. 69:315 (1997).

As the critical temperature is approached from above, the correlationlength increases as some power of the reduced temperature,

ξ ∝(

T − Tc

Tc

)−ν

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 4 of 28

Page 9: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

The Ising Model

↓↓↑↓↑↑

↑↑↓↓↓↓

↑↑↓↓↑↑

↓↓↓↑↓↓

↑↓↑↓↓↓

↑↓↓↑↑↓

T > TC

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

T ≈ TC

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

T < TC

Critical Exponents

For θC = T−TCTC

we have for any macroscopic variable F (θC) in thevicinity of the critical point θC ≈ 0,

F (θC) = const.× θ−λC .

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 10: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

The Ising Model

↑↓↓↑↓↓

↓↑↓↑↑↓

↓↓↓↑↓↑

↓↑↓↑↓↑

↓↑↓↓↑↓

↓↓↓↑↑↑

T > TC

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

T ≈ TC

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

T < TC

Critical Exponents

For θC = T−TCTC

we have for any macroscopic variable F (θC) in thevicinity of the critical point θC ≈ 0,

F (θC) = const.× θ−λC .

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 11: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 12: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 13: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 14: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 15: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 16: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 17: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 18: Are Powerlaws Useful?

Making Sense of Power Laws in Physics

Renormalization Group Theory

↑↑↑↑

↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑↑↑↑

↑↑↑

↑↑

↓↓

↑ ↑

↑ ↑

↓↑ ↑

Renormalization Group Transformation (RGT)The partition function, Z of a physical system must be invariant underthe RGT:

Z =∑[Si ]

exp (−βH[Si ]) =∑[S ′α]

exp(−βH ′[S ′

α])= Z ′

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 5 of 28

Page 19: Are Powerlaws Useful?

Behaviour Around Critical Points

The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.

In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.

Critical PointsAt the critical point the correlation length diverges as a power law

ξ ∝ θ−νc

Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28

Page 20: Are Powerlaws Useful?

Behaviour Around Critical Points

The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.

Critical PointsAt the critical point the correlation length diverges as a power law

ξ ∝ θ−νc

Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28

Page 21: Are Powerlaws Useful?

Behaviour Around Critical Points

The fixed points of the RGT define the possible macroscopic states ofthe system, e.g. ferromagnet vs. paramagnet.Of particular interest are the non-trivial fixed points, 0 < Tc, J <∞,which mark the occurrence of a phase transition.In this case changing the scale does not change the physics. For ourspin-system, for example, there is long-range correlation whichextends beyond the lattice spacing.

Critical PointsAt the critical point the correlation length diverges as a power law

ξ ∝ θ−νc

Note that knowledge of the critical point does not tell us necessarilywhat the state of the system on either side is.

Are Power Laws Useful? Michael P.H. Stumpf Critical Phenomena and Scaling Behaviour 6 of 28

Page 22: Are Powerlaws Useful?

Heureka!

Mason Porter, http://www.quickmeme.com/meme/3sqh80/

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 7 of 28

Page 23: Are Powerlaws Useful?

Simple Scaling Laws

West et al., PNAS, 99:2473 (2002).May & Stumpf, Science, 290:2084 (2000).

Plausible Theories and Simple ModelsEven when simple, plausible physical arguments can be put forward

In an area that is twice as large, we can accommodate fourtimes as many species, N ∝ A2

the empirical results are better described by other phenomenologicaldistributions.

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 8 of 28

Page 24: Are Powerlaws Useful?

Simple Scaling Laws

West et al., PNAS, 99:2473 (2002).May & Stumpf, Science, 290:2084 (2000).

Plausible Theories and Simple ModelsEven when simple, plausible physical arguments can be put forward

In an area that is twice as large, we can accommodate fourtimes as many species, N ∝ A2

the empirical results are better described by other phenomenologicaldistributions.

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 8 of 28

Page 25: Are Powerlaws Useful?

Illusions of Invariance

We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.

The Inverse is not true! A slope of 1.0 does not imply invariance.

For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have

R2 =Var[log(w)] − Var[log(c)]

Var[log(w)]=

Var[log(m)]

Var[log(m)] + Var[log(c)]

Nee et al., Science, 309:1236 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28

Page 26: Are Powerlaws Useful?

Illusions of Invariance

We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.

The Inverse is not true! A slope of 1.0 does not imply invariance.

For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have

R2 =Var[log(w)] − Var[log(c)]

Var[log(w)]=

Var[log(m)]

Var[log(m)] + Var[log(c)]

Nee et al., Science, 309:1236 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28

Page 27: Are Powerlaws Useful?

Illusions of Invariance

We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.

The Inverse is not true! A slope of 1.0 does not imply invariance.

For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].

We then have

R2 =Var[log(w)] − Var[log(c)]

Var[log(w)]=

Var[log(m)]

Var[log(m)] + Var[log(c)]

Nee et al., Science, 309:1236 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28

Page 28: Are Powerlaws Useful?

Illusions of Invariance

We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.

The Inverse is not true! A slope of 1.0 does not imply invariance.

For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have

R2 =Var[log(w)] − Var[log(c)]

Var[log(w)]=

Var[log(m)]

Var[log(m)] + Var[log(c)]

Nee et al., Science, 309:1236 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28

Page 29: Are Powerlaws Useful?

Illusions of Invariance

We consider off-spring weaning weight, w , and maternal weight, m,and seek to understand w/m. If this ratio is invariant then log(w)plotted against log(m) should have a regression slope of 1.0.

The Inverse is not true! A slope of 1.0 does not imply invariance.

For example, consider the model where log(w) = log(m) + log(c)where log(c) is a non-normally distributed error, ε, c ∼ U[0, 1].We then have

R2 =Var[log(w)] − Var[log(c)]

Var[log(w)]=

Var[log(m)]

Var[log(m)] + Var[log(c)]

Nee et al., Science, 309:1236 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 9 of 28

Page 30: Are Powerlaws Useful?

Scale-Free Networks

The Beginnings

A:

Actor collaboration; B: WWW; C: Power grid

Barabasi & Albert, Science, 286:510 (1999).

Ever since many real-worldnetworks have been“discovered” to be

scale-free. Partial alignment of Human and Fly protein-interaction network (PIN).

What Are “Scale-Free” Networks?Typically this means that the degree distributions is scale-free, i.e.

Pr(αk)Pr(k)

= const. ∀k .

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 10 of 28

Page 31: Are Powerlaws Useful?

Scale-Free Networks

The Beginnings

A:

Actor collaboration; B: WWW; C: Power grid

Barabasi & Albert, Science, 286:510 (1999).

Ever since many real-worldnetworks have been“discovered” to be

scale-free. Partial alignment of Human and Fly protein-interaction network (PIN).

What Are “Scale-Free” Networks?Typically this means that the degree distributions is scale-free, i.e.

Pr(αk)Pr(k)

= const. ∀k .

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 10 of 28

Page 32: Are Powerlaws Useful?

Are Networks Scale-Free?

Saccharomyces cerevisiae PIN

If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by

Lm(θm) =

n∏i=1

Prm(k |θ).

This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).

So far, no network has been found to bescale-free when proper statistical analysis wasapplied.

Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28

Page 33: Are Powerlaws Useful?

Are Networks Scale-Free?

Saccharomyces cerevisiae PIN

If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by

Lm(θm) =

n∏i=1

Prm(k |θ).

This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).

So far, no network has been found to bescale-free when proper statistical analysis wasapplied.

Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28

Page 34: Are Powerlaws Useful?

Are Networks Scale-Free?

Saccharomyces cerevisiae PIN

If D = {d1, . . . , dn} is the empirical degreedistribution and Prm(k |θ) the probability toobserve degree k for model m ∈M, with{M1, . . . , Mq}, and parameter θ then thelikelihood is given by

Lm(θm) =

n∏i=1

Prm(k |θ).

This allows us to compare different models inlight of the data (using e.g. the AIC or BIC toenforce parsimony).

So far, no network has been found to bescale-free when proper statistical analysis wasapplied.

Stumpf & Ingram, Europhys. Lett. 71:152 (2005); Tanaka et al., FEBS Lett. 579:5140 (2005); Khanin & Wit, J.Comp.Biol.13:810 (2006).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 11 of 28

Page 35: Are Powerlaws Useful?

Real Networks Are Scale Rich

Tanaka, Phys.Rev.Lett. 94:168101 (2005).

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 12 of 28

Page 36: Are Powerlaws Useful?

Other Statistical Challenges

Incomplete and Noisy Data

Sub-nets of scalefree networks arenot scale-free.Stumpf et al., PNAS 102:4221,(2005);Wiuf & Stumpf, Proc.Roy.Soc. A462:1181 (2006).

Truncated Power Laws

Pr(k) ∝ k−λ for klow < k < khigh

It is hard to see what is gained by this: the statistical power is lowerthan that of other mixture models, and the elegance of theinterpretation is no longer given.

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 13 of 28

Page 37: Are Powerlaws Useful?

Other Statistical Challenges

Incomplete and Noisy Data

Sub-nets of scalefree networks arenot scale-free.Stumpf et al., PNAS 102:4221,(2005);Wiuf & Stumpf, Proc.Roy.Soc. A462:1181 (2006).

Truncated Power Laws

Pr(k) ∝ k−λ for klow < k < khigh

It is hard to see what is gained by this: the statistical power is lowerthan that of other mixture models, and the elegance of theinterpretation is no longer given.

Are Power Laws Useful? Michael P.H. Stumpf Empirical Power Laws 13 of 28

Page 38: Are Powerlaws Useful?

Evolving Networks

α

δ

δ

γ

γ

α

δ

δAre Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 14 of 28

Page 39: Are Powerlaws Useful?

Evolving Networks

Model-Based Evolutionary Analysis• For sequence data we use models of nucleotide substitution in

order to infer phylogenies in a likelihood or Bayesian framework.• None of these models — even the general time-reversible model

— are particularly realistic; but by allowing for complicating factorse.g. rate variation we capture much of the variability observedacross a phylogenetic panel.

• Modes of network evolution will be even more complicated andexhibit high levels of contingency; moreover the structure andfunction of different parts of the network will be intricately linked.

• Nevertheless we believe that modelling the processes underlyingthe evolution of networks can provide useful insights; in particularwe can study how functionality is distributed across groups ofgenes.

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 14 of 28

Page 40: Are Powerlaws Useful?

Network Evolution Models

(a) Duplication attachment (b) Duplication attachmentwith complimentarity

(c) Linear preferentialattachment

wi

wj

(d) General scale-free

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 15 of 28

Page 41: Are Powerlaws Useful?

ABC on Networks

Summarizing Networks• Data are noisy and incomplete.• We can simulate models of network

evolution, but this does not allow us tocalculate likelihoods for all but verytrivial models.

• There is also no sufficient statistic thatwould allow us to summarize networks,so ABC approaches require somethought.

• Many possible summary statistics ofnetworks are expensive to calculate.

Full likelihood: Wiuf et al., PNAS (2006).

ABC: Ratman et al., PLoS Comp.Biol. (2008).

Stumpf & Wiuf, J. Roy. Soc. Interface (2010).

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 16 of 28

Page 42: Are Powerlaws Useful?

Graph Spectrum

a

b

c

d e0 1 1 1 01 0 1 1 01 1 0 0 01 1 0 0 10 0 0 1 0

a b c d e

abcde

A =

Graph SpectraGiven a graph G comprised of a set of nodes N and edges (i, j) ∈ Ewith i, j ∈ N, the adjacency matrix, A, of the graph is defined by

ai,j =

{1 if (i, j) ∈ E ,

0 otherwise.

The eigenvalues, λ, of this matrix provide one way of defining thegraph spectrum.

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 17 of 28

Page 43: Are Powerlaws Useful?

Spectral Distances

A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,

D(A, B) =∑

i,j

(ai,j − bi,j)2.

However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance

D(A, B) > D ′h(A, B) =

∑i,j

(ai,j − bh(i),h(j))2,

Given a spectrum (which is relatively cheap to compute) we have

D ′(A, B) =∑

l

(λ(α)l − λ

(β)l

)2

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28

Page 44: Are Powerlaws Useful?

Spectral Distances

A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,

D(A, B) =∑

i,j

(ai,j − bi,j)2.

However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance

D(A, B) > D ′h(A, B) =

∑i,j

(ai,j − bh(i),h(j))2,

Given a spectrum (which is relatively cheap to compute) we have

D ′(A, B) =∑

l

(λ(α)l − λ

(β)l

)2

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28

Page 45: Are Powerlaws Useful?

Spectral Distances

A simple distance measure between graphs having adjacencymatrices A and B, known as the edit distance, is to count the numberof edges that are not shared by both graphs,

D(A, B) =∑

i,j

(ai,j − bi,j)2.

However for unlabelled graphs we require some mapping h fromi ∈ NA to i ′ ∈ NB that minimizes the distance

D(A, B) > D ′h(A, B) =

∑i,j

(ai,j − bh(i),h(j))2,

Given a spectrum (which is relatively cheap to compute) we have

D ′(A, B) =∑

l

(λ(α)l − λ

(β)l

)2

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 18 of 28

Page 46: Are Powerlaws Useful?

ABC using Graph Spectra

For an observed network, N, and a simulated network, Sθ, we use thedistance between the spectra

D ′(N, Sθ) =∑

l

(λ(N)l − λ

(S)l

)2,

in our ABC SMC procedure. Note that this distance is a close lowerbound on the distance between the raw data; we therefore do nothave to bother with summary statistics.Also, calculating graph spectra costs as much as calculating otherO(N3) statistics (such as all shortest paths, the network diameter orthe within-reach distribution).Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 19 of 28

Page 47: Are Powerlaws Useful?

Protein Interaction Network Data

Species Proteins Interactions Genome size Sampling fraction

S.cerevisiae 5035 22118 6532 0.77

D. melanogaster 7506 22871 14076 0.53

H. pylori 715 1423 1589 0.45

E. coli 1888 7008 5416 0.35

Model

Mod

el p

roba

bilit

y

0.0

0.1

0.2

0.3

0.4

0.5

DA DAC LPA SF DACL DACR

Organism

S.cerevisae

D.melanogaster

H.pylori

E.coli

Model Selection• Inference here was based on all

the data, not summarystatistics.

• Duplication models receive thestrongest support from the data.

• Several models receive supportand no model is chosenunambiguously.

Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28

Page 48: Are Powerlaws Useful?

Protein Interaction Network Data

Species Proteins Interactions Genome size Sampling fraction

S.cerevisiae 5035 22118 6532 0.77

D. melanogaster 7506 22871 14076 0.53

H. pylori 715 1423 1589 0.45

E. coli 1888 7008 5416 0.35

Model

Mod

el p

roba

bilit

y

0.0

0.1

0.2

0.3

0.4

0.5

DA DAC LPA SF DACL DACR

Organism

S.cerevisae

D.melanogaster

H.pylori

E.coli

Model Selection• Inference here was based on all

the data, not summarystatistics.

• Duplication models receive thestrongest support from the data.

• Several models receive supportand no model is chosenunambiguously.

Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28

Page 49: Are Powerlaws Useful?

Protein Interaction Network Data

Species Proteins Interactions Genome size Sampling fraction

S.cerevisiae 5035 22118 6532 0.77

D. melanogaster 7506 22871 14076 0.53

H. pylori 715 1423 1589 0.45

E. coli 1888 7008 5416 0.35

Model

Mod

el p

roba

bilit

y

0.0

0.1

0.2

0.3

0.4

0.5

DA DAC LPA SF DACL DACR

Organism

S.cerevisae

D.melanogaster

H.pylori

E.coli

Model Selection• Inference here was based on all

the data, not summarystatistics.

• Duplication models receive thestrongest support from the data.

• Several models receive supportand no model is chosenunambiguously.

Thorne & Stumpf, J.Roy.Soc. Interface, 9:2653 (2012).

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 20 of 28

Page 50: Are Powerlaws Useful?

PIN Model Evolution

Are Power Laws Useful? Michael P.H. Stumpf Simple Models of Network Evolution 21 of 28

Page 51: Are Powerlaws Useful?

Power Laws As Phenomenological Models

We have seen above that power laws emerge naturally in the contextof continuous phase transitions; but we have also seen that they offerat best limited insights for finite systems.Why do they nevertheless appear so often?

Revisiting the Renormalization GroupLet f (X ) be a probability distribution. Then the RGT acting on it isdefined as

Taf (X = x) := |a|∫∞−∞ f (ax − s)f (s)ds.

Taf (X ) is the pdf of the random variable

Y =X1 + X2

awith X1, X2 ∼ f (X ).

Here a controls the qualitative properties of the transformation.Calvo et al., J.Stat.Phys. 141:409 (2010).

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 22 of 28

Page 52: Are Powerlaws Useful?

Power Laws As Phenomenological Models

We have seen above that power laws emerge naturally in the contextof continuous phase transitions; but we have also seen that they offerat best limited insights for finite systems.Why do they nevertheless appear so often?

Revisiting the Renormalization GroupLet f (X ) be a probability distribution. Then the RGT acting on it isdefined as

Taf (X = x) := |a|∫∞−∞ f (ax − s)f (s)ds.

Taf (X ) is the pdf of the random variable

Y =X1 + X2

awith X1, X2 ∼ f (X ).

Here a controls the qualitative properties of the transformation.Calvo et al., J.Stat.Phys. 141:409 (2010).

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 22 of 28

Page 53: Are Powerlaws Useful?

Renormalization Group Transformation of PDFs

Here we are after the fixed points, f0, of the RGT,

Taf0(x) = f0(x)

When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,

Eg [Y n] = ETaf [Y n] =1an

n∑i=0

(ni

)Ef[X i]Ef

[X n−i]

Some Fixed Points of the RGT

a <√

2 a =√

2 a >√

2

limm→∞ET m

a f0 [x2] =∞ Ef0 [x

2n] = Ef0 [x2])n (2n)!

n!2n limm→∞ET m

a f0 [x2] = 0

— N(0, 1) δ(x)

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28

Page 54: Are Powerlaws Useful?

Renormalization Group Transformation of PDFs

Here we are after the fixed points, f0, of the RGT,

Taf0(x) = f0(x)

When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,

Eg [Y n] = ETaf [Y n] =1an

n∑i=0

(ni

)Ef[X i]Ef

[X n−i]

Some Fixed Points of the RGT

a <√

2 a =√

2 a >√

2

limm→∞ET m

a f0 [x2] =∞ Ef0 [x

2n] = Ef0 [x2])n (2n)!

n!2n limm→∞ET m

a f0 [x2] = 0

— N(0, 1) δ(x)

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28

Page 55: Are Powerlaws Useful?

Renormalization Group Transformation of PDFs

Here we are after the fixed points, f0, of the RGT,

Taf0(x) = f0(x)

When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,

Eg [Y n] = ETaf [Y n] =1an

n∑i=0

(ni

)Ef[X i]Ef

[X n−i]

Some Fixed Points of the RGT

a <√

2 a =√

2 a >√

2

limm→∞ET m

a f0 [x2] =∞ Ef0 [x

2n] = Ef0 [x2])n (2n)!

n!2n limm→∞ET m

a f0 [x2] = 0

— N(0, 1) δ(x)

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28

Page 56: Are Powerlaws Useful?

Renormalization Group Transformation of PDFs

Here we are after the fixed points, f0, of the RGT,

Taf0(x) = f0(x)

When all moments of f (X ) exist and are finite then we can determinethe moments of the transformed distribution,

Eg [Y n] = ETaf [Y n] =1an

n∑i=0

(ni

)Ef[X i]Ef

[X n−i]

Some Fixed Points of the RGT

a <√

2 a =√

2 a >√

2

limm→∞ET m

a f0 [x2] =∞ Ef0 [x

2n] = Ef0 [x2])n (2n)!

n!2n limm→∞ET m

a f0 [x2] = 0

— N(0, 1) δ(x)

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 23 of 28

Page 57: Are Powerlaws Useful?

Central Limit Theorems

The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.

Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),

ϕ0(t/a)2 = ϕ0(t)

General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,

ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)

)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28

Page 58: Are Powerlaws Useful?

Central Limit Theorems

The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),

ϕ0(t/a)2 = ϕ0(t)

General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,

ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)

)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28

Page 59: Are Powerlaws Useful?

Central Limit Theorems

The conventional central limit theorem emerges as the fixed point ofthe RGT for distributions where all moments are finite.Now we look at the characteristics function, φ(t) = Ef [eitX ] and obtainfor the fixed points, φ0(t),

ϕ0(t/a)2 = ϕ0(t)

General Fixed PointsThe general fixed points of the RGT are given by the Levy-stable laws,

ϕ0(t) = Sα,A)(k) := exp(−A|t |αθ(t) − A|t |αθ(−k)

)with |a| = 21/α, A the complex conjugate of A and θ(x) the Heavisidestep function.For α = 2 we recover the Gaussian, and for all α < 2 we obtain theheavy-tailed stable laws. For α < 1 the mean is infinite.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 24 of 28

Page 60: Are Powerlaws Useful?

Power Law vs. Gaussian Distribution

From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.

Gaussian Distributions are stable under aggregation andmarginalization.

Scaling Distributions are stable under aggregation, mixture,maximisation and marginalization.

More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).

It is thus also easy to come up with simple mechanisms that give riseto dispersed data.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28

Page 61: Are Powerlaws Useful?

Power Law vs. Gaussian Distribution

From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and

marginalization.

Scaling Distributions are stable under aggregation, mixture,maximisation and marginalization.

More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).

It is thus also easy to come up with simple mechanisms that give riseto dispersed data.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28

Page 62: Are Powerlaws Useful?

Power Law vs. Gaussian Distribution

From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and

marginalization.Scaling Distributions are stable under aggregation, mixture,

maximisation and marginalization.

More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).

It is thus also easy to come up with simple mechanisms that give riseto dispersed data.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28

Page 63: Are Powerlaws Useful?

Power Law vs. Gaussian Distribution

From the above we see that the conventional CLT is in fact a veryspecial case of a much more general form of the CLT.Thus we would expect fat-tailed distributions (by whichever sensibledefinition) to occur frequently.Gaussian Distributions are stable under aggregation and

marginalization.Scaling Distributions are stable under aggregation, mixture,

maximisation and marginalization.

More Normal Than NormalIn this sense scaling or fat-tailed distributions should be expected tooccur very frequently. For low-variability data we obtain the Gaussianas a fixed point.Willinger et al., Proceedings of the 2004 Winter Simulation Conference, 130 (2004).

It is thus also easy to come up with simple mechanisms that give riseto dispersed data.

Are Power Laws Useful? Michael P.H. Stumpf More Normal Than Normal 25 of 28

Page 64: Are Powerlaws Useful?

Power Laws, Evidence, Usefulness

MechanisticSophistication

Statistical Support

AllometricScaling

Zipf’s Law

Internet

C. elegansnervoussystem

S. cerevisiae PINStumpf & Porter, Science, 335:665 (2012).

Are Power Laws Useful? Michael P.H. Stumpf Summary 26 of 28

Page 65: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?

1. Would a power law relationship offer profound new insights?

◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 66: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?

◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 67: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.

◦ Could you just have a very dispersed random variable, Y?2. Does it extend over at least three orders of magnitude in both

variables (sanity check)?3. Does the power law relationship hold up in comparison to other

distributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 68: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 69: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 70: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 71: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 72: Are Powerlaws Useful?

How to Check if You Have a Power Law in Your Data?

Y ∝ X−λ YES or NO ?1. Would a power law relationship offer profound new insights?◦ Simply reporting a power law or scaling relationship is not exciting.◦ Could you just have a very dispersed random variable, Y?

2. Does it extend over at least three orders of magnitude in bothvariables (sanity check)?

3. Does the power law relationship hold up in comparison to otherdistributions (log-normal, stretched exponential, negativebinomial)?

4. Have you got a non-trivial and meaningful theoretical model thatgives rise to the power law and which yields mechanistic insights?

A Useful Scientific TheoryFailing at any of these hurdles does not mean that the scientificproblem is boring or trivial. Power laws add a lot to the theory ofcritical phenomena/fluid dynamics etc. but very little elsewhere.

Are Power Laws Useful? Michael P.H. Stumpf Summary 27 of 28

Page 73: Are Powerlaws Useful?

Acknowledgements

• Imperial CollegeLondon◦ Thomas Thorne◦ William Kelly

• OxfordUniversity◦ Robert May◦ Mason Porter

• KopenhagenUniversity◦ Carsten Wiuf

Theoretical Systems Biology Group

www.theosysbio.bio.ic.ac.uk

Are Power Laws Useful? Michael P.H. Stumpf Summary 28 of 28