Physics and Complexity

David Sherrington University of Oxford & Santa Fe Institute

Blaise Pascal Medal in Physics 2010 European Academy of Sciences

Athens 5 Nov 2010

Blaise Pascal (1623-1662)

Mathematician

Physicist

Engineer &

Philosopher

Innovative Genius

Physics

Dictionary definition: Branch of science concerned with the nature and properties of matter and energy

But today I want to use it as a mind-set with valuable methodologies

and to show application to complex systems in many different arenas

Complexity

•  Many body systems

•  Cooperative behaviour complex – not simply anticipatable from microscopics

•  occurs even with simple individual units and simple interaction rules

– with surprising conceptual similarities between superficially different systems

Aim today

Illustrate use of statistical physics methodology to understand complexity and its ubiquity

via simple models, pictures and comparisons

Give flavour of concepts

Point out some relationships to pioneering Pascal

Key ingredients

Frustration Conflicts

Disorder Quenched / self-induced / time-dependent

Methodologies

•  Analysis •  Experiment

•  Conceptualization •  Computer simulation

Symbiosis

But only a broad brush picture today

The Dean’s Problem

•  Dean to allocate N students to two dorms

•  Some students like one another; prefer same dorm •  Others dislike one another; prefer different dorms

•  Cannot satisfy all →

•  Best compromise for whole student body?

- Frustration or or…

J: Inter-student friendship: +/-

The Dean’s Problem as combinatorial optimization

H = + JijSiS j

( ij )∑

Dorm A/B

Maximise a Happiness function

Students, i,j

J: Inter-student friendship: +/-

The Dean’s Problem as combinatorial optimization

H = + JijSiS j

( ij )∑

Dorm A/B

Maximise a Happiness function

Students, i,j

Very difficult for general {J} with both positive & negative Jij ; 2N choices; NP-complete N students: N large

J: Inter-student friendship

The Dean’s Problem as combinatorial optimization

H = + JijSiS j

( ij )∑

Dorm A/B

Maximise a Happiness function

Students, i,j

Very difficult for general {J} with both positive & negative Jij ; 2N choices; NP-complete

Random {J}: characterize by drawing from a probability distribution P(J)

N students: N large

Aside: Pascal and Fermat

Invented the mathematical theory of probability

Spin Glasses

•  Magnetic alloys; e.g. CuMn, AuFe

•  Theory: All sites occupied, random exchange

•  Computer simulation: experiments on model systems similar behaviour

Pascal: Computing

Invented and built the first mechanical

calculating machine

Pascaline

Fore-runner of modern computers

Recognised and honoured by naming of a computer programming language

‘Spin glass’ properties •  Non-periodic/amorphous collective ordering

•  Non- or very slow equilibration

•  Aging, rejuvenation, memory

AuFe

Dean’s model ~ Range-free Spin Glass Model (SK)

.

H = − JijSiS j

( ij )∑

Hamiltonian Exchange interactions Spins ~ magnetic moments

Unhappiness Friendship Dorm allocation

A B

Dean’s problem

Spin glass

Our first equivalence

Minimise

Paradigmic control function

.

H = − JijSiS j

( ij )∑

Statistical physics: Typical properties Expectation values of physical observables

Range-free: soluble but very subtle

with several different identifications of symbols and different types of disorder and frustration

For this talk

Random: Characterize by probability distribution P(J): quenched

Finite-temperature equilibrium: P(S) ~ exp(-H{J}(S)/T)

Paradigmic cartoon

Rugged Landscape

Many metastable states

Hierarchy Valleys within valleys

Hard to minimise/maximize: sticks: glassy

Cost to minimise

Coordinates

Simple algorithms

Smooth local

motion

c.f. Fitness to maximise

Where does this cartoon come from?

Simulations, analytic calculations, anzätze

qSS '

= N −1 σ ii∑ S

σ i

S '

P(q) = WSWS 'δ (q − qSS ' )

SS '∑

qαδ = qβδ ≤ qβδ

Conventional system: single δ fn Complex system: structure

e.g. SK: ultrametric phylogenetic tree

Overlap

Overlap distribution

Via some novel mathematical structures that I believe Pascal would have loved

Parisi’s ansatz

Replica Symmetric

Replica Symmetry Broken Hierarchical sub-division

q0 q1

q2

qαβ ; α ,β = 1,..n, n→ 0 n x n matrix but with non-integral n 0

Spin glass

Pascal’s triangle

.

Pascal_modulo 2

Sierpinski gasket (Hierarchical fractal)

Reminder

Temperature •  Natural for real physics

•  Often useful for practical optimization to introduce an artificial temperature

and take limit as it is reduced

•  Other analogues in other problems

P(S) ~ exp(-H{J}(S)/TA)

Landscape paradigm for hard optimization

Cost

Peaks are obstacles

Steepest descent gets stuck

Simulated annealing

Probabilistic hill-climbing Add artificial annealing ‘temperature’

Variables

Cost TA

Annealing temperature

P(move) ~ exp(−ΔC / TA )

Simulated annealing

Gradually reduce TA

Variables

Cost TA

Annealing temperature

Simulated annealing

Variables

Cost

Hopefully

H = − JijSiS jij∑

+/- ; excitatory/inhibitory

Neural network: Hopfield model

Neurons, rate of firing

T ~ synaptic sigmoidal response rounding

T O

I I O

Synapses

Ii = Jij

j∑ S j

Control function

Attractors

•  Associative memory ‘attractors’ memorized patterns

•  Retrieval basins

•  Many memories many attractors

require frustration

Stored in {J}

Phase space

Neural ‘rugged landscape’

Valleys ~ attractors Sculpture ~ learning {Si } {Jij}

Different timescales fast retrieval slow learning

Proteins

Must fold fairly easily

Minimal frustration

Proteins: Heteropolymrers

Many amino acids Frustrated interactions

Random heteropolymers

In general, very frustrated Fold poorly, glassy

Evolution:

Initial random soup

Fast: attempt to fold

Slower time-scale: Reproduction/mutation Good folders selected

Folding funnel Wolynes et. al.

Analogies

Glassy/slow More minimal frustration/faster

Spin glass Neural network SK Hopfield

Random heteropolymer Protein Wolynes

Price

Time

Different strategies (Disorder)

Common information

(Mean field) (Range-free)

Learn from Experience (Dynamics II)

Not all can win (Frustration)

Buy & sell (Dynamics I)

Stockmarket

Minority game N agents 2 choices

Aim to be in minority

Individual strategies Collective consequence •  act on common information (e.g. minority choice for last m steps) •  preferences modified by experience (keep point-score)

Simple minimalist model

Phase transitions & ergodicity-breaking

Non-ergodic

Ergodic

Scaled size of information space

Ergodic Non-ergodic

Spin glass susceptibility Minority game: volatility

Suggests similar concepts & methodologies

AuFe

Experimentt

Simulation

.

Recall

Minority game

a

Phase space

H = + Jijsiij∑ sj

Jij= ξ

i

µ

µ

∑ ξj

µ

Many repellors

Analogy with Hopfield neural network but different

c.f. attractors in neural network

Effective control function

Stochastic noise/’temperature’

•  Energy Free Energy – Temperatures smoothes free energy

•  Reduces ruggedness

– Neural networks •  Noise reduces false minima in effective

landscape

– Minority game •  Stochastic uncertainty reduces oscillations •  & tabala raza volatility

Random K-SAT; K-spin glass

HARD-SAT

N/M

0

UNSAT/ Spin Glass

SAT/ Para

αc-1

α d-1

Simple algorithms stick Theoretically achievable limit

Phase transitions T

Dynamical transition

Thermodynamic trans

Extension

Fascinating Physics Novel maths

Complexity Science

Transfers Opportunities

.

Blaise Pascal (1623-1662)

I think Pascal would have enjoyed the challenges posed by complex many-body systems

I am honoured by the award of this prize bearing his name

and by the EAS recognition

Th

Collaborators

Tomaso Aste Jay Banavar Arnaud Buhot Andrea Cavagna Premla Chandra Tuck Choy Ton Coolen Dinah Cragg Lexie Davison Malcolm Dunlop Alex Duering Sam Edwards David Elderfield

Fernando Nobre Dominic O’Kane Reinhold Oppermann Richard Penney Albrecht Rau Hans-Juergen Sommers Nicolas Sourlas Byron Southern Mike Thorpe Tim Watkin Andreas Wendemuth Werner Wiethege Michael Wong

Julio Fernandez Juan Pedro Garrahan S.K.Ghatak Irene Giardina Peter Gillin Paul Goldbart Lev Ioffe Peter Kahn Scott Kirkpatrick Stephen Laughton Esteban Moro Peter Mottishaw Normand Mousseau Hidetoshi Nishimori

Thanks Teachers, collaborators,

students, postdocs, friends

Tomaso Aste

Jay Banavar

Ludovic Berthier

Stefan Boettcher

Arnaud Buhot

Andrea Cavagna

Premla Chandra

Tuck Choy

Ton Coolen

Dinah Cragg

Lexie Davison

Andrea De Martino

Malcolm Dunlop

Alex Duering

David Elderfield

Julio Fernandez

Tobias Galla

Juan Pedro Garrahan

S.K.Ghatak

Irene Giardina

Peter Gillin

Paul Goldbart

Lev Ioffe Robert Jack Alexandre Lefevre

Turab Lookman

Peter Kahn

Scott Kirkpatrick

Helmut Katzgraber

Stephen Laughton

Francesco Mancini

Marc Mezard

Esteban Moro

Peter Mottishaw

Normand Mousseau

Hidetoshi Nishimori Fernando Nobre Dominic O’Kane

Reinhold Oppermann

Giorgio Parisi

Richard Penney

Albrecht Rau

Avadh Saxena

Manuel Schmidt

Hans-Juergen Sommers

Nicolas Sourlas

Byron Southern

Mike Thorpe

Tim Watkin

Andreas Wendemuth

Werner Wiethege

Stephen Whitelam

Peter Wolynes

Michael Wong

Phil Anderson Sam Edwards Arvid Herzenberg Frank Jackson Harry Kelly Walter Kohn