John Doyle Control and Dynamical Systems Caltech Theory of Complex Networks

John DoyleControl and Dynamical

Systems Caltech

Theory of

Complex

Networks

FinanceTransportation

Energy

Information

Consumer

Utilities

Manufacturing

Commerce

Health

Our lives are run by/with networks

Emergency

ConvergentNetworks

Transportation

Energy

Information

Consumer

Manufacturing

CommerceHealth

Environment

Emergency

Finance

Utilities

Convergent networking: the promise

Ubiquitous computing, communications, and

control• that is embedded and

intertwined• via sensors and actuators • in complex networks of

networks, with layers of protocols and feedback.

Resulting in:

• Seamless integration and automation of everything

• Efficient and economic operation

• Robust and reliable services

ConvergentNetworks

Transportation

Energy

Information

ConsumerManufacturing

CommerceHealth

Environment

Emergency

Finance

Utilities

Convergent networking: the reality

• Right now, back in Los Angeles, we can experience (in addition to smog, earthquakes, fires, floods, riots, lawyers,…)– Widespread and prolonged power outages from lightning strikes in

Washington (or just “nonequilibrium market fluctuations”).

– Widespread and prolonged flight delays from weather or ATC software glitches in Chicago or Atlanta.

– Internet meltdowns caused by hackers in Moscow.

– Financial meltdowns caused by brokers in Singapore.

• What can we expect?– Widespread and prolonged meltdowns of integrated power,

transportation, communication, and financial networks caused by lightning strikes in Singapore or a new release of MS Windows 2020?

Elements of systems

• Sense the environment and internal state

• Extract what’s novel

• Communicate or store what’s novel

• Extract what’s useful

• Compute decisions based on what’s useful

• Take action

• Evaluate consequences

• Repeat

DataIs not novel informationIs not useful Information

Is not knowledge Is not understanding

Is not wisdomIs not action Is not results

HARDER

We want results

Two great abstractions of the 20th Century

1. Separate systems engineering into control, communications, and computing

– Theory

– Applications

2. Separate systems from physical substrate• Facilitated massive, wildly successful, and explosive

growth in both mathematical theory and technology…

• …but creating a new Tower of Babel where even the experts do not read papers or understand systems outside their subspecialty.

Tower of Babel

• Issues for theory– Rigor– Relevance– Accessibility

• Spectacular success on the first two• Little success on the last one, which is critical for a

multidisciplinary approach to systems biology• Perhaps all three is impossible?• (In contrast, there are whole research programs in

“complex systems” devoted exclusively to accessibility. They have been relatively “popular,” but can be safely ignored in biology.)

Biology and advanced technology

• Biology– Integrates control, communications, computing– Into distributed control systems– Built at the molecular level

• Advanced technologies will do the same• We need new theory and math, plus

unprecedented connection between systems and devices

• Two challenges for greater integration:– Unified theory of systems– Multiscale: from devices to systems

Compute

Communicate Communicate

StoreCommunicate

Communications and computing

Compute

Sense

EnvironmentEnvironment

Act

Communicate Communicate

StoreCommunicate

Computation

Devices

Dynamical SystemsDynamical Systems

DevicesCommunication Communication

Control

From• Software to/from human• Human in the loop

To• Software to Software• Full automation• Integrated control,

comms, computing• Closer to physical

substrateCompute

Communicate Communicate

Store

Communicate

Computation

Devices

Dynamical SystemsDynamical Systems

Devices

Communication Communication

Control

• New capabilities & robustness• New fragilities & vulnerabilities

Theoretical foundations

• Computational complexity: decidability, P-NP-coNP

• Information theory: source and channel coding

• Control theory: feedback, optimization, games

• Dynamical systems: dynamics, bifurcation, chaos

• Statistical physics: phase transitions, critical phenomena

• Unifying theme: uncertainty management

• Different abstractions and relaxations

• Integrating these theories involves new math, much not traditionally be viewed as “applied,” e.g..– Perturbation theory of operator Banach algebras

– Semi-algebraic geometry

Uncertainty management

• Each domain faces similar abstract issues and tradeoffs, but with differing details:

• Sources of uncertainty

• Limited resources

• Robust strategies

• Fundamental tradeoffs

• Ignored issues

Control theory

• Sources of uncertainty: plant uncertainty and sensor noise

• Limited resources: sensing, actuation, and computation

• Robust strategies: feedback control and related methods

• Fundamental tradeoffs: Bode’s integral formula, RHP zeros, saturations, …

• Ignored issues: communications in distributed control, software reliability

Information theory

• Sources of uncertainty: source and channel

• Limited resources: storage, bandwidth, and computation

• Robust strategies: coding

• Fundamental tradeoffs: capacity, rate-distortion

• Ignored issues: feedback and dynamics

Computation complexity

• Sources of uncertainty: intractability, problem instance

• Limited resources: computer time and space

• Robust strategies: algorithms

• Fundamental tradeoffs: P/NP/Pspace/undecidable

• Ignored issues: real-time, uncertainty in physical systems

Software correctness

• Sources of uncertainty: bugs, user inputs

• Limited resources: computer time and space

• Robust strategies: formal verification

• Fundamental tradeoffs: computational complexity

• Ignored issues: real-time, uncertainty in physical systems

Multiscale physics

• Sources of uncertainty: initial conditions, unmodeled dynamics, quantum mechanics

• Limited resources: computer time and space, measurements

• Robust strategies: coarse graining, renormalization??

• Fundamental tradeoffs: energy/matter, entropy, quantum, etc…

• Ignored issues: robustness, rigor, computation, etc• (This looks mostly fixable.)

Unified theory of uncertainty management

• Sources of uncertainty: plant, multiscale physics, sensors, channels, bugs, user inputs

• Limited resources: computer time and space, energy, materials, bandwidth, actuation

• Robust strategies: ??

• Fundamental tradeoffs: ??

• Ignored issues: human factors

Progress

• Unified view of web and internet protocols– Good place to start

– Add feedback and dynamics to communications

– Observations: fat tails (Willinger)

– Theory: Source coding and web layout (Doyle)

– Theory: Channel coding and congestion control (Low)

• Unified view of robustness and computation– Anecdotes from engineering and biology

– New theory (especially Parrilo)

– Not enough time today…

Bonus!

• “Unified systems” theory helps resolve fundamental unresolved problems at the foundations of physics

• Ubiquity of power laws (statistical mechanics)• Shear flow turbulence (fluid dynamics)• Macro dissipation and thermodynamics from micro

reversible dynamics (statistical mechanics)• Quantum-classical transition• Quantum measurement• Thus the new mathematics for a unified theory of

systems is directly relevant to multiscale physics• The two challenges (unify and multiscale) are

connected.

Network protocols.

HTTP

TCP

IP

Routers

Files

packetspacketspacketspacketspacketspackets

Web servers

web traffic

Is streamed out on the net.

Creating internet traffic

Webclient

Web/internet traffic

Web servers

web traffic

Is streamed out on the net.

Creating internet traffic

Webclient

Let’s look at some web traffic

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

Decimated dataLog (base 10)

Forest fires1000 km2

(Malamud)

WWW filesMbytes

(Crovella)

Data compression

(Huffman)

Los Alamos fire

Cumulative

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

FrequencyFires

Web filesCodewords

Cumulative

Log (base 10)

-1/2

-1

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

Decimated dataLog (base 10)

Forest fires1000 km2

(Malamud)

WWW filesMbytes

(Crovella)

Data compression

(Huffman)

Los Alamos fire

Cumulative

>1e5 files

>4e3 fires

10-2

10-1

100

100

101

102

20th Century’s 100 largest disasters worldwide

US Power outages (10M of customers)

Natural ($100B)

Technological ($10B)

10-2

10-1

100

100

101

102

Log(Cumulative frequency)

Log(size)

= Log(rank)

0 2 4 6 8 10 12 140

20

40

60

80

100

size

rank

Natural ($100B)

Technological ($10B)

100

101

102

1

2

3

10

100

10-2

10-1

100

Log(size)

Log(rank)

100

101

102

20th Century’s 100 largest disasters worldwide

US Power outages (10M of customers)

Natural ($100B)

Technological ($10B)

Slope = -1(=1)

10-2

10-1

100

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

Decimated dataLog (base 10)

Forest fires1000 km2

WWW filesMbytes

Data compression

Cumulative

-1/2

-1

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency Forest fires1000 km2

WWW filesMbytes

Data compression

Cumulative

-1/2

-1

exponential

100

101

102

103

10-4

10-3

10-2

10-1

100

loglog

.5

1

semilogy

20 40 60 80 100

0.2

0.6

1

linearPlotting power laws

and exponentials

exp

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency Forest fires1000 km2

WWW filesMbytes

Data compression

Cumulative

exponential

All events are close in size.

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency Forest fires1000 km2

WWW filesMbytes

Data compression

Cumulative

-1/2

-1

Most events are small

But the large events are huge

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency Forest fires1000 km2

WWW filesMbytes

Data compression

Cumulative

-1/2

-1

Most events are small

But the large events are huge

Robust

Yet Fragile

Robustness of HOT systems

Robust

Fragile

Robust(to known anddesigned-foruncertainties)

Fragile(to unknown

or rareperturbations)

Uncertainties

Large scale phenomena is extremely non-Gaussian

• The microscopic world is largely exponential

• The laboratory world is largely Gaussian because of the central limit theorem

• The large scale phenomena has heavy tails (fat tails) and power laws

Size of events x vs. frequency

log(size)

)1()( xxpdx

dPlog(probability)

log(Prob > size)

xPlog(rank)

-1 0 1 2 3 4 5

0

-1

-2

-3

-4

log10(x)

log10(P)

x integer

1e3 samples from a known distribution:

x10

10

=1

xP

10( )

10P X x

x

)( xXP

Slope = -

=1

=0

=1

=0Cumulative Distributions

dx

dPxp )(

Slope = -(+1)Noncumulative

Densities

=0

=1

Cumulative Distributions

NoncumulativeDensities

Correct

Wrong

=0

The physics view

• Power laws are “suggestive of criticality”• Self-organized criticality (SOC)• Examples where this holds:

– Phase transitions in lab experiments– Percolation models– Rice pile experiments

• No convincing examples in technology, biology, ecology, geophysical, or socio-economic systems

• Special case of “new science of complexity”• Complexity “emerges” at a phase transition or

bifurcation “between order and disorder.”• Doesn’t work outside the lab.

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

Forest fire

SOC = .15

SOC = .15

= .15Cumulative distributions

=.15Noncumulative densities, logarithmic binning

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

FrequencyFires

Web filesCodewords

Cumulative

Log (base 10)

-1/2

-1

The HOT view of power laws(w/ Jean Carlson, UCSB)

• The central limit theorem gives power laws as well as Gaussians

• Many other mechanisms (eg multiplication noise) yield power laws

• A model producing a power law is per se uninteresting• A model should say much more, and lead to new

experiments and improved designs, policies, therapies, treatments, etc.

The HOT view of power laws

• Engineers design (and evolution selects) for systems with certain typical properties:

• Optimized for average (mean) behavior

• Optimizing the mean often (but not always) yields high variance and heavy tails

• Power laws arise from heavy tails when there is enough aggregate data

• One symptom of “robust, yet fragile”

HOT and fat tails?

• Surprisingly good explanation of statistics (given the severity of the abstraction)

• But statistics are of secondary importance

• Not mere curve fitting, insights lead to new designs

• Understanding design

Examples of HOT fat tails?

• Power outages• Web/Internet file traffic• Forest fires• Commercial aviation delays/cancellations• Disk files, CPU utilization, …• Deaths or dollars lost due to man-made or natural

disasters?• Financial market volatility?• Ecosystem and specie extinction events?• Other mechanisms, examples?

Detailed simulations

Examples with additional mechanisms?

• Word rank (Zipf’s law)

• Income and wealth of individuals and companies

• Citations, papers

• Social and professional networks

• City sizes

• Many others….

• (Simon, Mandelbrot, …)

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

Decimated dataLog (base 10)

WWW filesMbytes

(Crovella)

Cumulative Most files are small

(mice)

Most packets are in large files (elephants)

NetworkNetwork

Sources

Mice

Elephants

Router queues

NetworkNetwork

Sources

Mice

Elephants

Router queues

Delay sensitive

Bandwidth sensitive

Log(bandwidth)

Log(delay)

cheap

Expensive

• We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later.

Delay

BW

BW = Bandwidth sensitive trafficDelay = Delay sensitive traffic

Log(bandwidth)

Log(delay)

Delay

BWBulk transfers (most packets)

Web navigation, voice (most files)

• Mice: many small files of few packets which the user presumably wants ASAP• Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay• Most files are mice but most packets are in elephants…•…which is the manifestation of fat tails in the web and internet.

Log(bandwidth)

Log(delay)

Delay

BWBulk transfers (most packets)

Web navigation, voice (most files)

Claim I: Current traffic dominated by these two types of flows

Claim II: Intrinsic feature of many future network applications

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

Decimated dataLog (base 10)

WWW filesMbytes

(Crovella)

Cumulative Most files are small

(mice)

Most packets are in large files (elephants)

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

Size of events

Frequency

WWW filesMbytes

Data compression

Cumulative

exponential

All events are close in size.

Source coding for data compression

Based on frequencies of source word occurrences,

Select code words.

To minimize message length.

Source coding for data compression

Objectives:• Optimally compress file• Tractable compression• Tractable decompression

Shannon:• Optimally compress ensemble• Tractable compression• Tractable decompression

Kolmogorov:• Optimally compress file• Undecidable compression• Intractable decompression

• Surprise: natural and practical• Stochastic relaxation

• Philosophically important• Turing, Godel, Chaitin, …

Shannon coding

• Ignore value of information, consider only “surprise”• Compress average codeword length (over stochastic

ensembles of source words rather than actual files)• Constraint on codewords of unique decodability• Equivalent to building barriers in a zero dimensional tree• Optimal distribution (exponential) and optimal cost are:

DataCompression

length log( )

exp( )i i

i i

l p

p cl

Avg. length =

log( )

i i

i i

p l

p p

Shannon source coding

1i i iJ p l r Minimize expected

length

source words with probabilities pi

length of codewords li

unique decodability

2 11

log( )

il

i

i i

rl r

Kraft’s inequality

2 11

log( )

il

i

i i

rl r

Kraft’s inequality =Prefix-less code

0

110

11

100

101

110

111

1110

1111

11100

1110111110

11111

010010111011100111011111011111

Codewords

2 1il

2 11

log( )

il

i

i i

rl r

0

110

11

100

101

110

111

1110

1111

11100

1110111110

11111

010010111011100111011111011111

Codewords

2 1il

0 dimensional (discrete) tree

Kraft’s inequality =Prefix-less code

cut in a 0-dim tree

2 1il

Kraft’s inequality =Prefix-less code

Channel noise

Coding = building barriers

Source coding Channel coding

Control = building barriers

( ) log( )l r r

log( )i il p

1i i iJ p l r

Leads to optimal solutions for codeword lengths.

With optimal cost log( )i iJ p p

Minimize

Equivalent to optimal barriers on a discrete tree (zero dimensional).

( ) log( )l r r

log( )i il p

1i i iJ p l r

log( )i iJ p p • Compressed files look like white noise.• Compression improves robustness to limitations in

resources of bandwidth and memory.• Compression makes everything else much more fragile:

– Loss or errors in compressed file– Statistics of source file

• Information theory also addresses these issues at the expense of (much) greater complexity

0 1 2-1

0

1

2

3

4

5

6

DC

Data

Avg. length =

log( )

i i

i i

p l

p p

How well does the model predict the data?

length log(

exp( )

)i i

i i

l p

p cl

0 1 2-1

0

1

2

3

4

5

6

DC

Data + Modellength log(

exp( )

)i i

i i

l p

p cl

Avg. length =

log( )

i i

i i

p l

p p

How well does the model predict the data?

Not surprising, because the file was compressed using

Shannon theory.

Small discrepancy due to integer lengths.

Why is this a good model?

• Lots of models will reproduce an exponential distribution

• Shannon source coding lets us systematically produce optimal and easily decodable compressed files

• Fitting the data is necessary but far from sufficient for a good model

Web layout as generalized “source coding”

• Keep parts of Shannon abstraction:– Minimize downloaded file size– Averaged over an ensemble of user access

• Equivalent to building 0-dimensional barriers in a 1- dimensional tree of content

document

split into N files to minimize download time

A toy website model(= 1-d grid HOT design)

# links = # files

Optimize 0-dimensional cuts in a 1-dimensional document

More complete website models

(Zhu, Yu, Effros)

• Necessary for web layout design• Statistics consistent with simpler models• Improved protocol design (TCP)• Commercial implications still unclear

Generalized “coding” problems

Web

Data compression

• Optimizing d-1 dimensional cuts in d dimensional spaces…

• To minimize average size of files • Models of greatly varying detail all give a consistent

story.• Power laws have 1/d.• Completely unlike criticality.

PLR optimization

RrlpJ iiiMinimize

expected loss

P: uncertain events with probabilities pi

L: with loss li

R: limited resources ri

P L R

DC source codewords decodability

WWW user access files web layout

document

split into N files to minimize download time

1l r r = density of links or filesl = size of files

pi = Probability

of event

drl

li = volume enclosed

ri = barrier density

d-dimensional

i

d

id

i lrl

1

,

Resource/loss relationship:

1)(

r

crl

RrlpJ iii

PLR optimization

d

= 0 data compression = 1 web layout

= “dimension”

RrlpJ iii

PLR optimization

= 0 data compression

=0 is Shannon

source coding

0

0

1

)log(

)(

r

c

r

rl

0

0)log()log(

1

1

1

ii

iiii

pRc

ppRpp

J

p

01

0)log()(

r

cr

rl

1

)1/(1

)1/(1

j

j

ii

p

Rpcl

RrlpJ iii

11

ipcR

Minimize average cost using standard Lagrange multipliers

With optimal cost

Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.

01

0)log()(

r

cr

rl

1

)1/(1

)1/(1

j

j

ii

p

Rpcl

RrlpJ iii

11

ipcR

Minimize average cost using standard Lagrange multipliers

With optimal cost

Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.

011

0)log(

1

1

1

i

ii

p

pp

J

1

)1/(1

)1/(1

j

j

ii

p

Rpcl

To compare with data.

ip

Forward engineering

il ir

Reverse engineering

(1 1/ )

1 2

ˆ i

i

p

c l c

1

)1/(1

)1/(1

j

j

ii

p

Rpcl

ip il ir

Reverse engineering

To compare with data.

(1 1/ )

1 2

ˆ i

i

p

c l c

1

)1/(1

)1/(1

j

j

ii

p

Rpcl

1

ˆ

ˆi

i i ik i

P

p l l

Cumulative

ii Pl ˆ,plot

sizesfromdata

computeusingmodel

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

Typical web traffic

log(file size)

> 1.0log(freq > size)

p s-

Web servers

Heavy tailed web traffic

Is streamed out on the net.

Creating fractal Gaussian internet traffic (Willinger,…)

2

3 H

Fat tail web traffic

Is streamed onto the Internet

creating long-range correlations with 2

3 H

time

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

Are individual websites distributed like this?

Roughly, yes.

-6 -5 -4 -3 -2 -1 0 1 2-1

0

1

2

3

4

5

6

WWWDC

Data + Model/Theory

How has the data changed since 1995?

Steeper. Consistent with more use of cross hyperlinks.

More complete website models

(Zhu, Yu, Effros)

• More complex hyperlinks leads to steeper distributions with 1< < 2

• Optimize file sizes within a fixed topology:• Tree: 1• Random graph: 2

• No analytic solutions

The broader Shannon abstraction

• Information = surprise… and therefore ignoring– Value or timeliness of information

– Topology of information

• Separate source and channel coding– Data compression

– Error-correcting codes (expansion)

• Eliminate time and space– Stochastic relaxation (ensembles)

– Asymptopia

• Brilliantly elegant and applicable, but brittle• Better departure point than Kolmogorov, et al

What can we keep?

• Separation:– Source and channel

– Congestion control and error correction

– Estimation and control

• Tractable relaxations– Stochastic embeddings

– Convex relaxations

• Add to information:– Value

– Time and dynamics

– Topology

– Feedback

• More subtle treatment of computational complexity

• Naïve formulations intractable

What must we change?

Log(bandwidth)

Distortion

achievable

notRate distortion theory

studies tradeoffs between bandwidth and distortion

from lossy coding.

Log(bandwidth)

Log(delay)

cheap

Expensive

• We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later.

Delay

BW

BW = Bandwidth sensitive trafficDelay = Delay sensitive traffic

Log(bandwidth)

Log(delay)

Delay

BWBulk transfers (most packets)

Web navigation, voice (most files)

• Mice: many small files of few packets which the user presumably wants ASAP• Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay• Most files are mice but most packets are in elephants…•…which is the manifestation of fat tails in the web and internet.

Log(bandwidth)

Log(delay)

Delay

BWBulk transfers (most packets)

Web navigation, voice (most files)

Claim I: Current traffic dominated by these two types of flows

Claim II: Intrinsic feature of many future network applications

NetworkNetwork

Sources

Mice

Elephants

Router queues

NetworkNetwork

Sources

Mice

Elephants

Router queues

Delay sensitive

Bandwidth sensitive

Log(bandwidth)

Log(delay)

Delay

BWBulk transfers (most packets)

Web navigation, voice (most files)

Claim (channel): We can tweak TCP using ECN and REM to make these flows co-exist.

Currently: Delays are aggravated by queuing delay and packet drops from congestion caused by BW traffic?

Specifically:• Keep queues empty (ECN/REM). • BW slightly improved (packet loss)• Delay greatly improved (queuing)• Provision network for BW• “Free” QOS for Delay• Network level stays simple

Log(bandwidth)

Log(delay)

Delay

BW

Claim (source): Many (future) applications are natural and intrinsically coded into exactly this kind of fat-tailed traffic.

Expensive

The rare traffic that can’t or won’t will be expensive, and essentially pay for the rest.

Fat tailed traffic is “intrinsic”

• Two types of application traffic are important: communications and control

• Communication to and/or from humans (from web to virtual reality)

• Sensing and/or control of dynamical systems• Claim: both can be naturally “coded” into fat-tailed

BW + delay traffic • This claim needs more research

Log(bandwidth)

Log(delay)

Delay

BW

Expensive

Abstraction I

• Separate source and channel coding• Source is coded into

– Delay sensitive mice

– Bandwidth sensitive elephants

• “Channel coding” = congestion control

Log(bandwidth)

Log(delay)

Delay

BW

Expensive

Putting loss back into the picture

• Packet loss can be handled by coding (application) or retransmission (transport)

• Need coherent theory to perform tradeoffs• Currently, congestion control and reliable transport are

intertwined• What benefits would derive from some decoupling,

enabled by ECN or other explicit congestion control strategies?

Log(BW)

Log(d)

Loss?

Optimization/control framework

• Application specific cost functions J(app,delay,loss,BW) (assume to be minimized)

• Network resources:lines, routers, queues (energy, spectrum, deployment, repair, stealth, security, etc)

• Comm/control network is embedded in other networks (transportation, energy, military action, …)

• Robustness to uncertainties in users and resources• Need to flesh out details for future scenarios

Optimization/control framework• Global optimal allocation sets lower bound on

achievable performance• Control problem is to find decentralized strategies

(eg TCP/IP) with (provably) near optimal performance and robustness in dynamical setting

• Duality theory key to using network • Coding and control interact in unfamiliar ways• Naïve formulations intractable:

– Computation intractable– Requires too much information not available to

decentralized agents

• Key is to find tractable relaxations

Optimization/control framework• Pioneered by Kelly et al and extended by

Low et al.• Ambitious goal: foundation for (much?)

more unified theory of computation, control, and communications

• Hoped for outcome:– Rich theoretical framework– Motivated by practical problems– Yielding principled design of new protocols– And methods for deploying and managing

complex networks

Scalable Congestion Control

LINKS

SOURCES

( )fR s

( )TbR s

ROUTING + DELAY

p : link prices

y : aggregate link flows

x : source rates

q : aggregate prices per source

(Paganini, Doyle & Low ’01)

Robustness, evolvability/scalability, verifiability

Ideal performance

Robustness

Evolvability

Verifiability

Typical design IP

Robustness of HOT systems

Robust

Fragile

Robust(to known anddesigned-foruncertainties)

Fragile(to unknown

or rareperturbations)

Uncertainties

Feedback and robustness

• Negative feedback is both the most powerful and most dangerous mechanism for robustness.

• It is everywhere in engineering, but appears hidden as long as it works.

• Biology seems to use it even more aggressively, but also uses other familiar engineering strategies:– Positive feedback to create switches (digital systems)

– Protocol stacks

– Feedforward control

– Randomized strategies

– Coding

The Internet hourglass

IP

Web FTP Mail News Video Audio ping napster

Applications

TCP SCTP UDP ICMP

Transport protocols

Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM

Link technologies

From Hari Balakrishnan

The Internet hourglass

IP

Web FTP Mail News Video Audio ping napster

Applications

TCP SCTP UDP ICMP

Transport protocols

Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM

Link technologies

From Hari Balakrishnan

Everythingon IP

IP oneverything

Commodities,Hardware

Consumers,Applications

RobustMesoscale

Robust, yet fragile

Hardware

Applications

TCP/IP

UncertaintyUncertainty

UncertaintyUncertaintyCommodities,

Hardware

Consumers,Applications

RobustMesoscale

Robust

Commodities,Hardware

Consumers,Applications

RobustMesoscale

Yet fragile

Difficult to change

Yet fragile

Protocols allow for the creation of large complex networks, with rare but catastrophic cascading failures.

Software

Hardware

Early computing

Analogsubstrate

Variousfunctionality

Digital

Software

Hardware

Hardware

Applications

OperatingSystem

ModernComputing

Robust, yet fragile

Analogelectronics

Variousfunctionality

Digital

Uncertainsubstrate

Variedfunctionality

Robustmesoscale

Commodities

Consumers

Money

Commodities

Consumers

Barter

Commodities

Consumers

Money

Investments

Investors

Markets,Insitutions

The hourglass

Dress Shirt Slacks Lingerie Coat Scarf Tie

Garments

Cloth

Sewing

Wool Cotton NylonRayon Polyester

Material technologies

Producers

Consumers

Energy

Energy

• 110 V, 60 Hz AC• Gasoline• ATP, glucose, etc• Proton motive force

Hardware

Applications

TCP/IP

• Decentralized• Asynchronous

Robust to:• Network topology• Application traffic• Delays, link speeds

High performanceNecessity:

Essentially only onedesign is possible

Hardware

Applications

TCP/IP

• Decentralized• Asynchronous

Robust to:• Network topology• Application traffic• Delays, link speeds

High performanceNecessity:

Essentially only onedesign is possible

The existing designis incredible, but…

It’s a product of evolution,and is not optimal.

1 dimension

All

None

desi

gnControl Theory

Statistical Physics

Dynamical Systems

Information TheoryComputational

Complexity

Theory ofComplex systems?

1 dimension

All

None

desi

gnControl Theory

Statistical Physics

Dynamical Systems

Information TheoryComputational

Complexity

Biology• Non-equilibrium• Highly tuned or optimized• Finite but large dimension

1 dimension

All

None

desi

gnControl Theory

Statistical Physics

Dynamical Systems

Information TheoryComputational

Complexity

Theory needs• Integrated horizontally and vertically• Horizontal: control, communications, computing• Vertical: multiscale physics

• Status: nascent but promise results• Bonus: unexpected synergy

• Ubiquity of power laws• High shear turbulence• Dissipation• Quantum/classical transition• Quantum measurement

1 dimension

All

None

desi

gnControl Theory

Statistical Physics

Dynamical Systems

Information TheoryComputational

Complexity