Embed Size (px)
Citation preview
Co-factors
Fatty acids
Sugars
Nucleotides
Amino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Genes
DNA replication
Trans*
John DoyleJohn G Braun Professor
Control and Dynamical Systems,BioEng, and ElecEng
Caltech
GGGDP GTP
GDPGTP
In
In
Out
P
Ligand
Receptor
RGS
www.cds.caltech.edu/~doyle
MultiscalePhysics
SystemsBiology
Network Centric,Pervasive,Embedded,Ubiquitous
Core theory
challenges
My interests
Collaborators and contributors(partial list)
Biology: Csete,Yi, El-Samad, Khammash, Tanaka, Arkin, Savageau, Simon, AfCS, Kurata, Smolke, Gross, Kitano, Hucka, Sauro, Finney, Bolouri, Gillespie, Petzold, F Doyle, Stelling, Caporale,…
Theory: Parrilo, Carlson, Murray, Vinnicombe, Paganini, Mitra Papachristodoulou, Prajna, Goncalves, Fazel, Liu, Lall, D’Andrea, Jadbabaie, Dahleh, Martins, Recht, many more current and former students, …
Web/Internet: Li, Alderson, Chen, Low, Willinger, Kelly, Zhu,Yu, Wang, Chandy, …
Turbulence: Bamieh, Bobba, McKeown, Gharib, Marsden, …Physics: Sandberg, Mabuchi, Doherty, Barahona, Reynolds,Disturbance ecology: Moritz, Carlson,…Finance: Martinez, Primbs, Yamada, Giannelli,…
Current Caltech Former Caltech OtherLongterm Visitor
Thanks to
• NSF• ARO/ICB• AFOSR• NIH/NIGMS• Boeing• DARPA• Lee Center for Advanced Networking (Caltech)• Hiroaki Kitano (ERATO)• Braun family
Background progress
• Spectacular progress, both depth and breadth– Biological networks– Technological networks– Theoretical foundations
• Remarkably consistent, convergent, coherent• Role of protocols, architecture, feedback, and
dynamics
• Yet seemingly persistent errors and confusion both within science between science and public & policy
Warning
• My view of complexity is not just different, but largely opposite from SK…
• I like the “poetry” but not the science.• Surprisingly, we largely agree on the details of
biology (how the devices work)– DNA, RNA, proteins– Transcriptional regulation, signal transduction– Cancer stem cells– Etc, etc,…
• We just don’t agree on how it is organized• Unfortunately, my story is more complicated
because it involves mathematics and technology, and very new discoveries in cell physiology (sorry)
Tradeoff
• I could use the hour to explain why these ideas don’t apply to biology:– Edge of chaos
– Self-organized criticality
– Scale-free networks
– Etc etc etc
• I’ll do a little of that, but see me off-line if you want to hear more. This isn’t really controversial among experts, but appears to be more widely.
• Mostly I’ll try to explain what ideas do apply to biology (and technology).
Why cells are not critical (simple version)
• When you check the connectivity of bio (and techno) networks
• They are very highly connected, deeply into the chaotic regime
• Yet the are not chaotic?!?!?
• The are also not randomly interconnected… but are very far from random. It is a mistake to ignore evolution (and function) in biology.
Why cells are not critical (slightly more complicated version)
• The edge of chaos (EOC) or criticality (SOC) is unaffected by random rewiring (provided it preserves the order parameter)
• Bio and tech networks are ruined by random rewiring (and thus are not EOC or SOC)
• But are often robust to large scale (protocol obeying) rearrangements
Background progress: Biological networks
(With molecular biology details of components)
+ systems biology Organizational principles are increasingly apparent
Beginning to see principles of architecture (as well as components and circuits)
• Unfortunately, also pervasive, substantial errors and resulting confusion in the analysis of “complexity”: (e.g. ‘irreducible complexity and intelligent design’, ‘scale-free networks’, ‘self-organized criticality’, ‘edge-of-chaos,’…)
• These errors should be straightforward to correct but have so far been remarkably persistent.
Background progress: Technological networks• Complexity of advanced technology biology • Components extremely different• Yet, striking convergence at network level:
– Architecture– Layering and protocols– Feedback control
• The same analysis errors (e.g. regarding router topology) here have largely been eliminated (within the Internet, if not the larger scientific, community)
Background progress: Mathematics
New mathematical frameworks suggests • apparent network-level evolutionary convergence• within/between biology/technology is not accidental
But follows necessarily from universal requirements: – efficient, – adaptive, – evolvable, – robust (to both environment and component )
Background progress: Mathematics
New mathematical frameworks suggests • apparent network-level evolutionary convergence• within/between biology/technology is not accidentalBut follows necessarily from universal requirements:
efficient, adaptive, evolvable, robust
For example (which we won’t talk about much today):• New theories of Internet and related networking
technologies confirm engineering intuition (Kelly, Low, many others…)
• Also lead to test and deployment of new protocols for high performance networking (e.g. FAST TCP)
Background progress: Mathematics
• Blends (from engineering) theories from
– optimization,
– control,
– information, and
– computational complexity
• with diverse elements in areas of mathematics (e.g. operator theory and algebraic geometry) not traditionally thought of as applied
Key concepts
• Robust yet fragile
• The meaning of “complexity”
• Architecture
• Hard limits
• Small models
• Short proofs
Robust Yet Fragile
Human complexity
Efficient, flexible metabolism Complex development and Immune systems Regeneration & renewal Complex societies Advanced technologies
Obesity and diabetes Rich parasite ecosystem Inflammation, Auto-Im. Cancer Epidemics, war, … Catastrophic failures
• Evolved mechanisms for robustness allow for, even facilitate, novel, severe fragilities elsewhere
• often involving hijacking/exploiting the same mechanism
Modern cars, planes, computers, networks, etc • have exploding internal complexity• are simpler to use and more robust.• tend to work perfectly or not at all.
Modern cars, planes, computers, networks, etc • have exploding internal complexity• are simpler to use and more robust.• tend to work perfectly or not at all.
RobustRobust
FragileYet
Nightmare? Biology: We might accumulate more complete
parts lists but never “understand” how it all works.
Technology: We might build increasingly complex and incomprehensible systems which will eventually fail completely yet cryptically.
Nothing in the orthodox views of complexity says this won’t happen (apparently).
Confusion about basics
Questions and issues:
• The essence of “complexity”?
• The foundational challenges and issues?
• The sources of so much (unnecessary) confusion?
• Existing success stories?
• Differences from the failures?
• Encouraging new research?
Defining “complexity”
• Aim: simple but universal taxonomy
• Widely divergent starting points from math, biology,
technology, physics, etc,
• Can be organized into a coherent and consistent picture
• Even very different notions of complexity
– Organized complexity
– Emergent complexity
– Irreducible complexity
The essence of simplicity
Simple questions:• Elegant experiments,
flexible platforms• Small, simple models
– explain data – testable predictions– computable
• Elegant unifying theorems
Simple answers:• Simple outcomes, data;
robust, repeatable• Robust, predictable
– explain data – testable predictions– both computable
• Short proofs
The essence of simplicity
Simple questions:• Elegant experiments,
flexible platforms• Small, simple models
– explain data – testable predictions– computable
• Elegant unifying theorems
Simple answers:• Simple outcomes, data;
robust, repeatable• Robust, predictable
– explain data – testable predictions– both computable
• Short proofs
The essence of simplicity
Simple questions:• Elegant experiments• Small models• Elegant theorems
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
Integrated infrastructure: experimental, software, computational, theory
To iterate between these elements
A minimal notion of simplicity
Simple questions:• Elegant experiments• Small models• Elegant theorems
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
A minimal notion of simplicity
Simple questions:• Small models
Simple answers:• Robust, predictable
• Reductionist science: Reduce the apparent complexity of the world directly to an underlying simplicity.
• Physics has always epitomized this approach• Molecular biology has successfully mimicked physics• Bio and tech networks need more• How to describe the “space” of complexity?
Two dimensions of complexity
Small
models
Large
models
Robust Simplicity Organization
FragileEmergence
Irreducibility
1. Small vs large descriptions, models, theorems
2. Robust vs fragile features in response to perturbations in descriptions, components, or the environment.
Smallmodels
Largemodels
Robust Simplicity Organization
Fragile Emergence
Irreducibility
Where we’re going
• Aim: simple but universal taxonomy • Widely divergent starting points from math, biology,
technology, physics, etc, • Can be organized into a coherent and consistent picture• Even very different notions of complexity
– Organized complexity– Emergent complexity– Irreducible complexity
Small
models
Large
models
RobustSimplicity
Fragile
Simple questions:• Elegant experiments• Small models• Elegant theorems
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
Small
models
Large
models
Robust Simplicity
FragileEmergence
Small LargeRobust Simplicity
Fragile Emergence
Simple questions:• Elegant experiments• Small models• Elegant theorems
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
• Godel: Incompleteness, Turing: Undecidability• Even simple questions can be “complex” and fragile• Profoundly effected mathematics and computation• Modest impact on science, primarily through emphasis on “emergent complexity”
1 0
1bounded 1 , =
2k k kc z cz z z
“Emergent” complexity
• Simple question• Undecidable
• No short proof• Chaos• Fractals
Mandelbrot
• The most fragile details of complex systems will likely always be experimentally and computationally intractable.
• Fortunately, we care more about understanding, avoiding, and managing fragility (than about all its details)
Small
models
Large
models
Robust Simplicity Organization
FragileEmergence
Organized complexity
Organized complexity
Small LargeRobust/Short Simplicity Organization
Fragile/Long EmergenceSimple questions:
• Elegant experiments• Small models• Elegant theorems
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
The triumph (and horror) of organization• Complex, uncertain, hostile environments• Unreliable, uncertain, changing components• Limited testing and experimentationYet predictable, robust, adaptable, evolvable systems
Simple questions:• Elegant experiments• Small models• Elegant theorems
Small LargeRobust/Short Simplicity Organization
Fragile/Long Emergence
The triumph (and horror) of organization• Complex, uncertain, hostile environments• Unreliable, uncertain, changing components• Limited testing and experimentationYet predictable, robust, adaptable, evolvable systems
• Requires highly organized interactions, by design or evolution
• Completely different theory and technology from emergence
Simple answers:• Simple outcomes• Robust, predictable• Short proofs
Cruise control
Electronic ignition
Temperature control
Electronic fuel injection
Anti-lock brakes
Electronic transmission
Electric power steering (PAS)
Air bags
Active suspension
EGR control
Organized
Cruise control
Electronic ignition
Temperature control
Electronic fuel injection
Anti-lock brakes
Electronic transmission
Electric power steering (PAS)
Air bags
Active suspension
EGR control
Emergent
Organized
Organized complexity
• Neither reductionist science, nor (more significantly) “emergent” complexity theory is much help with organized complexity (and often just leads to further obfuscation).
• This no longer should be a controversial statement, given recent history, but it still is
• Organized complexity has a long history, but has exploded in importance in the last few decades with network technology and biology
• What are the keys to understanding organized complexity?
Math, bio, & tech
• Small and Large apply to the description of experiments, theorems, models, systems• Bio and tech systems have enormously long and complex descriptions, yet extraordinarily robust behaviors• Indeed, robustness drives their complexity, and more fragile systems could be much simpler• Much of the apparent complexity of modern mathematics is to create a robust and rigorous proof infrastructure
Small Large
Robust Simplicity Organization
FragileEmergence
Robust
yet fragile
Small Large
Robust Simplicity
FragileEmergence
OrganizationOrganization
• Even systems most designed/evolved for extreme robustness, also have fragilities and this is not accidental
• The most designed/evolved systems have systematic, universal spirals of robustness/complexity/fragility
• Understanding/controlling this “spiral” is arguably the central challenge in organized complexity
• There are hard constraints on robustness/fragility • Thus robustness is never “free” and is paid for with
fragility somewhere
Issues
• Emergence and Organization are opposites, but can be viewed in this unified framework
• Emergence celebrates fragility• Organization seeks to manage robustness/fragility
• Much confusion is caused by failure to “get this”
• The most fundamental challenge of organizing complexity for robust and evolvable systems is inherent and unavoidable robustness/fragility tradeoffs
Small Large
Robust Simplicity Organization
FragileEmergence
Robust Yet Fragile
Human complexity
Efficient, flexible metabolism Complex development and Immune systems Regeneration & renewal Complex societies Advanced technologies
Obesity and diabetes Rich parasite ecosystem Inflammation, Auto-Im. Cancer Epidemics, war, … Catastrophic failures
• Evolved mechanisms for robustness allow for, even facilitate, novel, severe fragilities elsewhere (often involving hijacking/exploiting the same mechanism)
• Universal challenge: Understand/manage/overcome this robustness/complexity/fragility spiral
Key concepts
• Robust yet fragile• Architecture
• Hard limits• Small models• Short proofs
Motivate with familiar examples
Drill down withsome math details
Helpful background?
• Robust yet fragile• Architecture
• Hard limits• Small models• Short proofs
Basic• Biochem (prok.)• Networks (TCP/IP)
• Information theory• Control theory• CS cmplxt th. (P/NP/coNP)• SOS/SDP
Robustness = Invariance of[a system] and[a property] to [a set of perturbations]
RobustRobust
FragileYet
[a system] can have[a property] robust for [a set of perturbations]
Yet be fragile for
Or [a different perturbation][a different property] different perturbation
different perturbations
[a system] can have[a property] robust for [a set of perturbations]
Yet be fragile for
[a different property]
Robust
Fragile
different perturbations
Robust
FragileDiseases of complexity
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex development
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune response
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune response
Autoimmune disease
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune responseRegeneration/renewal
Autoimmune disease
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune responseRegeneration/renewal
Autoimmune diseaseCancer
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune responseRegeneration/renewalComplex society
Autoimmune diseaseCancer
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune responseRegeneration/renewalComplex society
Autoimmune diseaseCancer
Epidemics
different perturbations
Robust
FragileDiseases of complexity
ParasitesComplex developmentImmune responseRegeneration/renewalComplex society
Autoimmune diseaseCancer
Epidemics
Robust
Yetfragile
different perturbations
Robust
FragileComplexity?
Robust
Yetfragile
Greater complexity can provide improved robustness.
But there are unavoidable
tradeoffs.
different perturbations
FragileRobust
ComplexFragile
Robust
Complex
FragileRobust
Complex
Universal challenge: Understand/manage/overcome
this spiral
Robust
Fragile
Uncertainty
Hard limits?
Robust
Yetfragile
new conservation laws of
robustness/fragility.
If exploited, net benefits are possible.
If not, disasters
loom.
Hard limits and tradeoffs
On systems and their components• Thermodynamics (Carnot)• Communications (Shannon)• Control (Bode) • Computation (Turing/Gödel)
• Fragmented and incompatible• Cannot be used as a basis for
comparing architectures• New unifications are encouraging
Hard limits and tradeoffs
On systems and their components• Thermodynamics (Carnot)• Communications (Shannon)• Control (Bode) • Computation (Turing/Gödel)
• Fragmented and incompatible• Cannot be used as a basis for
comparing architectures• New unifications are encouraging
• Information theory• Control theory• CS cmplxt th. • SOS/SDP
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel SC
u
CC
log S d
log S d
SC
CClog( )a
http://www.glue.umd.edu/~nmartins/
Nuno C Martins and Munther A Dahleh, Feedback Control in the Presence of Noisy Channels: “Bode-Like” Fundamental Limitations of Performance.Nuno C. Martins, Munther A. Dahleh and John C. Doyle Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information(Both in IEEE Transactions on Automatic Control)
Variety of producers
Electric powernetwork
Variety ofconsumers
• Good designs transform/manipulate energy• Subject to hard limits
• Good designs transform/manipulate robustness• Subject to hard limits• Unifies theorems of Shannon and Bode (1940s)• Claim: This is the most crucial (known) limit
against which network complexity must cope
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel
log S d
log S d
benefits
feedback
SC
CCstabilizeremotesensing
remote control
log( )a
costs
Robust
log( )a
Fragile
-e=d-u
Control
uPlant
d
delay
u
Bode
ES
D
log log logS E D
log net entropy reductionE DS d H H log log logS d E d D d
-e=d-u
Control
uPlant
d
delay
u
log S d
Bode
ES
D
log S d
benefits costs
log S
log S d
a
-e=d-u
Control
uPlant
d
delay
u
log S d
Bode
a
ES
D
log S d
benefits costs stabilize
log S d a
a
-e=d-u
Control
uPlant
d
delay
u
log S d
Bode
log S d
benefits costs stabilize
Negative is good
a
ES
D
log S d a
log( )alog S d
log S d
benefits costs
Robust
log( )a Yet fragile
Bode’s integral formula
log( )alog S d
log( )a
log S d
benefits costs
Disturbance-e=d-u
Control
u
Plant
d delayd
delay
u
Cost of control
Cost of stabilization
-e=d-u
Control
Plant ControlChannel
u
CC Cost of remote control
log( )alog S d
log S d
benefits costs
log( )alog S d
CC
Disturbance-e=d-u
Control
Plant
dd
ControlChannel
u
CC
log S d
log S d
benefits
feedback
CCstabilize remote control
log( )a
costs
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel
SC
u
CC
log S d
log S d
benefits
feedback
SC
CCstabilize
remotesensing
remote control
log( )a
costs
Benefit of remote sensing
log( )alog S d
log S d
benefits costs
C
log( )alog S d
CC
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel
SC
u
CC
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel
SC
u
CC
log S d
log S d
benefits
feedback
SC
CCstabilize
remotesensing
remote control
log( )a
costs
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel SC
u
CC
Bode/Shannon is likely a better p-to-p comms theory to serve as a foundation for networks than either Bode or Shannon alone.
log S d
log S d
SC
CClog( )a
Variety of producers
Electric powernetwork
Variety ofconsumers
• Good designs transform/manipulate energy• Subject to hard limits
• Good designs transform/manipulate robustness• Subject to hard limits• Unifies theorems of Shannon and Bode (1940s)• Claim: This is the most crucial (known) limit
against which network complexity must cope
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dd
r
ControlChannel
log S d
log S d
benefits
feedback
SC
CCstabilizeremotesensing
remote control
log( )a
costs
Robust
log( )a
Fragile
[a system] can have[a property] robust for [a set of perturbations]
Yet be fragile for
Or [a different perturbation]
[a different property]Robust
Fragile
[a system] can have[a property] robust for [a set of perturbations]
Robust
Fragile
• But if robustness/fragility are conserved, what does it mean for a system to be robust or fragile?
• Some fragilities are inevitable in robust complex systems.
• But if robustness/fragility are conserved, what does it mean for a system to be robust or fragile?
Small Large
Robust Simple Organized
FragileEmergent
Robust
Fragile
• Robust systems systematically manage this tradeoff.• Fragile systems waste robustness.
• Some fragilities are inevitable in robust complex systems.
Emergent
Variety of producers
Electric powernetwork
Variety ofconsumers
• Good designs transform/manipulate energy• Subject (and close) to hard limits
• Robust designs transform/manipulate robustness• Subject (and close) to hard limits• Fragile designs are far away from hard limits and
waste robustness.
Disturbance-e=d-u
ControlSensor
ChannelEncode
PlantRemoteSensor
dControlChannel
log S d
log S d
SC
CClog( )a
Robust
log( )a
FragileControl
ControlChannel
Example: Hurricane Katrina?
• Robust? Predictable? – Rhetoric
– Coast Guard response
• Fragile? Unpredictable? – Everything else
Hard limits and tradeoffs
On systems and their components• Thermodynamics (Carnot)• Communications (Shannon)• Control (Bode) • Computation (Turing/Gödel)
• Fragmented and incompatible• Cannot be used as a basis for
comparing architectures• New unifications are encouraging
Robust/fragile
is unifyingconcept
Nightmare? Biology: We might accumulate more complete
parts lists but never “understand” how it all works.
Technology: We might build increasingly complex and incomprehensible systems which will eventually fail completely yet cryptically.
Nothing in the orthodox views of complexity says this won’t happen (apparently).
HOPE? Interesting complex systems are robust yet fragile.
Focus robustness on environment/component uncertainty.
Identify the fragility, evaluate and protect it.
Nothing in the orthodox views of complexity says this can’t happen (apparently).
The full
picture
• Some fragilities are inevitable in robust complex systems.• But are there circumstances in which large descriptions, long proofs, and high fragility are desirable?
Small Large
Robust Simplicity Organization
FragileEmergence
Irreducibility?
• Yes, all are important features of cryptography and security, including host-pathogen interactions.
Small Large
RobustSimplicity
Organization
FragileEmergence
Irreducibility• Nightmare: that technology, biology, and medicine (and
social sciences) get stuck with only spiraling complexity, large models/descriptions, no coherent understanding, and uncontrollable fragilities.• Hope: that more rigorous methods can provide systematic tools for managing complexity and robustness/fragility.• There is both tremendous progress and substantial confusion, and robustness/fragility is at the heart of both.
Recap
Nightmare
Small Large
Robust Simplicity Organization
FragileEmergence
Irreducibility
Errors and
confusion
Organization & the arrogance of technology• Underestimating the impact and consequences of
fragilities created by complexity • Failure to manage complexity/robustness/fragility spiral• Classic (high-impact) consequences in global warming,
antibiotic resistance, software failures, spam and viruses, terrorism (see also “war against”), etc…
Errors and
confusion
Organization and the source of complexity• Complexity in bio and tech networks is driven by control
systems (where/when/how), not part count (who/what).• Bio: Protein-coding gene count weakly correlated with
organized complexity of organism (Mattick, Smolke, …)• Tech: Explosion in complexity in computer networks,
autos, planes, devices, supply chains, package delivery, etc is in where/when/how more than who/what.
Small Large
Robust Simplicity Organization
FragileEmergence
Irreducibility
Small Large
Robust Simplicity Organization
FragileEmergence
Irreducibility
Errors and
confusion
Irreducible complexity, intelligent design, and creationism
• Overly mystical (like Emergence) and underestimates the organized complexity of evolved organisms
• If biology were irreducibly complex, it would– require a (rather incompetent) creator, since it would be too
fragile to evolve– also be so fragile as to require constant intervention of
supernatural control mechanisms
Irreducibility and “intelligent design”
Rube Goldberg
The essential ID argument
If biology is like this,then it could not have evolved.
• This is actually true, and in fact…• If biology is like this,• then it would be too fragile to persist,• and would need the constant intervention of supernatural forces
The flaw
• It is too fragile to actually build.• Neither biology nor (most of) technology is anything like this.• Who said otherwise? Lots of real scientists!• Oops!
This is a cartoon.
Small Large
Robust Simple Organized
FragileEmergent
Irreducible
EmergentIrreducible
Organized
Too fragile to evolve or work in
the real world.
• Emergent and Irreducible have much in common.• Popular among nonexperts, politically powerful.• Both within science (Emergent ) and between science and society (Irreducible ). Oops!!!
1960s-Present: “Emergent complexity”
Simple questions:• Elegant experiments
• Small models
• Elegant theorems
Complexity:• “New sciences”• Unpredictability• Chaos, fractals• Critical phase transitions• Self-similarity• Universality• Pattern formation • Edge-of-chaos• Order for free• Self-organized criticality• Scale-free networks
Dominates scientific thinking today
Unfortunately, has become a source of systematic errors.
Small
Robust Simplicity
FragileEmerge
nce
Small Large
Robust Simplicity Organization
FragileEmergence
Irreducibility
Errors and
confusion
Emergence & “New Sciences” of Complexity, Nets, etc…
• Confusion about origin and nature of chaos, power laws, phase transitions, fractals, simulation, etc
• Lack of minimal mathematical and statistical rigor• Misapplication to the organized complexity of biology,
ecology, technology (and social systems?)• Classic (high-impact) errors in ecosystems, wildfires,
Internet topology, bio networks, physiology, etc• Fortunately, minimal impact in medicine/technology
• Emergence and Organization both involve high variability, but with opposite mechanisms
– Emergence as the result of bifurcations to chaos, and phase transitions and criticality– Organization as the result of high performance robust control systems
• Additional confusion is caused by failure to “get this” distinction
Small Large
RobustSimplicity
Organized
FragileEmergent
High variability?
• Emergence and Organization both involve high variability, but with opposite mechanisms
– Emergence as the result of bifurcations to chaos, and phase transitions and criticality– Organization as the result of high performance robust control systems
• Additional confusion is caused by failure to “get this” distinction
Small Large
RobustSimplicity
Organized
FragileEmergent
Key concepts
• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs
This is a parsimonious and coherent ontology for organized complexity.
There are many “success stories” but they are still fragmented.
Key concepts
• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs
Robust
Fragile
What creates extremes of robustness
(and thus fragility)?
Key concepts
• Robust yet fragile
• Architecture
• Hard limits
• Small models
• Short proofs
What creates extremes of robustness
(and thus fragility)?
“Architecture” is a central challenge
• “The bacterial cell and the Internet have – architectures– that are robust and evolvable (yet fragile?)”
• What does “architecture” mean here?
• What does it mean for an “architecture” to be robust and evolvable?
• Robust yet fragile?
“Architecture” in organized complexity
Architecture involves or facilitates– System-level function (beyond components)– Organization and structure– Protocols and modules– Design or evolution – Robustness, evolvability, scalability– Various -ilities (many of them)– Perhaps aesthetics
but is more than the sum of these
Hard limits and tradeoffs
On systems and their components• Thermodynamics (Carnot)• Communications (Shannon)• Control (Bode) • Computation (Turing/Gödel)
• Fragmented and incompatible• Cannot be used as a basis for
comparing architectures• New unifications are encouraging
Assume different architectures a priori.
Defining “Architecture”
• The elements of structure and organization that are most universal, high-level, persistent
• Must facilitate system level functionality• And robustness/evolvability to uncertainty and
change in components, function, and environment• Architectures can be designed or evolve, but when
possible should be planned• Usually involves specification of
– protocols (rules of interaction) – more than modules (which obey protocols)
“Architecture” in organized complexity
• Design of architectures is replacing design of systems
• Architecture is central in biology and technology, but has been largely overlooked in other areas of “complexity”
• “Emergent” complexity can have “order,” “structure” and (ill-defined notions like) “self-organization” …
• … but “architecture” plays little role…
• Architecture also has little to do with aspects of networks that can be modeled using graph theory (or power laws)
A few asides• Robust yet fragile, architecture-based, bio and techno
networks have high variability everywhere• High variability is fundamental and important.• High variability yields power laws because of their
strong invariance: Central Limit Theorem, marginalization, maximization, mixtures
• Power laws are “more normal than Normal”• This strong statistical invariance also yields power laws
due to analysis errors, which are amazingly widespread• Architecture also has little to do with aspects of networks
that can be modeled using graph theory (or power laws)
“Architecture” examples
• There are universal architectures that are ubiquitous in complex technological and biological networks
• Examples include – Bowties for flows of materials, energy, redox,
information, etc (stoichiometry)– Hourglasses for layering and distribution of
regulation and control (fluxes, kinetics, dynamics)• Nascent theory confirms (reverse engineers) success
stories but has (so far) limited forward engineering applications (e.g. FAST TCP/AQM)
large fan-in of diverse
inputs
large fan-out of diverse
outputs
with thin knot of
universal carriers
Bowties
Examples of
universal carriers
• Packets in the Internet• 60 Hz AC in the power grid• Lego snap• Money in markets and economics• Lots of biology examples (coming up)
The Internet hourglass
Web FTP Mail News Video Audio ping napster
Applications
Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM
Link technologies
The Internet hourglass
IP
Web FTP Mail News Video Audio ping napster
Applications
TCP
Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM
Link technologies
The Internet hourglass
IP
Web FTP Mail News Video Audio ping napster
Applications
TCP
Ethernet 802.11 SatelliteOpticalPower lines BluetoothATM
Link technologies
IP under everything
IP oneverything
IP
TCP/AQM
Applications
Link
Top of “waist” provides robustness to variety and
uncertainty above
Bottom of “waist” providesrobustness to variety
and uncertainty below
Diversefunction
Diversecomponents
UniversalControl
Hourglasses• Hourglasses organize layered control architectures• Examples include:
– Internet: TCP/IP – Cell:
Transcription/translation/degradation plus allosteric regulation
– Lego: Physical assembly plus Mindstorm NXT controller
fan-in of diverse
inputs
fan-out of diverse
outputs
universal carriers
Diversefunction
Diversecomponents
UniversalControl
Bowties andHourglasses
• More and clearer examples of bowties than hourglasses
• Other hourglasses exist but are harder to explain
Four big bowties in bacterial S• Metabolism, biosynthesis, assembly
1. Carriers: Charging carriers in central metabolism
2. Precursors: Biosynthesis of precursors and building blocks
3. Trans*: DNA replication, transcription, and translation
• Signal transduction
4. 2CST: Two-component signal transduction
(Named after their central “knot”)
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Proteins
Pre
curs
ors
DNA replication
Trans*
Cat
abol
ism
Carriers
Tra
nsm
itte
r
Rec
eive
rVariety ofLigands &Receptors
Variety ofresponses
1. Carriers
2. Precursors
3. Trans*
4. 2CST
Pre
curs
ors
Trans*
Carriers
Tra
nsm
itte
r
Rec
eive
r
1. Carriers
2. Precursors
3. Trans*
4. 2CST
“Modules” are less important than protocols:
the “rules” by which modules interact.
Cat
abol
ism
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids Proteins
Pre
curs
ors
DNA replication
Trans*
Carriers
1. Carriers
2. Precursors
3. Trans*
These bowties involve the flow of material and energy.
( )
Mass &Reaction
Mass&Energy Energyflux
Balance
dxSv x
dt
d
dt
Stoichiometry plus regulation
Matrix of integers “Simple,” can be
known exactly Amenable to high
throughput assays and manipulation
Bowtie architecture
Vector of (complex?) functions Difficult to determine and
manipulate Effected by stochastics and
spatial/mechanical structure Hourglass architecture Can be modeled by optimal
controller (?!?)
One big bowtie in Internet S
• All sender files transported as packets • All files are reconstructed from packets by receiver• All advanced technologies have protocols specifying
“knot” of carriers, building blocks, interfaces, etc• This architecture facilitates control, enabling robustness
and evolvability• It also creates fragilities to hijacking and cascading
failure
Variety offiles
Variety of
filespackets
Why bowties?• Metabolism, biosynthesis, assembly
1. Carriers: Charging carriers in central metabolism
2. Precursors: Biosynthesis of precursors and building blocks
3. Trans*: DNA replication, transcription, and translation
• Signal transduction4. 2CST: Two-component signal transduction
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Polymerization and complex
assemblyP
roteinsP
recu
rsor
s
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Core metabolism
DNA replication
Regulation & control
Trans*
Cat
abol
ism
Carriers
Metabolism, biosynthesis, assembly
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Polymerization and complex
assemblyP
roteinsP
recu
rsor
s
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Core metabolism
DNA replication
Trans*
Cat
abol
ism
Carriers
Metabolism, biosynthesis, assembly
The architecture of stoichiometry.
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Proteins
Pre
curs
ors
Autocatalytic feedback
Nut
rien
ts
Core metabolism
DNA replication
Trans*
Cat
abol
ism
Carriers
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Proteins
Pre
curs
ors
Autocatalytic feedback
Nut
rien
ts
Core metabolism
DNA replication
Trans*
Cat
abol
ism
Carriers
EnvironmentEnvironment EnvironmentEnvironment
Huge Variety
Huge Variety
Bacterial cell
Co-factors
Fatty acids
Sugars
NucleotidesAmino Acids
Catabolism
Pre
curs
ors
Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Huge Variety
Same 12
in all cells
Same 8
in all cells
100
same
in all
organism
s
Co-factors
Fatty acids
Sugars
NucleotidesAmino Acids
Catabolism
Polymerization and complex
assemblyProteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Regulation & control
Carriers
Core metabolism
Genes
DNA replicationRegulation & control
Trans*
The architecture of the cell.
Co-factors
Fatty acids
Sugars
NucleotidesAmino Acids
Catabolism
Pre
curs
ors
Carriers
Core metabolism
Nested bowties
Catabolism
Pre
curs
ors
Carriers
Co-factors
Fatty acids
Sugars
NucleotidesAmino Acids
Catabolism
Pre
curs
ors
Carriers
Catabolism
TCAPyr
Oxa
Cit
ACA
Gly
G1P
G6P
F6P
F1-6BP
PEP
Gly3p
13BPG
3PG
2PG
ATP
NADH
TCAPyr
Oxa
Cit
ACA
Gly
G1P
G6P
F6P
F1-6BP
PEP
Gly3p
13BPG
3PG
2PG
Pre
curs
ors
TCAPyr
Oxa
Cit
ACA
Gly
G1P
G6P
F6P
F1-6BP
PEP
Gly3p
13BPG
3PG
2PG
ATP
Autocatalytic
NADH
Pre
curs
ors
Carriers
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
ATP
NADH
Oxa
Cit
ACA
Regulatory
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
ATP
NADH
Oxa
Cit
ACA
( )
Mass &Reaction
Energyflux
Balance
dxSv x
dt
Stoichiometry matrix
S
Regulation of enzyme levels by transcription/translation/degradation
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
Oxa
Cit
ACA
( )
Mass &Reaction
Energyflux
Balance
dxSv x
dt
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
ATP
NADH
Oxa
Cit
ACA
( )
Mass &Reaction
Energyflux
Balance
dxSv x
dt
Allosteric regulation of enzymes
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
ATP
NADH
Oxa
Cit
ACA
Mass &Reaction
( ) Energyflux
Balance
dxSv x
dt
Allosteric regulation of enzymes
Regulation of enzyme levels
TCA
Gly
G1P
G6P
F6P
F1-6BP
PEP Pyr
Gly3p
13BPG
3PG
2PG
ATP
NADH
Oxa
Cit
ACA
Allosteric regulation of enzymes
Regulation of enzyme levels
Fast
Slow
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Polymerization and complex
assemblyP
roteinsP
recu
rsor
s
Autocatalytic feedback
Taxis and transport
Nut
rien
ts
Core metabolism
DNA replication
Trans*
Cat
abol
ism
Carriers
100 104 to ∞in one
organisms
Huge Variety
12
8
Polymerization and complex
assembly
Autocatalytic feedback
Genes
ProteinsDNA
replication
Trans*
104 to ∞in one
organismsHuge
VarietyFew polym
erases
Highly conserved
Why bowties?• Metabolism, biosynthesis, assembly
1. Carriers: Charging carriers in central metabolism
2. Precursors: Biosynthesis of precursors and building blocks
3. Trans*: DNA replication, transcription, and translation
• Signal transduction4. 2CST: Two-component signal transduction
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
50 such “two component” systems in E. Coli• All use the same protocol
- Histidine autokinase transmitter- Aspartyl phospho-acceptor receiver
• Huge variety of receptors and responses• Also multistage (phosphorelay) versions
Signal transduction
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Huge variety
Two components
Highly conserved
Huge variety
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Signal transduction
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Variety ofLigands &Receptors
Variety ofresponses
Tra
nsm
itte
r
Rec
eive
r
Ligands &Receptors
Responses
Tra
nsm
itte
r
Rec
eive
r
Ligands &Receptors
Responses
Tra
nsm
itte
r
Rec
eive
r
Ligands &Receptors
Responses
Shared protocols
Flow of “signal”
Recognition,specificity
InternetsitesUsers
Tra
nsm
itte
r
Rec
eive
r
Ligands &Receptors
Responses
Shared protocols
Flow of “signal”
Recognition,specificity
Note: The wireless system in this room and the Internet to which it is connected work the same way.
Lap
top
Rou
ter
Flow of packets
Recognition,specificity
Huge Variety
Novariety
Hugevariety
• Virtually unlimited variability and heterogeneity (within and between organisms) • E.g: Time constants, rates, molecular counts and sizes, fluxes, variety of molecules, …• Very limited but critical points of homogeneity (within and between organisms)
Nested bowties & advanced technologies:
Everything is made this way: cars, planes, buildings, laptops,…
Variety of producers
Electric power
Variety ofconsumers
Variety ofconsumers
Variety of producers
Energy carriers
• 110 V, 60 Hz AC• (230V, 50 Hz AC)• Gasoline• ATP, glucose, etc• Proton motive force
Standard interface
Nested bowties & advanced technologies:
Everything is made this way: cars, planes, buildings, laptops,…
Huge Variety
Novariety
Hugevariety
• Provides “plug and play modularity” between huge variety of input and output components• Facilitates robust regulation and evolvability on every timescale (“constraints that deconstrain”)• But has extreme fragilities to parasitism and predation (knots are easily hijacked or consumed)
Why?
Evolving evolvability?
“Random”Variation
“Random”Variation
StructuredSelection
?
Evolving evolvability?
StructuredVariation
StructuredVariation
StructuredSelection
“facilitated”“structured”“organized”
Architecture
Random variation
StructuredSelection
Randomand
small
Architecture
Variation
Co-factors
Fatty acids
Sugars
Nucleotides
Amino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Genes
DNA replication
Trans*
Architecture
E. coli genome
Structured variation
StructuredSelection
Structuredand
largeArchitecture
Variation
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Proteins
Pre
curs
ors
Autocatalytic feedback
Nut
rien
ts
Core metabolism
DNA replication
Regulation & control
Trans*
Cat
abol
ism
Carriers
Fragility example: Viruses
Viruses exploit the universal bowtie/hourglass structure to hijack the cell machinery.
Pre
curs
ors
Trans*Carriers
Fragility example: Viruses
Viruses exploit the universal bowtie/hourglass structure to hijack the cell machinery.
Genes
Co-factorsFatty acidsSugars
Nucleotides
Amino Acids
Proteins
Pre
curs
ors
Autocatalytic feedback
Nut
rien
ts
Core metabolism
DNA replication
Regulation & control
Trans*
Cat
abol
ism
Carriers
Fragility example: Viruses
Pre
curs
ors
Trans*Carriers Viral
genesViral
proteins
Co-factors
Fatty acids
Sugars
Nucleotides
Amino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Genes
DNA replication
Trans*
Regulation & control? E. coli
genome
Energy &materials
Co-factors
Fatty acids
Sugars
Nucleotides
Amino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Genes
DNA replication
Trans*
Regulation & control E. coli
genome
Co-factors
Fatty acids
Sugars
NucleotidesAmino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Regulation & control
Carriers
Core metabolism
Genes
DNA replicationRegulation & control
Trans*
Not to scale
essential: 230 nonessential: 2373 unknown: 1804 total: 4407
http://www.shigen.nig.ac.jp/ecoli/pec
Gene networks?
Co-factors
Fatty acids
Sugars
Nucleotides
Amino Acids
Catabolism
Polymerization and complex
assembly
Proteins
Pre
curs
ors
Autocatalytic feedback Taxis and transport
Nut
rien
ts
Carriers
Core metabolism
Genes
DNA replication
Trans*
Regulation & control E. coli
genome
