Emergent complexity

Chaos and fractals

Uncertain Dynamical Systems

c-plane

1 1k k kz cz z

0c

planec

bounded open and connectedkc Mset z

bounded is "Cantor dust"kc Mset z

1 1k k kz cz z c i

boundedkc Mset z

planez

1 1k k kz cz z

Julia sets

1 1k k kz cz z 0c

bounded open and connectedkc Mset z

Undecidable No algorithm to

determine has bounded run timec Mset

Overcoming computational

complexity

1 1 ,k k kz cz z z c c

boundedkc Mset z

What does this have to do with complex systems?

• This classic computational problem illustrates an important idea, but in an easily visualized way.

• Most computational problems involve uncertain dynamical systems, from protein folding to complex network analysis. Not easily visualized.

• Natural questions are typically computationally intractable, and conventional methods provide little encouragement that this can be systematically overcome.

Main idea

1

"Fragile" means

membership changes when

the map is perturbed:

1k k kz c z z

e.g. the boundary moves.

Main idea

Points near the boundary are “fragile.”

Merely stating the obvious in this case.

But illustrates general principle that can be exploited by the right

algorithms.

5

10

15

20

25

30

# iterations

Points not in M.

5

10

15

20

25

30

# iterations

Color indicates number of

iterations of simulation to show point is not in M.

But simulation cannot show that points are in M.

1 1 k k kz c z z

boundedkc Mset z

planec

1 1k k kz cz z

But simulation is fundamentally limited• Gridding is not scalable• Finite simulation inconclusive

0

1

2z

It’s easy to prove that this disk is in M.

Other points in M are fragile to the definition of the map.

1 1 k k kz c z z

Merely stating the obvious.

Main idea

1 1k k kz cz z

planec

2

1

221 0

1 1

k k

k k k

k

V z z

V z V z

z cz z

c z

2 decreases

1 1

V z z

c z

Sufficient condition

1 1k k kz cz z

2 decreases

1 1

V z z

c z

21

2 11 1 0 1z

2c

1c Mset

Trivial to prove that these points are in Mandelbrot set.

1c 2 1c

Main idea

The longer the proof, the more fragile the remaining regions.

The proof of this region is a bit longer.

Main idea

And so on…

Proof even longer.

Easy to prove these points are in Mset.

Easy to prove these points are not in Mset.

Proofs get harder.(But all still “easy.”)

What’s left gets more fragile.

Complexity»Chaos

Fractals

Emergent complexity.

Complexityimplies fragility

What matters to organized

complexity.

Emergent complexity

cHow might this help

with organized complexity and

“robust yet fragile”?

• Long proofs indicate a fragility.• Either a true fragility (a useful answer) or an

artifact of the model (which must then be rectified)• Potentially fundamentally changes computational

complexity for organized complexity• Brings back together two research areas that have

been separated for decades:• Numerical analysis and ill-conditioning• Computational complexity (P, NP/coNP,

undecidable)

Proof?

New proof methods that is scalable and systematic (can be

automated).

Breaking hard problems

• SOSTOOLS proof theory and software• Nested family of (dual) proof algorithms• Each family is polynomial time• Recovers most “gold standard” algorithms

as special cases, and immediately improves• No a priori polynomial bound on depth

(otherwise P=NP=coNP)• Conjecture: Complexity implies fragility

Safety Verification and Reachability Analysis

• Safety critical applications.• Exhaustive simulation is not exact.• Set propagation is computationally expensive.

Find a barrier certificate B(x)

( ) 0

( ) 0

( ) 0

B x x UnsafeSet

B x x InitialSet

Bf x x StateSpace

x

Initial set

Unsafe set

B(x) = 0

Scalable computation using SOS machinery.

• Parametric• Memoryless• Dynamic (IQC)

Hybrid, Uncertain, Stochastic

Hybrid systems can be handled easily,even for systems with uncertainty:

• Use supermartingales as certificates.• Get guaranteed bound on reach probability.

Also stochastic hybrid systems:

(Prajna, Jadbabaie – HSCC ’04)

(Prajna, Jadbabaie, Pappas – CDC ’04)

Feedback control

Variable supply/demand

Physical network

Components

Functionalrequirements

Hardware constraints

“Horizontal” Decompositions

“Ver

tica

l”

laye

ring

Unifying role of dual proofs and decomp-ositions

Main idea

Think of this as a robustness problem.

1 1

0k k kz cz z

0c

planec

How robust is stability to perturbations in c?

Globally stable.

1 1k k kz cz z

0c

planec

Region of convergence.

How robust is stability to perturbations in c?

1 1k k kz cz z planec planez

planec 1 1k k kz cz z planez

Simulation is fundamentally limited

planec 1 1k k kz cz z

planez

Simulation is fundamentally limited

0

1

2z

5

10

15

20

25

30

# iterations

10

20

30

40

50

60

iterations

10

20

30

40

50

60

iterations

-2

10

20

30

40

50

60

iterations

-2

10

20

30

40

50

60

iterations

10

20

30

40

50

60

iterations

70

120

iterations

120

iterations

60

1 1k k kz cz z

1 1k k kx cx x

realc

realx

1 1k k kx cx x

realc

kx kx

0c 0c

1kx 1kx

realc

realc

1 1k k kx cx x

realx

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1 1k k kx cx x

realc

1

10,1

x cx x

xc

realx

Fixed points

StableUnstable

Stable

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1x cx x realc

realx

Unstable

Stable

Stable

Fixed points

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1x cx x realc

realx

Unstable

Stable

Equilibria

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1 1k k kx cx x

realc

realx

2

1

221 0

1 1

k k

k k k

k

V x x

V x V x

x cx x

c x

2 decreases

1 1

V z z

c z

1 1k k kz cz z

planec

2

1

221 0

1 1

k k

k k k

k

V z z

V z V z

z cz z

c z

2 decreases

1 1

V z z

c z

Sufficient condition

1 1k k kz cz z

2 decreases

1 1

V z z

c z

21

2 11 1 0 1z

2c

1c Mset

Trivial to prove that these points are in Mandelbrot set.

1c 2 1c

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1x cx x realc

realx

Unstable

Equilibria

realc

Bifurcations for 1 1k k kx cx x

c

“last 200 x”

“last 200 x”

c

Zoom-in

“last 200 x”

c

Zoom-in

-3 -2 -1 0 1 2 3

-2

-1

0

1 1k k kx cx x 0 realc

Bounded for -2< 0 realc

stable

2 2 2

2 2 221 1 1 22 1 1 2 1 2

2 2x x c x c

c c

Bifurcations to chaos

-2

-1

0

-3 -2 -1 0 1 2 3-2

-1

0

1

2

1

221 0

1 1

k k

k k k

k

V x x

V x V x

x cx x

c x

2 decreases

1 1

V z z

c z

-2

-1

0

-3 -2 -1 0 1 2 3-2

-1

0

1

2

1

221 0

1 1

k k

k k k

k

V x x

V x V x

x cx x

c x

1 1

1 1 1 ?

k

k k

c x

c cx x

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

-3 -2 -1 0 1 2 3-2

-1

0

1

2

3

4

1 1k k kx cx x

realc

realx

2

1

221 0

1 1

k k

k k k

k

V x x

V x V x

x cx x

c x

2 decreases

1 1

V z z

c z

2 2 22 2 221 1 1 2

2 1 1 2 1 22 2

2 0 2 0

x x c x cc c

c c c

-3 -2 -1 0 1 2 3

-2

-1

0

1 1k k kx cx x

Bounded for - 2 0 real?c

stable

Bifurcations to chaos

2 22 2 1 2

2 0

c x c

c c

1 1k k kx cx x Invariant set?

2 22 2 1 1 2c cx x c

Invariance

2 22

2 222 1 22 1 1 2

2 0

c x cc cx x c

c c

2 22

2 222 1 2, 2 1 1 2 ?

2 0

c x cc cx x c

c c

-3 -2 -1 0 1 2 3

-2

-1

0

1 1k k kx cx x

Bifurcations to chaos

Special case of SOS

Special case of SOS

Special case of SOS

Special case of SOS

Contradiction!

-3 -2 -1 0 1 2 3

-2

-1

0

1 1k k kx cx x

2 22

2 222 1 2, 2 1 1 2 ?

2 0

c x cc cx x c

c c

What is the shortest proof possible?

Can prove the whole yellow region using SOSTOOLS!

1c 2 1c

Lyapunov argument

How to prove membership?2-period lobes

12

3

3

4

4

4Proof

lengths

Prove membership of 2-period lobe:• Using a stability argument of the 2-period map.• Using an invariance argument.

-1

Formulate the invarianceProblem as the emptinessof a semialgebraic set.

Then use SOSTOOLS to construct the certificate.

-6 -4 -2 0 2 4 6-6

-4

-2

0

2

4

6

x1

x 2

1 1

1 1 2

2 32 1 20.1 2 0.1

x x x

x x x x x

Discrete → Continuous

Let 0 be an equilibrium of

( ), .

Let be a region containing 0 and let

: be a continuously differentiable

function such that

( ) 0 in

nx f x x

D

V D

V x D

V

( ) ( ) 0 in

Then 0 is asymptotically stable.

Vx f x D

x

Lyapunov’s theorem

( ) 0 in

( ) ( ) 0 in

V x D

VV x f x D

x

-6 -4 -2 0 2 4 6-6

-4

-2

0

2

4

6

x1

x 2

Can we test these conditions

algorithmically?

Use the Sum of Squares decomposition!

( ) 0 in

( ) ( ) 0 in

V x D

VV x f x D

x

| ( ) 0, 1, ,niD x g x i n

( ), SOS, SOS, ( ) 0, ( ) 0i iV x p q x x

1

( ) ( )N

i ii

V x x p g

1

( ) ( )N

i ii

V x x q g

is SOS

is SOS

Find

such that

SOSTOOLS

( ) 0 in

( ) ( ) 0 in

V x D

VV x f x D

x

Then equilibrium is asymptotically stable.

( , ), ,(0, ) 0

( , ) 0 for ,

( , ) 0 for ,

x f x p x D p Pf p

V x p x D p P

V x p x D p P

Robust Stability?

Describe both and as semilalgebraic sets.D P

Use SOSTOOLS to construct V(x,p).

www.cds.caltech.edu/sostools

Chemical oscillator? ,

2 3constant

X A A BX Y X

Y B

2

2

x a x x y

y b x y

Nondimensional state equations

2

2

x a x x y

y b x y

3

Limit cycle for

b a b a

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

a

b

Can be computed analytically, which is not scalable.

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

a

b

0 0.1 0.2 0.3 0.4 0.5 0.6

1

1.5

2

2.5

3

a = 0.1, b = 0.13

2

2

x a x x y

y b x y

Numerical simulation.

1 1.5 2 2.50

0.2

0.4

0.6

0.8

1

x

y

2.2 2.6 3 3.40

0.2

0.4

0.6

0.8

1

a = 1, b = 2

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

a

b

a = 0.6, b = 1.1

(1.1, 0.6) (2, 1)

2

2

x a x x y

y b x y

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

a

b

a a

b b

,b a

2

2

x a x x y

y b x y

2

2

0

0

a x x y

b x y

equilibrium

2

2

x a x x x x y y

y b x x y y

2 2

2 2

x x

y y

a a

b b

Search for ( , ) satisfying

the Lyapunov conditions using

SOSTOOLS.

V x y

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

a

b

,b a

4 order ( , ) th V x y

Modeling Analysis

Set of possible system behaviors

Set of bad system behaviors

Proof of robustness

More on Saturday