Hybrid Systems Modeling, Analysis, Control Datta Godbole, John Lygeros, Claire Tomlin, Gerardo Lafferiere, George Pappas, John Koo Jianghai Hu, Rene Vidal,

Hybrid Systems Modeling, Analysis, Control

Datta Godbole, John Lygeros, Claire Tomlin, Gerardo Lafferiere, George Pappas, John Koo

Jianghai Hu, Rene Vidal, Shawn Shaffert, Jun Zhang,

Slobodan Simic, Kalle Johansson, Maria Prandini

David Shim, Jin Kim, Omid Shakernia, Cedric Ma, Judy Liebmann and Ben Horowitz

(with the interference of) Shankar Sastry

What Are Hybrid Systems?

Dynamical systems with interacting continuous and discrete dynamics

Why Hybrid Systems?

• Modeling abstraction of– Continuous systems with phased operation (e.g. walking robots,

mechanical systems with collisions, circuits with diodes)– Continuous systems controlled by discrete inputs (e.g. switches,

valves, digital computers)– Coordinating processes (multi-agent systems)

• Important in applications– Hardware verification/CAD, real time software– Manufacturing, chemical process control,– communication networks, multimedia

• Large scale, multi-agent systems– Automated Highway Systems (AHS)– Air Traffic Management Systems (ATM)– Uninhabited Aerial Vehicles (UAV), Power Networks

Control Challenges

• Large number of semiautonomous agents

• Coordinate to– Make efficient use of common resource

– Achieve a common goal

• Individual agents have various modes of operation

• Agents optimize locally, coordinate to resolve conflicts

• System architecture is hierarchical and distributed

• Safety critical systems

Challenge: Develop models, analysis, and synthesis tools for designing and verifying the safety of multi-agent systems

Proposed Framework

Control TheoryControl of individual agentsContinuous modelsDifferential equations

Computer ScienceModels of computationCommunication modelsDiscrete event systems

Hybrid Systems

xç = f 1(x; y)yç = f 2(x; y)

xç = g1(x; y)yç = g2(x; y)

x ô5

q1 q2x > 4à! x 2[0;1]; y = 1

y > 10à! x = 0; y2[1;3]yô10

x 2[0;1]y2[0;1]

Different Approaches

Automated Highway Systems

• Goal– Increase highway throughput

– Same highway infrastructure

– Same level of safety

– Same level of passenger comfort

• Introduce automation– Partial: driver assistance, intelligent cruise control, warning system

– Full: individual vehicles, mixed traffic, platooning

• Complex problem– Technological issues (is it possible with current technology)

– Social/Political issues (insurance and legal issues, equality)

Safety-Throughput Tradeoff

• Contradictory demands– Safety: vehicles far and moving slowly

– Throughput: vehicles close and moving fast

• Proposed compromise– Allow low relative velocity collisions

– In emergency situations

• Two possible safe arrangements– Large spacing (leader mode)

– Small spacing (follower mode)

• Platooning concept

Control Hierarchy

• Implementation requires automatic control

• Control hierarchy proposed in [Varaiya 93]– Regulation layer: braking, acceleration and steering

– Coordination layer: maneuvers implemented by communication protocols

– Link layer: flow control, lane assignment

– Network layer: routing

• Hybrid phenomena appear throughout– Switching controllers for regulation

– Switching between maneuvers

– Lane and maneuver assignment

– Degraded modes of operation

Air Traffic Management Systems

• Studied by NEXTOR and NASA

• Increased demand for air travel– Higher aircraft density/operator workload

– Severe degradation in adverse conditions

– High business volume

• Technological advances: Guidance, Navigation & Control– GPS, advanced avionics, on-board electronics

– Communication capabilities

– Air Traffic Controller (ATC) computation capabilities

• Greater demand and possibilities for automation– Operator assistance

– Decentralization

– Free flight

Automated Platoons on I-15

Computable Hybrid Systems

Current ATM System

TRACON

TRACON

CENTER ACENTER B

20 Centers, 185 TRACONs, 400 Airport TowersSize of TRACON: 30-50 miles radius, 11,000ftCenters/TRACONs are subdivided to sectorsApproximately 1200 fixed VOR nodesSeparation Standards Inside TRACON : 3 miles, 1,000 ft Below 29,000 ft : 5 miles, 1,000ft Above 29,000 ft : 5 miles, 2,000ft

VOR

SUA

GATES


Current ATM System Limitations

• Inefficient Airspace Utilization– Nondirect, wind independent, nonoptimal routes

• Centralized System Architecture– Increased controller workload resulting in holding patterns

• Obsolete Technology and Communications– Frequent computer and display failures

• Limitations amplified in oceanic airspace– Separation standards in oceanic airspace are very conservative

In the presence of the predicted soaring demand for airtravel, the above problems will be greatly amplified leading to both safety and performance degradation in the future


A Future ATM Concept

TRACON

TRACON

CENTER ACENTER B

• Free Flight from TRACON to TRACON– Increases airspace utilization

• Tools for optimizing TRACON capacity– Increases terminal area capacity and throughput

• Decentralized Conflict Prediction & Resolution– Reduces controller workload and increases safety

PROTECTED ZONE

ALERT ZONE

Hybrid Systems in ATM

• Automation requires interaction between – Hardware (aircraft, communication devices, sensors, computers)

– Software (communication protocols, autopilots)

– Operators (pilots, air traffic controllers, airline dispatchers)

• Interaction is hybrid– Mode switching at the autopilot level

– Coordination for conflict resolution

– Scheduling at the ATC level

– Degraded operation

• Requirement for formal design and analysis techniques– Safety critical system

– Large scale system

Control Hierarchy

• Flight Management System (FMS)– Regulation & trajectory tracking

– Trajectory planning

– Tactical planning

• Strategic planning– Decentralized conflict detection

and resolution

– Coordination, through communication protocols

• Air Traffic Control– Scheduling

– Global conflict detection and resolution

Hybrid Research Issues

• Hierarchy design

• FMS level– Mode switching

– Aerodynamic envelope protection

• Strategic level– Design of conflict resolution maneuvers

– Implementation by communication protocols

• ATC level– Scheduling algorithms (e.g. for take-offs and landings)

– Global conflict resolution algorithms

• Software verification

• Probabilistic analysis and degraded modes of operation

Other Applications

• Uninhabited Aerial Vehicles (UAV)– Automated aerial vehicles (airplanes and/or helicopters)

– Coordinate for search and rescue, or seek and destroy missions

– Control hierarchy similar to ATM

– Mode switching, discrete coordination, flight envelope protection

• Power Electronic Building Blocks (PEBB)– Power electronics, with sensing, control, communication

– Improve power network efficiency and reliability for utilities, hybrid electric vehicle, universal power ships

– Control hierarchy: load balancing/shedding, network stabilization, pulse width modulation

– Hybrid phenomena: modulation, input characteristic switching, scheduling

TCP/IPTCP/IP

Wireless LAN

GROUNDSTATIONVIRTUAL COCKPIT

GRAPHICALEMMULATION

WIRELESSHUB

UAV Laboratory Configuration

Motivation

• GoalGoal– Design a multi-agent multi-modal control system for Unmanned Aerial Vehicles Design a multi-agent multi-modal control system for Unmanned Aerial Vehicles

(UAVs)(UAVs)• Intelligent coordination among agentsIntelligent coordination among agents• Rapid adaptation to changing environmentsRapid adaptation to changing environments• Interaction of models of operationInteraction of models of operation

– Guarantee Guarantee • Safety Safety • PerformancePerformance• Fault toleranceFault tolerance• Mission completionMission completion

Conflict ResolutionConflict ResolutionCollision AvoidanceCollision AvoidanceEnvelope ProtectionEnvelope ProtectionTracking ErrorTracking Error

Fuel ConsumptionFuel ConsumptionResponse TimeResponse TimeSensor FailureSensor Failure

Actuator FailureActuator Failure Path FollowingPath FollowingObject SearchingObject SearchingPursuit-EvasionPursuit-Evasion

Hierarchical Hybrid Systems

• Envelope Protecting ModeEnvelope Protecting Mode

• Normal Flight ModeNormal Flight Mode

SafetyInvariant LivenessReachability

Tactical Planner

The UAV Aerobot Club at Berkeley

• Architecture for multi-level rotorcraft UAVs 1996- to date• Pursuit-evasion games 2000- to date• Landing autonomously using vision on pitching decks 2001- to

date• Multi-target tracking 2001- to date• Formation flying and formation change 2002

Hybrid Automata

• Hybrid Automaton

– State space

– Input space

– Initial states

– Vector field

– Invariant set

– Transition relation

• Remarks:– countable,

– State

– Can add outputs, etc. (not needed here)

H = (X ; V; I ni t; f ; I nv; R )

X = X C âX DV = VC âVDI ni t òXf : X âV ! <

n

I nvòX âVR : X âV ! 2X

X D ; VD X C =<n; VC ò<

m

x = (q; y) 2X

Executions

• Hybrid time trajectory, , finite or infinite with

• Execution with and

– Initial Condition:

– Discrete Evolution:

– Continuous Evolution: over , continuous, piecewise

continuous, and

• Remarks:– x, v not function, multiple transitions possible

– q constant along continuous evolution

– Can study existence uniqueness

ü= f[üi ; ü0i]gNi=0

ü0ià1= üi ôü0iÿ = (ü; x; v) x : ü! X ; v : ü! V

x(ü0) 2I ni tx(üi+1) 2R (x(ü0i); v(ü0i))

x(x(t); v(t)) 2I nv;8t 2[üi ; ü0i)yç = f (x; v)

[üi ; ü0i] v

Controller Synthesis: Example

• 2D conflict resolution

• Ensure aircraft remain more than 5nmi from each other

Hybrid Automaton Specification

• Discrete input variable determines maneuver initiation

• Safety specification

x2r + y2r õd2

More Abstractly ...

• Consider plant hybrid automaton, inputs partitioned to:– Controls, U

– Disturbances, D

• Controls specified by “us”

• Disturbances specified by the “environment”– Unmodeled dynamics

– Noise, reference signals

– Actions of other agents

• Memoryless controller is a map

• The closed loop executions are

g : X ! 2U

EH g = f(ü; x; (u; d)) 2EH j8t 2ü; u(t) 2g(x(t))g

Controller Synthesis Problem

• Given H and find g such that

• A set is controlled invariant if there exists a controller such that all executions starting in remain in

Proposition: The synthesis problem can be solved iff there exists a unique maximal controlled invariant set with

• Seek maximal controlled invariant sets & (least restrictive) controllers that render them invariant

• Proposed solution: treat the synthesis problem as a non-cooperative game between the control and the disturbance

8(ü; x; (u; d)) 2EH g; 8t 2ü; x(t) 2FF òX

W òXW W

I ni t òW òF

Gaming Synthesis Procedure

• Discrete Systems: games on graphs, Bellman equation

• Continuous Systems: pursuit-evasion games, Isaacs PDE

• Hybrid Systems: for define

– states that can be forced to jump to for some

– states that may jump out of for some – states that whatever does can be

continuously driven to avoiding by – Initialization:

while do

end

K ; L ò F

Preu(K ) òXPred(K

c) òX

Reach(K ; L ) òX

W0 = F ; Wà1 = ;; i = 0W i6=W

ià1

i = i + 1W i+1 = W i

nReach(Preu(Wi); P red(W

i c))

u

d

K

K L

d

uK d

Proposition: If the algorithm terminates, the fixed point isthe maximal controlled invariant subset of F

Proposition: If the algorithm terminates, the fixed point isthe maximal controlled invariant subset of F

Algorithm Interpretation

X

(W i)c

Pred(Wi)c

Preu(Wi)

Reach(Pred(Wi)c; P reu(W

i))

Computation

• One needs to compute , and

• Computation of the Pre is straight forward (conceptually!): invert the transition relation

• Computation of Reach through a pair of coupled Hamilton-Jacobi partial differential equations

• Semi-decidable if Pre, Reach are computable

• Decidable if hybrid automata are rectangular, initialized.

Preu Pred Reach

Pred(K ) = fx 2X jx 2Kc_

Preu(K ) = fx 2K j9u 2U;8d2D ; (x; (u; d)) 62I nv^R (x; (u; d)) òK g

8u 2U;9d2D ;R (x; (u; d)) \ Kc6=;g

Application: Control of Automated Highway Systems

• Design of vehicle controllers & performance estimation

• Two concepts– platooning & individual vehicles

Network

Link

Coordination

Regulation •Lane keeping•Vehicle following

•Maneuver selection•inter-vehicle comm

•Dynamic routing

•Flow optimization Entry

Exit

LaneChange

PlatoonFollowing

Join

Split

Speed,vehiclefollowing


Vehicle Following & Lane Changing

• Control actions: (vehicle i) -- braking, lane change

• Disturbances: (generated by neighboring vehicles) -- deceleration of the preceding vehicle

-- preceding vehicle colliding with the vehicle ahead of it

-- lane change resulting in a different preceding vehicles

-- appearance of an obstacle in front

• Operational conditions:– state of vehicle i with respect to traffic

i

j

i-1 i-2

Game Theoretic Formulation

• Requirements– Safety (no collision)

– Passenger Comfort

– Efficiency• trajectory tracking (depends on the maneuver)

• Safe controller (J1): Solve a two-person zero-sum game

– saddle solution (u1*,d1*) given by

• Both vehicles i and i-1 applying maximum braking

• Both collisions occur at T=0 and with maximum impact

J x u d x t J Ct

10

03 1 1 0( , , ) inf ( );

J x u d u t J C mst

20

02 2

32 5( , , ) sup| ( )|; .

u U d D J x u d J x u d J x u d, , ( , , ) ( , , ) ( , , )* * * * * 10

1 10

1 1 10

1

Safe Vehicle Following Controller

• Partition the state space into safe & unsafe sets

0min,3

04

02

01 ),,(: xxxxS

Design comfortable andefficient controllers inthe interior•IEEE TVT 11/94

Safe set characterizationalso provides sufficientconditions for lane change•CDC 97, CDC98

Automated Highway System Safety

• Theorem 1: (Individual vehicle based AHS) – An individual vehicle based AHS can be designed to produce no inter-

vehicle collisions, – moreover disturbances attenuate along the vehicle string.

• Theorem 2: (Platoon based AHS)– Assuming that platoon follower operation does not result in any

collisions even with a possible inter-platoon collision during join/split, a platoon based AHS can be safe under low relative velocity collision criterion.

• References– Lygeros, Godbole, Sastry, IEEE TAC, April 1998– Godbole, Lygeros, IEEE TVT, Nov. 1994

Example: Aircraft Collision Avoidance

Two identical aircraft at fixed altitude & speed:

ud

uxv

uyvv

duy

x

dt

d

sin

cos

),,(xf

‘evader’ (control) ‘pursuer’ (disturbance)

x

y

uv

d

v

Continuous Reachable Set

x

y

[Mitchell, Bayen, Tomlin 2001][Tomlin, Lygeros, Sastry 2000]

Fast Wavefront Approximation Methods (Tomlin, Mitchell)

Visualization of Unsafe Set:Mitchell-Tomlin

Transition Systems

• Transition System

• Define for

• Given equivalence relation define

T = (Q; Î ;! ; QO; Q F )

û 2Î ; P òQ

Preû(P ) = fq2Q j9p2P and q ! û pgøòQ âQ

T=ø= (Q=ø; Î ;! ø; QO=ø; Q F=ø)

• A ~ block is a union of equivalence classes

QO

Q F

Bisimulations of Transition Systems

A partition ~ is a bisimulation iff – are ~ blocks

– For all and all ~ blocks is a ~ block

A partition ~ is a bisimulation iff – are ~ blocks

– For all and all ~ blocks is a ~ block

• Why are bisimulations important?

QO; Q Fû 2Î P ; P reû(P )

• Alternatively, for P 1; P 22Q=ø; P 1 \ Preû(P 2) = ; or P 1

QO

Q F

Preû(Q F )

Bisimulation Algorithm

initialize :

while such that

define

refine

Q=ø= fQO; Q F ; Q n(QO [ Q Fg9P 1; P 22Q=ø; û 2Î

;6=P 1 \ Preû(P 2)6=P 1R 1 = P 1 \ Preû(P 2); R 2 = P 1nPreû(P 2)Q=ø= (Q=ønfP 1g) [ fR 1; R 2g

QO

Q F

Preû(Q F )

• If algorithm terminates, we obtain a finite bisimulation

• Decidability requires the bisimulation algorithm to – Terminate in finite number of steps and

– Be computable

• For the bisimulation algorithm to be computable we need to– Represent sets symbollically,

– Perform boolean combinations on sets

– Check emptyness of a set,

– Compute Pre(P) of a set P

• Class of sets and vector fields must be topologically simple– Set operations must not produce pathological sets

– Sets must have desirable finiteness properties

Computability & Finitiness

Mathematical Logic

• Every theory of the reals has an associated language

• Decidable theories– Every formula is equivalent to a quantifier free formula

– Quantifier free formulas can be decided

• Quanitifier elimination

• Computational tools (REDLOG, QEPCAD)

(<; <;+ ;0;1)(<; <;+ ;â ; 0; 1)(<; <;+ ;â ; e

x; 0; 1)9w : xw2+ yw + z = 0

9t 9y : t õ0^y = 5^y = etx

9t 9y : t õ0^y = 5^y = x + t

9w : xw2+ yw + z = 0 ñ y2à4xz õ09t 9y : t õ0^y = 5^y = x + t ñ x ô5

9t 9y : t õ0^y = 5^y = etx ñ 0 < x ô5

O-Minimal Theories

• A definable set is

A theory of the reals is called o-minimal if every definable subset of the reals is a finite union of points and intervals

• Example: for polynomial

• Recent o-minimal theories

f(x1; . . .; xn) 2<njþ(x1; . . .; xn)g

(<; <;+ ; 0; 1)(<; <;+ ;â ; 0; 1)(<; <;+ ;â ; e

x; 0; 1)

(<; <;+ ;â ; ffêg; 0; 1)(<; <;+ ;â ; e

x; ffêg; 0; 1)

fx 2<jp(x) > 0g p(x)

Semilinear sets

Semialgebraic sets

Bounded Subanalytic sets

Exponential flows

Spirals ?

O-Minimal Hybrid Systems

A hybrid system H is said to be o-minimal if• the continuous state lives in

• For each discrete state, the flow of the vector field is complete

• For each discrete state, all relevant sets and the flow of the vector field are definable in the same o-minimal theory

Main Theorem Main Theorem

Every o-minimal hybrid system admits a Every o-minimal hybrid system admits a finitefinite bisimulation. bisimulation.

• Bisimulation alg. terminates for o-minimal hybrid systems

• Various corollaries for each o-minimal theory

<n


Consider hybrid systems where– All relevant sets are polyhedral

– All vector fields have linear flows

Then the bisimulation algorithm terminates

(<; <;+ ; 0; 1)

Consider hybrid systems where– All relevant sets are semialgebraic

– All vector fields have polynomial flows


(<; <;+ ;â ; 0; 1)


Consider hybrid systems where– All relevant sets are subanalytic

– Vector fields are linear with purely imaginary eigenvalues


Consider hybrid systems where– All relevant sets are semialgebraic

– Vector fields are linear with real eigenvalues


(<; <;+ ;â ; ffêg; 0; 1)

(<; <;+ ;â ; ex; 0; 1)


Consider hybrid systems where– All relevant sets are subanalytic

– Vector fields are linear with real or purely imaginary eigenvalues


(<; <;+ ;â ; ex;ffêg; 0; 1)

• New o-minimal theories result in new finiteness results• Can we find constructive subclasses?

– Must remain within decidable theory – Sets must be semialgebraic – Need to perfrom reachability computations

• Reals with exp. does not have quantifier elimination

(<; <;+ ;â ; 0; 1)

<n

Linear Hybrid Systems

A hybrid system H is said to be linear if• the continuous state lives in

• For each discrete state, all relevant sets are semialgebraic

• For each discrete state, the vector field is of the form

where matrix has rational entries

• Let . Then we can express

• Focus on the subformula

xç = Ax A

Y = fy j P (y)gPreü(Y) = f9y9t : P (y)^t õ0^x = eàtAy^

8t0: 0ô t0ô t ) I (eàtA )g

9t : t õ0^x = eàtAy

Diagonalizable, Rational Eigenvalues

• hLet be a linear vector field, rational, diagonalizable with rational eigenvalues. Then is definable in the decidable theory of reals

Example:

F (x) = A x APreü(Y)

x2ç = àx2x1ç = 2x1

Y = f(y1; y2) jy1 = 4^y2 = 3g9y1 = 49y2 = 39t õ0 : x1 = y1eà2t^x2 = y2e

t

ñ9y1 = 49y2 = 390 < z ô1 : x1 = y1z2^x2 = y2zà1

ñ9y1 = 49y2 = 390 < w1ô19w2 > 0 : w1w2 = 1^x1 = y1w

21^x2 = y2w2

ñx1x22à36 = 0^x2 > 0

Preü(Y) =

Diagonalizable, Imaginary Eigenvalues

• Procedure is conceptually similar if is diagonalizable with purely imaginary, rational eigenvalues

• Equivalence is obtained by

• Suffices to compute over a period

• Composing all the constructive results together gives in…

F (x) = A x

A

z1 = cos t; z2 = sin t

Let be a linear vector field, rational, diagonalizable with purely imaginary rational eigenvalues. Then is definable in the decidable theory of reals

A

Preü(Y)

Semidecidable Linear Hybrid Systems

Let H be a linear hybrid system H where for each discrete

location the vector field is of the form F(x)=Ax where

• A is rational and nilpotent

• A is rational, diagonalizable, with rational eigenvalues

• A is rational, diagonalizable, with purely imaginary, rational eigenvalues

Then the reachability problem for H is semidecidable.

• Above result also holds if discrete transitions are not necessarily initialized but computable

Decidable Linear Hybrid Systems


location the vector field is of the form F(x)=Ax where• A is rational and nilpotent• A is rational, diagonalizable, with rational eigenvalues• A is rational, diagonalizable, with purely imaginary, rational

eigenvalues

Then the reachability problem for H is decidable.

x2+ y2 > 4à! x 2[0;1]; y = 1

y > 10à! x = 0; y2[1;3]

x 2[0; 1]

yô10

y2[0;1]

x2+ y2ô4

xç = x + y

yç = ày yç = 4y

xç = à2x

Linear Hybrid Systems with Inputs


location, the dynamics are where A,B are

rational matrices and one of the following holds:

• A is nilpotent, and

• A is diagonalizable with rational eigenvalues, and

• A is diagonalizable with purely imaginary eigenvalues and

Then the reachability problem for H is decidable.

u(t) =Xi=0n

aiti

u(t) =Xi=0n

aieõ it i

u(t) =Xi=0n

ai sin(! it)

õ i 62Spec(A )

j ! i 62Spec(A )

xç = Ax + B u

Linear DTS (compare with Morari Bemporad)

• X = n, U = {u|Eu}, D = {d|Gd}, f = {Ax+Bu+Cd},

F = {x|Mx}.

• Pre(Wl) = {x | l(x)}

l(x) = u d | [Mlxl]c[Eu] [(Gd>)(MlAx+MlBu+MlCd l)]

• Implementation

– Quantifier Elimination on d: Linear Programming

– Quantifier Elimination on u: Linear Algebra

– Emptiness: Linear Programming

– Redundancy: Linear Programming

Decidability Results for Algorithm

The controlled invariant set calculation problem is

• Semi-decidable in general.

• Decidable when F is a rectangle, and A,b is in controllable canonical form for single input single disturbance.

Extensions:

Hybrid systems with continuous state evolving according to discrete time dynamics: difficulties arise because sets may not be convex or connected.

There are other classes of decidable systems which need to be identified.

Summary

• Methodology– Modeling Framework

– Game theoretic approach to controller synthesis

– Linear hybrid systems and computability

• Applications– Synthesis of safe conflict resolution maneuvers

– Safe controllers for automated highways

– Verification of avionic software (CTAS, TCAS)

– Flight Envelope Protection

– Flight Mode Switching

Newer Research

• Modeling – Robustness, Zeno (Zhang, Simic, Johansson)– Simulation, on-line event detection (Johannson, Ames)

• Control– Extension to more general properties (liveness, stability) (Koo)– Links to viability theory and viscosity solutions (Lygeros, Tomlin,

Mitchell, Bayen)– Numerical solution of PDEs (Tomlin, Mitchell)

• Analysis– Develop (exact/approximate) reachability tools (Vidal, Shaffert)– Complexity analysis (Pappas, Kumar)

• Probabilistic Hybrid Systems (Hu)• Observability of Hybrid Systems (Vidal)

Documents

Hybrid Systems Modeling, Analysis, Control Datta Godbole, John Lygeros, Claire Tomlin, Gerardo Lafferiere, George Pappas, John Koo Jianghai Hu, Rene Vidal,