An Introduction to the Soft Walls Project Adam Cataldo Prof. Edward Lee University of Pennsylvania Dec 18, 2003 Philadelphia, PA

An Introduction to the Soft Walls Project

Adam CataldoProf. Edward Lee

University of PennsylvaniaDec 18, 2003Philadelphia, PA

Outline

• The Soft Walls system• Objections• Control system design• Current Research• Conclusions

Introduction

• On-board database with “no-fly-zones”• Enforce no-fly zones using on-board

avionics• Non-networked, non-hackable

Autonomous control

Pilot

Aircraft

Autonomouscontroller

Softwalls is not autonomous control

Pilot

Aircraft

Softwalls

+

bias pilotcontrol

Relation to Unmanned Aircraft

• Not an unmanned strategy– pilot authority

• Collision avoidance

A deadly weapon?

• Project started September 11, 2001

Design Objectives

Maximize Pilot Authority!

Unsaturated Control

No-flyzone

Control applied

Pilot lets off controls

Pilot turns away from no-fly zone

Pilot tries to fly into no-fly

zone

Objections

• Reducing pilot control is dangerous– reduces ability to respond to emergencies

There is No Emergency That Justifies Landing Here

Objections


• There is no override– switch in the cockpit

Hardwall

Objections



• Localization technology could fail– GPS can be jammed

Localization Backup

• Inertial navigation• Integrator drift

limits accuracy range

Objections




• Deployment could be costly– Software certification? Retrofit older

aircraft?

Deployment

• Fly-by-wire aircraft– a software change

• Older aircraft– autopilot level

• Phase in– prioritize airports

Objections




• Deployment could be costly– how to retrofit older aircraft?

• Complexity– software certification

Not Like Air Traffic Control

• Much Simpler• No need for

air traffic controller certification

Objections




• Deployment could be costly– how to retrofit older aircraft?

• Deployment could take too long– software certification

• Fully automatic flight control is possible– throw a switch on the ground, take over plane

Potential Problems with Ground Control

• Human-in-the-loop delay on the ground– authorization for takeover– delay recognizing the threat

• Security problem on the ground– hijacking from the ground?– takeover of entire fleet at once?– coup d’etat?

• Requires radio communication– hackable– jammable

Here’s How It Works

What We Want to Compute

No-fly zone

Backwards reachable set

States that canreach the no-flyzone even with Soft Walls controlllerCan prevent aircraft from

entering no-fly zone

What We Create

• The terminal payoff function l:X -> Reals

• The further from the no-fly zone, the higher the terminal payoff

payoff

northwardposition

eastwardpositionno-fly zone

(constant over heading angle)

-

+

What We Compute

No-fly zone

terminal payoff

Backwards Reachable Set

optimal payoff

Our Control Input


optimal control at boundary

dampen optimal control away from boundary

State Space

optimal payoff function

How we computing the optimal payoff (analytically)

• We solve this equation for J*: Reals^n x (-∞, 0] -> Reals

• J* is the viscosity solution of this equation• J* converges pointwise to the optimal

payoff as T->∞• (Tomlin, Lygeros, Pappas, Sastry)

)()0,(

0),,(),(maxmin,0min),( **

ylyJ

evyfTyJTyJT y

UvDe

dynamicsspatial gradient

time derivativeterminal payoff

• (Mitchell)

• Computationally intensive: n statesO(2^n)

How we computing the optimal payoff (numerically)

northwardposition

eastwardposition

heading angle

no-fly zone

time0 1 M

Current Research

• Model predictive controller– Linearize the system– Discretize time– Compute an optimal control input for the

next N steps– Use the optimal input at the current step– Recompute at the next step

• Feasible in real time

Current Research

• Discrete Abstraction– Discretize time– Discretize space into a finite set

• Feasible for Verification

northwardposition

eastwardposition

heading angle

1

2

3

4

Conclusions

• Embedded control system challenge• Control theory identified• Future design challenges identified

Acknowledgements

• Ian Mitchell• Xiaojun Liu• Shankar Sastry• Steve Neuendorffer• Claire Tomlin

• State space X, a subset of Reals^n

• Control space U and disturbance Space D, compact subsets of Reals^u and Reals^d

Backup Slides—Problem Model

northwardposition

eastwardpositionheading angle

U

D

bank right-

bank left+

pilot

Soft Walls

Actions in Continuous Time

• Pilot (Player 1) action d:Reals -> D, a Lebesgue measurable function

• The set of all pilot actions Dt

time

bank left

(bank right)

Actions in Continuous Time

• Controller (Player 2) action u:Reals -> U, a Lebesgue measurable function

• The set of all controller actions Ut

time

bank left

(bank right)

Dynamics

• The state trajectory x:Reals -> X• The dynamics equation f:X x U x D ->

Reals^n• (d/dt)x(t) = f(x(t), u(t), d(t))

eastwardposition

northwardposition

heading angle

trajectory

Cataldo

Information Set

• At each time t, the controller knows pilot’s current action and the aircraft state

• We say s:Dt -> Ut is the controller’s nonanticipative strategy

• S is the set of nonanticipative strategies

eastwardposition

northwardposition

heading angle

trajectorytime

pilot bank left

(bank right)

controller knows

Information Set

• At each time t, the pilot knows pilot’s only the aircraft state

• We say the pilot uses a feedback strategy

eastwardposition

northwardposition

heading angle

trajectory

pilot knows

What We Want to Compute

No-fly zone

Backwards reachable set

States that canreach the no-flyzone even with Soft Walls controlllerCan prevent aircraft from

entering no-fly zone

The Terminal Payoff Function

• The terminal cost function l:X -> Reals


payoff

northwardposition



-

+

The Payoff Function

• Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals

• Aircraft state at time t is x(t)• y is any state yxtxlduyV

t

)0())((inf),,(

]0,(

distance from no-fly zone

time

trajectories that terminate at ygiven u and d as inputs

payoff at ygiven u and d

The Optimal Payoff

• At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff

)),(,(infsuparg)(

)),(,(infsup)(

*

*

ddsyVys

ddsyVyV

DtdSs

DtdSs

No-flyzone

optimalcontrol

strategy

optimaldisturbance

strategy

The Finite Duration Payoff

• We look back no more than T seconds to compute the finite duration payoff J: X x Ut x Dt x (-∞, 0] -> Reals and the optimal finite duration payoff J*:X x (-∞, 0] -> Reals

yxtxlduyVt

)0())((inf),,(]0,(

),),(,(infsup),(

)0())((inf),,,(

*

]0,[

TddsyJTyJ

yxtxlTduyJ

DtdSs

Tt

the only difference

Computing V*

• Assuming J* is continuous, it is the viscosity solution of

• J* converges pointwise to V* as T->∞• (Tomlin, Lygeros, Pappas, Sastry)

)()0,(

0),,(),(maxmin,0min),( **

ylyJ

evyfTyJTyJT y

UvDe

dynamicsspatial gradient

finite duration optimal payoff

What Our Computation Gives Us

No-fly zone

terminal payoff


optimal payoff

Control from Optimal Payoff Function


optimal control at boundary

dampen optimal control away from boundary

State Space

• (Mitchell)

• Computationally Intensive n states, O(2^n)

Numerical Computation of V*

northwardposition

eastwardposition

heading angle

no-fly zone

time0 1 M

• Same state space X

• Same control space U and disturbance Space D

What About Discrete-Time?

northwardposition

eastwardpositionheading angle

U

D

bank right-

bank left+

pilot

Soft Walls

Actions in Discrete Time

• Pilot (Player 1) action d:Integers -> D • Control (Player 2) action u:Integers -> U

• The set of all pilot actions Dk• The set of all control actions Uk

time

bank left

(bank right)

time

bank left

(bank right)

Dynamics

• The state trajectory x:Integers -> X• The dynamics equation f:X x U x D ->

Reals^n• x(k+1) = f(x(k), u(k), d(k))

eastwardposition

northwardposition

heading angle

trajectory

Same Information Set

eastwardposition

northwardposition

heading angle

trajectorytime

pilot bank left

(bank right)

controller knows

pilot knows

• We say s:Dk -> Uk is the controller’s nonanticipative strategy

• S is the set of nonanticipative strategies

Again We Compute

No-fly zoneBackwards reachable set

Can prevent aircraft from entering no-fly zone

Same Terminal Payoff Function• The terminal cost function l:X -> Reals


payoff

northwardposition



-

+

The Payoff Function

• Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals

• Aircraft state at time k is x(k)• y is any state yxkxlduyV

k

)0())((inf),,(

}0,1{...,

distance from no-fly zone

time

trajectories that terminate at ygiven u and d as inputs

payoff at ygiven u and d

The Optimal Payoff

• At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff

)),(,(infsuparg)(

)),(,(infsup)(

*

*

ddsyVys

ddsyVyV

DkdSs

DkdSs

No-flyzone

optimalcontrol

strategy

optimaldisturbance

strategy

The Finite Duration Payoff

• We look back no more than K seconds to compute the finite duration payoff J: X x Ut x Dt x {…, -1, 0} -> Reals and the optimal finite duration payoff J*:X x {…, -1, 0} -> Reals

yxkxlduyVk

)0())((inf),,(}0,1{...,

),),(,(infsup),(

)0())((inf),,,(

*

}0,1,...,{

KddsyJKyJ

yxkxlKduyJ

DkdSs

Kk

the only difference

Computing V*

• J* is the solution of

• J* converges pointwise to V* as K->∞

)()0,(

)}),,,((maxmin),,(min{)1,(

*

***

ylyJ

kevyfJkyJkyJUvDe

dynamicsfinite duration optimal payoff

How Do We Solve This Numerically?

)()0,(

)}),,,((maxmin),,(min{)1,(

*

***

ylyJ

kevyfJkyJkyJUvDe

No-fly zone

terminal payoff


optimal payoff

Documents

An Introduction to the Soft Walls Project Adam Cataldo Prof. Edward Lee University of Pennsylvania Dec 18, 2003 Philadelphia, PA