Goddard Space Flight Center 6 October 2004 Pursuit/Evasion of Fixed-wing Aircraft through Model-Predictive Control The new album by Jonathan Sprinkle Featuring,

Goddard SpaceFlight Center6 October 2004

Pursuit/Evasion of Fixed-wing Aircraft through Model-Predictive ControlThe new album by

Jonathan SprinkleFeaturing, Mike Eklund, Jin Kim

and Shankar Sastry

10/6/2004 Jonathan Sprinkle, et al., UC Berkeley 2

Overview

• Motivation• NMPC and PEGs

– Earlier work– Algorithm– Early results

• What’s the idea? (proof of concept)– Architecture– Tools – Simulation results

• Capstone Demonstration– Background– Development and simulation

environment– Experimental test flight results

• Featuring great movies!

• Future Work & Conclusions


Motivation

• UAVs are very useful for intelligence gathering– Small, low-observable, inexpensive, remote-

piloted (with autonomous tendencies…)

• Time-lag associated with remote control– Acceptable, or annoying, when in “steady-

state” flight• Waypoint flying, high-altitude loitering or

scanning

– Tactically interfering when rapid feedback is required

• Landing, hazardous weather, or stressful maneuvers


Motivation

• Consider the following:– Intelligence gathering UAV in flight– Alerted (internally, or by an observer) of an

adversary over the horizon• Either piloted, or perhaps fired (missile)

– Large amounts of data recently gathered, but not transmitted

• What to do?– Time lag is too large for remote “escape”– Adversary will most likely locate UAV shortly– Not enough time to transmit back all data


Motivation:

• Our proposal:– The pilot should be able to turn over

autonomy to the aircraft• will take evasive maneuvers as best it can• not necessarily guaranteed to save the aircraft,

but hopefully will be able to transmit back data

– Aircraft will return autonomy if it escapes– The use of various strategies for control will

enable this capability, and do so in an aircraft independent manner


Model Predictive Control

• MPC is a method for restricting/encouraging behavior

• A “fortune teller” controller• Restricts input ranges, as well as encourages

some inputs based on safety/stability concerns• Very useful for nonlinear systems, due to the

ability to get good optimizations with non-linear abstractions


How does it work?

• Basic algorithm:– Examine the mathematical abstraction of the system

(PDE)– Determine value along N time steps into the future– Optimize this value, according to some a priori

specifications (to )J = 0

J = Á(b1N ::M N) +

N ¡ 1X

k=0

L(x;u;b1::M ) = 0 (1)

Á(b1N ::M N) = C

m=MX

m=1

bTmB0m

bm (2)

L(xk;uk;bk1::M ) , C

Ã

xTk X 0xk + uT

k U 0uk +m=MX

m=1

bTmk

B0mbmk

!

(3)


System Model

• In the form of,

• Obviously, very system-dependent• Sometimes an abstraction of the actual

system in order to speed up computation

• Accuracy of the prediction, directly tied to the abstraction

• Eventually, arrive at a snapshot N steps in the future[x1;x2; : : : ;xN ]

_x = f (x;u)


Example: Aircraft Control

Begin

End

L(¢) , xTk X 0xk +uT

k U 0uk + bTm1

B01bm1

(1)



Begin

End

Obstacle


k U 0uk + bTm1

B01bm1

(1)

+ bTm2

B02bm2

(3)



Begin

End

Obstacle

Boundary


k U 0uk + bTm1

B01bm1

(1)

+ bTm2

B02bm2

(2)

+ bTm3

B03bm3

(3)


Example: Aircraft Control?

Enemy…

Begin

End

Obstacle

Boundary


k U 0uk + bTm1

B01bm1

(1)

+ bTm2

B02bm2

(2)

+ bTm3

B03bm3

(3)

+ bTm?

B0?bm?

(4)

(5)



• Now, what do you do?– Hope that you don’t get caught?– First, fight with you left hand, and then

surprise you opponent by not being left-handed

– Encode “getting away” from your opponent into the cost-function

“I admit it, you are better than I am”“Then why are you smiling?”“Because *I* am not left-handed”


Earlier UC Berkeley Pursuit/Evasion Experiments (David Shim, Jin Kim, et al)

• BErkeley Aerobot Project (BEAR), 1996-– Goal: to build a coordinated, intelligent

network with multiple heterogeneous agents• 11 Rotorcraft-based unmanned aerial vehicles

(UAVs) • 5 Unmanned ground vehicles (UGVs)• Shipdeck simulator (landing platform)

• Stochastic Pursuit-Evasion Games (PEG)– Self-localization– Target detection– Map building– Pursuit policy– Trajectory generation– Control / Action


Nonlinear Model Predictive Trajectory Control (NMPTC)

• Explicitly addresses nonlinear systems with constraints on operation and performance

• A cost minimization problem in the presence of state and input constraints– Control resulting in the minimum cost is

determined over a model predicted horizon

• Previously demonstrated in rotary wing UAVs[1]

[1] H.J. Kim, D.H. Shim and S. Sastry, ACC, 2002


NMPTC: Fixed wing application

• Dynamics and constraints are quite different than for rotary wing aircraft– Entirely new aircraft

model required

• Tactics– Function of

constraints on fixed wing aircraft, in particular

• Minimum airspeed • Maximum turn rate


NMPTC: For Pursuit/Evasion Games

• In this application, the low level control is done independently by the platform– Basically a tactics & trajectory generation

problem

• Same controller is used for pursuit and evasion– Cost function components and gains are

changed• E.g. AOT not used for pursuer

• Implemented as stand-alone application with Matlab interface for testing and controller tuning

• Integrated in OCP for Capstone Demonstration (using a real flight avionics feedback model)


NMPTC: Cost Function Definition

• Cost function is defined by:

1

0

0

, , 1 1

2 2

y x, y,u,p,d,a

where

1 y y y

2 and

1 1 1 x, y,u,d,a y y x x u u

2d d a a

x is the state

N

Nk

TN N N

T T T Tk k k k k k i k i i k

T Tn mk k k k

J L

P

L Q S R p B p

G T

vector u is the control vector

y is the trajectory error d is the pursuer/evader position difference

p is the evader

distance from the boundary

a is the angle off tail (AOT) or other tactical functions


NMPTC: Cost function illustration

, , 1 1

2 2

1 1 1 x, y,u,d,a y y x x u u

2d d a a

T T T Tk k k k k k i k i i k

T Tn mk k k k

L Q S R p B p

G T


NMPTC: Cost function minimization

• A gradient decent method is used to minimize the cost function

• Initialization with previous result reduces the number of iterations required– Usually 3-4 iterations are required

• The number of iterations in limited to prevent overruns in real-time– In rapidly changing situation this can result in

suboptimal solution– Sudden changes may take several time steps– However, this is not a problem because the

situation is changing and unpredictable anyway


NMPTC: PEG game conditions

• In this game the evader tries to cross the playing area while not being targeted by the pursuer

• The pursuer tries to target the evader

Target cone definition (θ=10˚,d=3 nm)Left: F15 not behind UAV, middle: F15 not pointed at

UAV, right: F15 behind AND pointed at UAV

F15

UAV

F15

UAV

F15

UAV

(b)(c)(a)

F15

UAV

F15

UAV

F15

UAV

(b)(c)(a)


NMPTC: Early simulation results






Example: Cost functions


Why are we doing this?


Software Enabled Control

“The goal of SEC is to develop new controls and software technology that will enable new applications that are impractical or intractable using current approaches.”


SEC Capstone Demonstration

• Capstone Demonstrations were proposed to highlight and test the technologies developed in the SEC program

• One would be a fixed wing UAV flight test– 6 participant technology developers (TDs)

• Honeywell, Northrop Grumman, U Minnesota, MIT, Stanford, and UCB/U Colorado/CalTech

– System Integrator was Boeing– OCP would be software framework– A T-33 trainer as UAV surrogate– An F-15 as wingman/opponent

• 12-14 month schedule May 03 – June 04


Demo: Non-Determinism in Experiments

• Customer comment:– “The timing and scope of these (pop-up events, like

faults, targets, threats) should not be totally known a-priori by the TDs.”

• Capstone Demo Approach– For flight test safety, platform owners and flight

crews are more comfortable with constraints on non-determinism

• Temporal non-determinism• Spatial non-determinism

• UCB Pursuit/Evasion Approach:– Two airplanes, no pre-determinism at all– Favorite responses at the last PI meeting:

• “You really don’t know who is going to win?!?!”• “What day is your test flight scheduled? I’ll be there”


Demo: Dryden Flight Research Center (DFRC)


Demo: DFRC Flight Demonstration Venue

• NASA provides range access, and approves test vehicles and missions

• Boeing provides vehicles and flight plans, and conducts flights

• NASA facilities– Fully instrumented Western Aeronautical Test Range– Mission Control Center – Restricted public access

• Boeing facilities– UCAV Flight Operations Control Center – Boeing only

• Flight test instrumentation, displays, telemetry recording, post-processing

– UCAV Van – DoD only• SEC Workstation control and display, Link 16 Terminal

– TDs were to be provided with a tent• But we got to share a camper van with A/C with the VIPs

some of the time


Demo: R-2508 Range Complex


Demo: Mission Area Example in R-2515


Demo: Mission Guidelines

• Day, Visual Flight Rules – “See and Avoid”• Mission Schedule

– Flight duration: 1:10 in mission area for both aircraft– Two flight per day can be supported

• Airspeed– T-33 alone: 250 kts typical– T-33 and F-15 joint operations: 300 kts typical

• Altitude– Normal operations: 10,000 ft to 20,000 ft MSL– Limited operations: Below 10,000 ft to surface with approval– Edwards-Dryden runways: 2,300 ft MSL– Terrain in R-2515: 2,300 ft to 6,000 ft MSL

• Formation– 1 nmi radial separation– +/- 500 ft


Demo: TD deliverables

• Software– Design consistent with defined OCP interfaces– Source code

• For emergency fixes and recompilations• For safety of flight reviews

– Binaries for linking– OCP application artifacts

• Simulink mdl file, ComponentInfo files• Test check cases

– Allow us to quickly assess correct functional behavior when hosting in our desktop development system and in platform integration labs (F-15, UCAV)

• Documentation– Describing software– Describing test check cases


Demo: State data available for control

• All UAV state data available – Limits due to lack of sensors on T-33, e.g.

• alpha, beta measurements

• Different set of F-15 state available– Transmitted by F-15 to the T-33

– Rate information• Data captured from UCAV avionics at 20 Hz• Baseline TD application execution rate is 2 Hz• Freshest 10 state data sets buffered as input at 2 Hz• From F-15 at 1 Hz


Demo: T-33 Software Architecture


Demo: Final integration and testing

• Development in desktop environment at UCB

• Sent to Boeing– Testing in UCAV AIC (HWIL) facility.

• Confidence testing of Laptop Demo Applications• Use scripted scenarios with actual mission plans

for Dryden

– Multi-Vehicle configuration testing– Integration of TD Software Components

• Integration into new OCP release B2.6.x


Demo: UCB PEG Development

• 20 – 60 min. games confirm NMPTC feasibility at real-time– Evader goal: get to final

waypoint or avoid evader– Pursuer goal: ‘target’ evader

• Pursuer and evader restricted to same performance limits

• Planes on the same logical plane, but separated by 6000ft altitude at all times

• Evader and pursuer have a few scenarios– UAV as evader – UAV can become pursuer

OCP Experiment Controller SnapshotT-33: Evader (yellow)F-15: Pursuer (blue)


Demo: Controller Coverage

• Final waypoint is the major component

• Travel to the waypoint can be interrupted by the actions of the pursuer– Relative positions– Angle-off-tail– Distance from each other– Proximity to boundaries of the

testing area– Physical limitations of the

aircraft• Velocity, climb/turn rate

• Use a simpler model for predictive behavior under certain conditions– Still use the Boeing-supplied

(DemoSim) model for behavioral simulations

Approx. same timePursuer goes fortarget cone

Evader turns away(regardless of endpoint)

Endpoint

Target cone definition (θ=10˚,d=3 nm)Left: F15 not behind UAV, middle: F15 not pointed at

UAV, right: F15 behind AND pointed at UAV

F15

UAV

F15

UAV

F15

UAV

(b)(c)(a)

F15

UAV

F15

UAV

F15

UAV

(b)(c)(a)


UCB PEA Experiment #1 Test Plan:UAV as Evader

• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15

• UAV “wins” by:– Reaching the RVPT– Not being targeted for 20 minutes

• F15 “wins” by targeting the UAV

• Note: F15 performance is restricted


Crew/Pilot Instructions

EventNo.

Event UAV Task F15 Task What to Expect

1 Move to stations Under pilot control, west of SA from INPT between 10,000’ and 12,000’ altitude, heading 90°

In SA, east of W117° 30’ between 18,000’ and 20,000’ altitude

2 Set up UAV Enters SA at INPT under pilot control J-UCAS Mission Plan sets up UAV in hand over

conditions

As above

3 Verification of UAV setup

Monitor Hdg, Alt, Spd commands consistent with handover conditions

As above

4 Start experiment Press CMD-ON button Engage UAV

5 Pursuit-Evasion proceeds

Attempts to reach RVPT while not being targeted by F15

Note: game can be aborted by pressing the “GO EGPT” button

Attempts to target UAV

Interesting maneuvers

6 “Win” condition detected

TD software monitors game conditions and outputs appropriate “win” conditions

As above “WIN: F15 - targeted UAV”“WIN: UAV - reached End Zone!!”“WIN: UAV – time expired”

7 End game or continue playing

Game continues after “win” condition by default to allow experiment to continue

Press GO EGPT to end

As above

8 Return to EGPT Press CMD-OFF button, pilot flies to EGPT Fly to EGPT or remain in SA if further experiments

UAV returning to INPT/EGPT


UCB PEA Experiment #2 Test Plan:UAV as Evader and Pursuer

• UAV attempts to cross Scenario Area (SA) from East to West without being targeted by the F15, however, UAV will attempt to target F15 if suitable conditions arise

• UAV “wins” by:– Reaching the END ZONE– Not being targeted for 20 minutes– Targeting the F15

• F15 “wins” by targeting the UAV

Note: F15 performance is restricted


Demo: Experiment #2 Test Plan

EventNo.

Event UAV Task F15 Task What to Expect

5 Pursuit-Evasion proceeds

Attempts to reach RVPT while not being targeted by F15

GIB can select one of three modes:

UAV as evader by pressing “Evade” button

UAV as pursuer by pressing “Pursue” button

UAV determines whether to evade or pursue by pressing the “Ev/Pu” button

Attempts to target UAV

More interesting maneuvers

Test plan is identical to Experiment #1 except for Event No. 5:


Demo: Experiment #1 Simulation Results

• Using 2nd UAV pretending to be an F15



• Using Fixed Wing Control Interface with simple way point plan to allow Boeing hardware testing



• Using 2nd UAV pretending to be an F15 to allow for controller tuning and software testing


Demo: Flight Test


Demo: Flight Test Results 1


Demo: Flight Test Results 1 (faster)


Demo: Flight Test Results 2


Demo: Flight Test Feedback

• Very positive responses• Pilot comment:

– UAV behaved as he would have expected a trained pilot

• Results still being analyzed for final report


Future: Improving implementation

• Cost functions of the NMPTC can be improved– Check against known fighter tactics, try to duplicate

behavior– Using desktop environment

• Enhancing/verifying our predictive model– Boeing’s model is an executable– We are using a mathematical version

• approximations are necessary• results in loss of accuracy, but increased performance


Conclusions

• MPC can be used to provide interesting behaviors for linear and non-linear control systems

• We hope to reduce the development cycle by at least– Providing a cost-function independent optimizer– Inventing an intuitive interface to generate the cost

function– Developing a method/tool to tune the cost function

for desired behaviors– Experimenting with ways to reverse engineer values

for the matrices, based on desired behaviors under stimuli

• Experience in the SEC program and with OCP has been very positive


Questions?

“Well HAL, I’m damned if I can find anything wrong with it.”“Yes. It’s puzzling. I don’t think I’ve ever seen anything quite like this before.”

-- 2001: A Space Odyssey


Future: NMPTC plans

• Currently implementing a new NMPTC problem using different models and designs

• Will be developing the NMPTC interface to provide the behavior for this new application

• Evaluate the new MATLAB MPC toolbox, to see what benefits it offers


Future: The NMPTC Problem

• Making it work is nice, but– How in the devil did we come up with those

• Equations• Individual components• Matrix values

– Is there a way to derive these from the application constraints?

• Additionally– How hard was it to write a fast optimizer?– Is there a way to make this interface easily usable?


Future: Toward a solution

• System-dependent– Can be derived for a particular system’s

mathematical definition– In general, quite easy to obtain

• Independent– Software engineering exercise– Once defined, will be reused

• Behavior-dependent– By far the hardest piece of the solution– Not generally derivable, but there are tricks

that should be available for all future implementers, that a parameterized approach can provide


And of course, NASA...

• How many of you spent at least 24 hours per day for the past 3 weeks working on H&RT Proposals? – Can I get an ‘amen’?

• Boeing-led– Open Robotic Space Control Platform (ORSCP)– for transitioning OCP to space exploration:– Co-Lead is UCB (Shankar Sastry)

• One of two SEC Capstone Demo participants involved

• Berkeley-led– Infrastructure & Architecture for Crew-centered

Operations (IACO)– for deployment of mission-critical decisions to

spacecraft and autonomous robots, rather than mission control


Demo: Simulation Results 1


Demo: Simulation Results 2

Documents

Goddard Space Flight Center 6 October 2004 Pursuit/Evasion of Fixed-wing Aircraft through Model-Predictive Control The new album by Jonathan Sprinkle Featuring,