Simulating Cooperative Control Algorithms Using

MATLAB, Simulink, and AMASE�

Zachry H. Basnighty, Steven J. Rasmussenz, Alex N. Starrx,

Matthew Duquette{, Krishnamoorthy Kalyanamk

Control Science Center of Excellence, Wright-Patterson AFB, OH, 45433, USA

The ability to use a realistic simulation to test cooperative control algorithms for UAVsprovides researchers a better understanding of the e�ectiveness of their algorithms. TheAMASE simulation provides this realistic environment to researchers in a simple and prac-tical package that remains exible to many users needs by utilizing a TCP/IP interface.With MATLAB in common use today among researchers and engineers, it is instinctiveto desire a simple method of combining the power and ubiquity AMASE o�ers with thefamiliarity of MATLAB. The MATLAB-AMASE Toolbox was created to ful�ll this need.With a simple interface requiring hardly more than a researcher’s bare algorithm, as wellas advancing integration with Simulink, the MATLAB-AMASE Toolbox provides the op-portunity to test cooperative control algorithms for UAVs in a realistic environment whilesaving time and e�ort on researchers’ behalf. The MATLAB-AMASE Toolbox has al-ready been applied to the Perimeter Patrol Problem successfully and a�orded a realisticperspective on algorithm performance with minimal e�ort.

I. Introduction

A signi�cant portion of current air vehicle technology research is focused on the design and utilizationof multiple and potentially diverse unmanned vehicles working together as a coordinated team. Such acapability requires cooperative control algorithms and associated engineering implementations that do notyet exist at a robust level of �delity. Several research organizations across academia, government, andindustry are actively working to develop potential solutions to this technology challenge; however they eachtypically employ an independent simulation capability to develop, evaluate, and re�ne such solutions.

The Air Vehicles Technology Assessment and Simulation (AVTAS) branch’s Multi-Agent SimulationEnvironment (AMASE)1 was constructed to provide a common framework for research and developmentof cooperative control algorithms. AMASE has been adopted by researchers at the Air Force ResearchLaboratory (AFRL) to develop, implement, and evaluate cooperative algorithms in-house, as well as eval-uate algorithms developed external to AFRL. In order to make AMASE more accessible to researchers, anAMASE toolbox for MATLAB has been constructed. This toolbox enables researchers to utilize the AMASEsimulation framework directly from MATLAB and Simulink without programming in any other language.

This paper describes design and implementation of the MATLAB-AMASE Toolbox. The next sectionpresents an introduction to the AMASE simulation framework. Section III describes development andimplementation of the MATLAB-AMASE Toolbox. Finally, Section IV introduces an alarm processingalgorithm and describes how it was implemented and simulated using connections from MATLAB to AMASEusing the MATLAB-AMASE Toolbox.�Approved for public release; distribution is unlimited. Case Number: 88ABW-2010-3713y2d Lt, USAF, UAV Control Engineer, AFRL/RBCA, 2210 8th St., WPAFB, OH 45433, AIAA Member.zPrincipal Engineer, Miami Valley Aerospace LLC, 2210 8th St., WPAFB, OH 45433, AIAA Member. Corresponding author:

steven.rasmussen@wpafb.af.mil.xStudent (Aerospace Engineer), University of Cincinnati, 2600 Clifton Ave., Cincinnati, OH 45221.{Aerospace Engineer, AFRL/RBCD, 2130 8th St., WPAFB, OH 45433, AIAA Senior Member.kNRC Research Associate, AFRL/BRCA, 2210 8th St., WPAFB, OH 45433, AIAA Member. This research was performed

while the author K. Kalyanam held a National Research Council Research Associateship Award at the Air Force ResearchLaboratory.

1 of 12

American Institute of Aeronautics and Astronautics

AIAA Modeling and Simulation Technologies Conference2 - 5 August 2010, Toronto, Ontario Canada

AIAA 2010-8359

This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

II. The AMASE Simulation Framework

In order to perform research and development of systems that enable coordinated teams of multipleand potentially diverse unmanned vehicles, researchers at AFRL collaborate among various academic, gov-ernment, and industry researchers. Since each organization typically employs an independent simulationcapability to develop, evaluate, and re�ne such systems, those simulation capabilities are often very di�erentfrom each other. These inherent inconsistencies make collaboration di�cult. Furthermore, the evaluationof potential systems on a level playing �eld is also di�cult and sometimes impossible. The AMASE sim-ulation framework was developed out of this need. It not only establishes a level playing �eld, but alsoprovides a common framework for independent teams to design to without fear of exposing the details oftheir proprietary technologies.

Simulation(Snapshot,

Constructive,Virtual)

Net

wor

k In

terf

ace

ScenarioSetupTool

XMLScenario

File

XML-based input standard allows anyone

to develop tools for data analysis and

preparation.

Terrain

Culture

Simulation(Snapshot,

Constructive,Virtual)

ScenarioSetupTool

ExternalApplication

ExternalApplication

Figure 1. A complete simulation framework was developed, including models, input �les, network interfaces, andhuman interfaces.

II.A. Protocol and Message Sets

In order to enable rapid development by a number of organizations working in parallel, the simulationcapability needed to be easily adopted. A number of existing tools and processes were considered. All wereeither too complex or too costly for projects with short schedules or narrow budgets. A customized solutionwas determined to be the best approach. A well de�ned interface between components was paramount tothe success of the simulation-based programs. A network-based interface system was developed in order toenable the use of several computer platforms and languages in one simulation. This protocol, labeled theLightweight Message Construction Protocol (LMCP), enables engineers to develop high-level descriptions ofdata needed to communicate between parties involved in the simulation. AFRL researchers used LMCP tocreate a completely open and publicly available message set intended to communicate between UAVs, controlstations, simulations, and control algorithms, the Common Mission Automation Services Interface (CMASI).While LMCP speci�es the lower level methods of how LMCP messages should be sent over TCP/IP, CMASIis an implemented set of basic LMCP messages that are required to communicate between UAV systems.CMASI is now the standard message set used in all such newer applications in the Air Vehicles Directorateand, thus, provides the basis of the AMASE communication mechanism.

The CMASI message set (also called an LMCP series) is detailed by an XML document, which describesthe messages that can be sent and what information each message should contain. An automatic codegenerator is used to take the series description and produce a set of source code libraries that implementsthe series in multiple programming languages (including Java, C++, C#, and Python). Having the messageset implemented in multiple languages allows simple integration with other software. Transmitting and

2 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

receiving these objects over a network connection, or by reading and writing them to a �le, allows varioussoftware tools to interact on a common level. Additionally, a set of formatted documentation is also producedby the auto-coding process, allowing engineers to review and implement the interface.

AMASE is not simply limited to using only the CMASI series, however. Since LMCP is the methodused to communicate, AMASE will accept any LMCP message. While only CMASI is guaranteed to beunderstood by AMASE, any LMCP series will still be heard. One could easily create a custom build ofAMASE to act on their own LMCP messages and the auto-coder will provide them a simple method tocreate that custom LMCP series.

II.B. The Framework

The simulation framework, see Figure 1, uses a number of modern tools and techniques. It is entirely writtenin Java, a modern programming language that has seen increased performance and popularity this decade.Data is managed using XML interfaces. Open-source libraries are employed to enable rapid, unrestrictedaccess to basic functions such as world geodesy and image manipulation. These techniques enable morerapid development of full-featured applications and leverage a wealth of knowledge in many disciplines.

Several new tools were developed and new techniques applied in the process of creating the UAV simula-tion. The core of the simulation is the modeling of multiple UAVs in an ISR mission context. The simulationtransmits mission-level data to connected applications, and accepts commands for each of the UAVs in thescenario based on the results of cooperative control algorithms. Figure 2 shows a screenshot of the simulationuser interface.

Figure 2. The simulation features a number of human-interface tools including route display, UAV status, tasks andconstraints, map imagery, and data manipulation.

Quantitative analysis is critical in the evaluation of cooperative control systems so the simulation featuresa number of analysis modules, reporting conditions such as UAVs violating ight boundaries and no- yzones, or the e�ectiveness of a search tasks. These analysis tools are used by engineers to evaluate and re�necooperative control algorithms.

What makes the AMASE simulation framework di�erent is its open and modi�able nature. The toolsand interface are government-owned. Since the simulation and support tools are Java-based, a complexapplication set was developed more rapidly than previously seen with legacy computer languages. Moderntools and techniques such as XML data exchange and network-based object exchange leverage a wealth ofknowledge in the open-source software community. The implementation of LMCP was far less complex thanother communications methods such as HLA, while allowing a high-degree of customization.

The LMCP data exchange development process also revealed a new way of doing business between sim-ulation partners: The development of interfaces served to govern the scope of the projects. By leading the

3 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

interface development process, government engineers were able to work with multiple contractor organiza-tions simultaneously. By de�ning a network-based interface, organizations were able to develop using theirpreferred tools, techniques, and platforms and communicate with the simulation through a standard TCP/IPinterface.

III. The MATLAB-AMASE Toolbox

In the research and engineering community, MATLAB remains a powerful data manipulation and pro-gramming tool used by many to aid research and development e�orts. With the use of MATLAB beingso common, it seems natural that a large portion of researchers developing cooperative control algorithmsfor UAVs do so using MATLAB to test and simulate their algorithms. While MATLAB is a powerful tool,it quickly becomes a complex problem to simulate an environment for a cooperative control algorithm tooperate. As previously mentioned, this is a main reason the AMASE simulation was created. In order tosave time and e�ort of researchers developing cooperative control algorithms, AMASE allows for integrationof cooperative control algorithms into a more realistic simulation environment than simple comparison oftheoretical data could provide, while not requiring a custom solution for each algorithm. Although AMASEprovides the ability to integrate in such a way using LMCP over TCP/IP, the integration of MATLABalgorithms with LMCP can still be simpli�ed. For this reason, the MATLAB-AMASE Toolbox was created.

The purpose of the MATLAB-AMASE Toolbox (MAT) is to create a simple method of interfacing theAMASE simulation through MATLAB. The provided MAT infrastructure provides two-way communicationwith AMASE, creating a myriad of possible ways to interact with and a�ect AMASE. MAT works as amiddle-man between AMASE and a researcher’s cooperative control algorithm, keeping track of all thesimulation data that de�nes the current state of the simulation as well as methods to send commands backto the simulation based on the cooperative control algorithm output. MAT’s two-way communication enablesthe cooperative control algorithm to access simulation data in order to make an informed decision on whatthe simulation must do, or how it should change, and take appropriate action to accomplish the change.While MAT provides the infrastructure to make decisions, it is up to the user to provide the methodologybehind those decisions, in the form of a module. Essentially, MAT manages user-created modules that aremeant to interact with AMASE. The user creates a module by programming a function in MATLAB designedto respond to MAT simulation data with corresponding actions as deemed necessary.

III.A. MATLAB-AMASE Toolbox Description

As previously mentioned, MAT acts as a middle-man by transferring and translating data and communicatingbetween the AMASE simulation and any number of user-created modules intended to interact with AMASE.Figure 3 gives a visual representation of the relationship between AMASE, MAT, and the user modules. MATconnects to AMASE and, in turn, supports an arbitrary number of user modules.

AMASE MAT

. . .

Module 2

Module 1 Java

Java/MATLAB MATLAB

Figure 3. The MATLAB-AMASE Toolbox provides a familiar MATLAB interface to the exible AMASE simulation.

The circle representing MAT in Figure 3 can be broken down further into three major components: theLMCP Client, the LMCP Processor, and the MATLAB-AMASE Manager (MAM or the Manager). Asdepicted in Figure 4, LMCP objects (messages) are sent from the AMASE simulation server over TCP/IPand intercepted by the LMCP Client Listener. The Client Listener adds each LMCP object received to

4 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

a bu�er of messages. This bu�er is then used by the LMCP Processor, which takes each LMCP messagefrom the bu�er and updates the simData MATLAB structure to appropriately represent the current stateof the entire AMASE simulation. Finally, the updated simData structure is passed to all active modules bythe MATLAB-AMASE Manager. Using the most current data, the modules are able to perform their user-appropriated functions, returning any necessary MissionCommands (or any other LMCP messages) back tothe Manager. Subsequently, the Manager sends new MissionCommands back to the AMASE server via theLMCP Processor and Client Sender.

Stop

Run

Initialize

LMCPClient ClientListener

ClientSender

Message Processor Manager

Module 1

Module 2

. . .

LMCP

LMCP

Buffer simData

MissionCommands

Java Java to MATLAB MATLAB

AMASE MAT

. . .

Module 2

Module 1 Java

Java/MATLAB MATLAB

Figure 4. The MATLAB-AMASE Toolbox consists of three main components: the LMCP Client, the LMCP Processor,and the MATLAB-AMASE Manager. Each module consists of three main functions: Initialize, Run, and Stop.

III.B. The LMCP Client

The LMCP Client is implemented as a Java class. By using MATLAB’s ability to import Java classes, MATcan instantiate an LMCP Client in MATLAB, allowing connectivity to the AMASE simulation server. TheLMCP Client, once created, serves two main functions: receiving and sending messages from the AMASEserver. The Client Listener is implemented as a Java thread that runs in the background listening for newmessages from the server and adds them to the message bu�er once received. The Client Sender simplypackages the messages received from modules and sends them back to the AMASE server as LMCP objects.

III.C. The LMCP Processor

The LMCP Processor is a MATLAB class that actually instantiates the Java LMCP Client. The LMCPProcessor translates AMASE LMCP Objects received from its LMCP Client’s message bu�er into a MATLABstructure to be used by MAT modules. This simData structure contains �elds describing every aspect of theAMASE simulation required to represent the current state of the simulation including UAV con�gurations,current UAV states, lists of no y zones, search tasks, targets, plan requests, and the current simulation timeand status. Each di�erent type of LMCP object has a unique identi�cation code. The LMCP Processor willcheck this LMCP ID code and update the appropriate simData �eld to accurately represent the current stateof the simulation. For example, for every time step of the simulation, the server sends the simulation statusand the current states of the UAVs being simulated. The LMCP Processor will take these updated statesand use them to replace the previous UAV states contained in simData. In this way, the LMCP Processor

5 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

keeps simData updated every cycle.

III.D. The MATLAB-AMASE Manager

The MATLAB-AMASE Manager is the core of the MATLAB-AMASE Toolbox. MAM is the function thatactually executes and manages MAT modules. A basic description of the ow of MAM is illustrated inFigure 5. To begin running any MAT modules, a con�guration �le is passed to MAM. The con�guration �leis contained in an XML format in order to further promote simplicity and cohesion between the user and theentire MATLAB-AMASE environment. The XML con�guration contains two lists: a list of network socketsto be used for communicating and a list of module speci�cations that are to be run.

III.D.1. LMCP Processor and Socket Initialization

Although Figure 4 shows only one LMCP Client and LMCP Processor, it is possible to have multiple instancesof these running simultaneously in the Manager. MAT in fact supports the ability to connect to multiplenetwork sockets using LMCP, each of which requires its own LMCP Processor with a private LMCP Client.The socket list speci�ed in the con�guration �le identi�es a unique socket ID number for each socket, the IPaddress and port number that socket should operate on, and a Mode value denoting the method that socket’sLMCP Processor should use to initialize simData. Three possible values exist for the Mode: 0 indicates theLMCP Processor should initialized simData from an external XML �le (likely the same scenario �le AMASEused to initialize), 1 indicates the LMCP Processor should wait for AMASE to reset and resend the initialsimulation data across the socket, and 2 indicates the LMCP Processor should skip initializing simData. InMode 2, simData is populated as the LMCP Processor receives messages in an impromptu manner and canbe useful for circumstances where the modules on that socket do not need the initial simulation data or forconnecting to other LMCP-based entities. When MAM is �rst started, the con�guration XML is �rst parsedand the socket and module information is extracted. Using this information, an LMCP Processor is createdand initialized for each speci�ed socket, including starting the LMCP Client Listeners to begin listening forLMCP objects on their socket.

III.D.2. Module Initialization

For each module to be loaded, the con�guration speci�es six required �elds: the name of the module, thesocket it will be attached to, the �le path to the module’s code, and name of the required initialization, run,and stop functions. In addition there are two optional �elds specifying a GUI create function if the user’smodule requires a GUI and the path to a module-speci�c con�guration �le. Following the LMCP Processorand socket initializations, each module’s initialization function is called. The initialization function is passedthe initial simData from its associated LMCP Processor and the module’s speci�c con�guration �le, resultingin that module’s initialized data and any initial LMCP Objects to be passed back to MAM and stored orsent, respectively. Afterward, for modules specifying a GUI create function, a GUI will be created using thatfunction and MAM will store the handle to the GUI. In the case that any LMCP Processor did not initializecorrectly, any modules assigned to that LMCP Processor will not be initialized or set as running.

III.D.3. The Main Manager Process

At this point, everything is initialized and the main loop of the Manager begins. For each module that iscurrently running, the main run function is called using the module data stored by MAM. Once the moduleperforms its intended function, control is returned to MAM as well as the updated module data and anyLMCP objects the module wishes to send. The Manager will store the updated module data and send anyLMCP objects received. If any command was given to stop the module or the module GUI was closed,MAM will mark that module as stopped and call that module’s stop function. The Manager will continueprocessing new messages from the AMASE server and running modules until the Manager GUI is exited, atwhich point MAM will stop all running modules, LMCP Processors, close all sockets and exit.

6 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

Config Info

Message List

Start Clients

Process New Messages

Extract Config Info

Initialize Modules

Run Module Routines

Close Modules

Stop Clients

GUI Closed?

MAT Config XML

Message List

Module/Sim Data

No No

Yes

Module/Sim Data

Message List

Initialize Sockets

Config Info

Figure 5. The MATLAB-AMASE Manager is ultimately responsible for initializing and running modules, as well asmanaging data to and from the AMASE server.

III.E. Applying the MATLAB-AMASE Toolbox with Modules

With MAT, the work of interfacing a MATLAB cooperative control algorithm with AMASE has been reducedto simply creating a MAT module. In order for one to write their own module, three basic components arerequired: an initialization function, a run function, and a stop function. The initialization function uses theinitial simData and an optional module-speci�c con�guration �le to initialize run function data. MAM passessimData and the con�guration �le to the initialization function and expects in return the initial module dataalong with any initial LMCP objects to be sent. Following this, the run function will contain the user codespecifying the main actions of the module. In the case of a cooperative control algorithm application, thisis most likely where the algorithm itself will exist or be called from. MAM passes this function the currentmodule data as well as the most current simData. The run function will return an updated version of itsinternal data structure to be used by the next iteration of the run function. Again, any LMCP objects willalso be communicated back to MAM. Finally, the stop function is called when the user stops the modulefrom the MAM Module GUI and should specify any clean-up actions that should be taken for the moduleonce stopped, including closing the GUI if one was created. In the most simple of cooperative controlapplications, the initialization and stop functions could be empty, with only the run function checking if anew PlanRequest has been sent, executing a cooperative control algorithm, and responding with appropriateMissionCommands.

While implementing a module is certainly the simplest way to take advantage of MAT, it is certainlynot the only way. MAM provides the infrastructure to run modules, but aside from MAM, there still existsthe LMCP Client and the LMCP Processor which on their own also allow the user to program a morecustomized solution in instances where the module framework provided may prove insu�cient. By usingthe LMCP Client and LMCP Processor alone, the user would be enabled to access simulation informationdirectly, while still a�orded the advantage of a simple pre-constructed interface.

7 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

III.F. Integrating the MATLAB-AMASE Toolbox with Simulink

The Simulink environment for MATLAB provides a more graphical interface for researchers to model and testalgorithms. With its block-style model programming, Simulink creates an intuitive development environmentfor users, especially those involved in control and signal theory. It is intended that the MATLAB-AMASEToolbox will eventually be expanded to fully implement a user-friendly Simulink addition to include blocksthat will serve similar functions to the current toolbox. Currently, a proof of concept Simulink model hasbeen developed that allows an arbitrary number of user-created S-Function modules to be placed in parallelin the model (See Figure 6). This solution is implemented using recycled code from the non-Simulink versionof MAT and, hence, follows the same basic ow. The initialization is set as having a constant sampling timeto ensure only one occurrence. At each sampling time, the main MAM process is iterated. The modules arerun and messages are sent consequently. By checking the existence of the MAM GUI handle, the simulationcan determine once the MAM GUI is closed and subsequently stop executing. While this implementationdoes provide an e�ective Simulink solution for MAT, it also relies more heavily on accessing GUI applicationdata than passing data as signals. With future development, however, a more robust and accommodatingsolution will arise. Following this, the ability to use Simulink in addition to the current MATLAB-AMASEToolbox will make the task of interfacing custom algorithms with AMASE virtually seamless for those whorely on MATLAB and Simulink to develop their work.

Figure 6. An initial model of the MATLAB-AMASE Toolbox has been implemented in Simulink, furthering MAT’s exibility and potential for creative convenience.

IV. Sample Application: Perimeter Patrol

One application of interest is integrated base defense. A vital part of integrated base defense is patrollinga perimeter and responding to alarms. Researchers at AFRL have developed algorithms to help automateUAV perimeter patrol. In order to move these algorithms towards implementation in a system for ighttest, they were simulated using MATLAB and the AMASE framework. The MATLAB-AMASE Toolboxwas used to facilitate the connection between AMASE and MATLAB. The next few sections introducethe perimeter patrol problem, describe the process of implementing perimeter patrol algorithms with theMATLAB-AMASE Toolbox, and then conclude with a description of the lessons learned from simulatingthe algorithms in this manner.

8 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

IV.A. The Perimeter Patrol Problem2

In a perimeter patrol scenario, UAVs are tasked to navigate around a given perimeter, while keeping theirsensors trained on that perimeter. At given locations along the perimeter, there are alarm stations. Thealarms are generated from an Unattended Ground Sensor (UGS) intended to detect incursions. When analarm is tripped, a UAV should respond to determine what caused the alarm. The UAV will orbit the UGSstation while the operator steers the gimballed camera looking for the source of the alarm. If an incursionis detected, the operator can assign other assets for interception.

The key question is how long the UAV should loiter before concluding that the alert was a false positiveand resume its patrol. This is the question the stochastic control optimization is intended to address. Oneapproach is to have the UAV y a patrol that passes over each of the UGS sites, or stations, at regularintervals. The objective is to reduce the expected response time once an alert occurs by being continuouslyon patrol, rather than launching on demand from a �xed site.

Initial e�orts considered a maximum response time Quality of Service (QoS) metric. The idea was toinclude this as a hard constraint in the optimization. This approach was considered intractable, especiallyif the alert queues could expand. An expected or average response time was investigated, but could notbe enforced as a constraint. Finally, the approach pursued was a weight between the expected informationgained while loitering versus the expected wait time of alerts in the queue.

The alert rate is a key factor in the problem. In general, while the probability of an actual incursion canbe quite remote, the nuisance trips are generally much larger and dominate the problem. In other words,signi�cant assets can be utilized in determining that an alert is a nuisance trip. Nuisance trips are the normalmode of the perimeter patrol. Servicing them as quickly as possible preserves assets.

The previous custom simulation used to test this algorithm (See Figure 7) was implemented simply usinga graph in MATLAB and rewriting points on the graph as needed to simulate a UAV moving around theperimeter and visiting alarm stations as required and dictated by the algorithm.

Figure 7. The original graphical representation of the Perimeter Patrol Problem algorithm simulation.

IV.B. Algorithm Implementation as a Module

In order to create a module for the perimeter patrol algorithm, the same steps as mentioned in SectionIII.E are followed. First, an initialization function is created. The perimeter patrol algorithm relies on theperimeter distances between alarms and the expected rate at which alarms will sound in order to calculate the

9 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

optimal action given a certain state. The initialization function for the module will calculate the distancesand take the predicted alarm rate, then use these two components to initialize the algorithm. In order to beable to test the e�ectiveness of the perimeter patrol algorithm, there must be a way to monitor the currentstates of the alarms and delays in servicing times. Therefore, this module will specify a GUI creation functionto provide a display of the alarm statuses and data. With no initial LMCP Objects needing to be sent, onlythe initialized algorithm data will be returned.

Next, the run function will be created. The current simData will be used to determine if any UAVspatrolling the perimeter have reached an alarm. If they have, a sub-function containing the actual algorithmwill be called to determine how long the UAV should loiter at that alarm, if at all. Once that is determined,a new MissionCommand will be created describing the new action and will be sent back to the Manager totell the server. At the same time, the GUI will also be updated with the current status of alarms and delaytimes. Finally, the stop function will simply close the alarm status GUI.

With the MAT module created, the only requirement left is to create both the XML �les required toinitialize AMASE and MAT. The AMASE XML �le will contain the UAV con�gurations and the MAT XMLwill contain the socket and module information as previously stated in Section III.D.

IV.C. Lessons Learned

In implementing the patrol algorithm their were a few di�culties encountered and many lessons learned thathave improved the functionality and ease of use of MAT. The �rst problem met was the fact that there wasno simple way to indicate to AMASE what a perimeter patrol task should look like. This is not a basic taskincluded in CMASI. The original solution to this problem was to specify to both AMASE and MAT thewaypoints that would constitute the perimeter. While this solution worked, the fact that the perimeter mustbeen known prior to beginning the simulation removed the exibility to be able to add a new perimeter patroltask during the simulation. This could be considered an acceptable loss if the researcher using MAT was onlyinterested in testing a specialized scenario (which was initially the case), but this would be unacceptable forextending the usefulness of the patrol algorithm by integrating it with other scenarios. The goal was to beable to give a UAV, in the middle of a larger scenario, a perimeter patrol task and have the UAV awlesslyget directed to the new task.

The �rst and most apparent solution to this problem would be to use the LMCP auto-coder to producea custom series for the perimeter patrol problem, including a Patrol Task and special commands to signalwhen alarms have been sounded, as mentioned in Section II.A. Although it would be possible and relativelysimple for someone with knowledge of Java to make an amendment to the AMASE code, as also mentionedin Section II.A, this would entirely con ict with the principle goal of making MAT easy for someone withknowledge only of MATLAB to integrate into their work.

For this application, a custom LMCP series was eventually created to simplify the communication processwith the patrol algorithm. However, in attempting to avoid the necessity of requiring the user to modifyAMASE code, a simple translator was instead implemented for the patrol module. The module was toldto simply assume any Line Search Task received from AMASE was actually intended to be a Patrol Task.Using this method it was now possible for the simulated UAVs to receive and act on Patrol Tasks duringan active scenario. While not the ideal solution, this method was e�ective and made for a relatively simpleaddition to the module.

Below, in Figure 8, is a screenshot of the MAT implementation of the perimeter patrol algorithm. Despitethese challenges in implementing the patrol module, the advantage of MAT in its ability to interface withAMASE has become clear. The creation of the MAT module now allows for a more realistic environmentto test the algorithm, resulting in more realistic measurments of average and worst-case delay in servicingthan was previously available using only a simple MATLAB-based simulation. The resulting data showedthe di�erence between a simple testing method depending on many simplifying assumptions and a realisticsimulation that takes into account realistic aircraft physics and abilities. AMASE displayed just how unpre-dictable some aspects of real-world simulations can be. While there may always be an optimal solution to agiven problem, that solution is only guaranteed optimal under certain conditions which may easily changewhen applied to the real world.

10 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

Figure 8. The new graphical representation of the Perimeter Patrol Problem algorithm simulation using a MAT module.

The obstacles mentioned above as well as the advantages a�orded by being able to interface with AMASEhave brought to light some issues worth addressing. While AMASE is incredibly capable, and the ability ofusing MAT to interface with AMASE is valuable, some particular implementations may prove more di�cultthan others. Speci�cally, a module that must rely on a custom LMCP series will not have the ability tonatively interact with AMASE in the way one would hope. By sticking with the broad range of CMASImessages available and supported by AMASE, one will be most able to interact fully with the simulation.However, the underlying concepts of research are to expand to new areas and push the envelope of availabletechnology. Therefore, relying simply on what is currently available is often unacceptable. In order to allowresearchers to ful�ll their duties and excel, AMASE must be able keep up with their needs. AMASE mustcontinue to grow its ever-expanding set of capabilities to support a continually broader range of messages.While it is impractical to be able to predict and implement every user’s need in their particular area, bystarting from a solid base, which CMASI provides, and using foresight to constantly make additions ofmessages that will provide the most use to future researchers employing this technology, AMASE will striveto provide the best possible utility in designing and testing cooperative control algorithms. Accordingly, MATwill keep pace with AMASE and further enable MATLAB users a convenient method to take advantage ofAMASE.

V. The MATLAB-AMASE Toolbox Beyond AMASE and CooperativeControl Algorithms

Although a good example implementation has been shown, the MATLAB-AMASE Toolbox has thepotential to a�ect AMASE in many di�erent aspects aside from the implementation of simulating cooperativecontrol algorithms. Examples include everything from a monitor module, which simply takes simulation dataand displays UAV state information mimicking the AMASE console itself, to a exible message sender modulethat provides a simple GUI to modify and send any LMCP object speci�ed, to a full- edged control module,which could allow changes to every aspect of the simulation in real time. The former two of these havealready been implemented and the latter is a possible future feature, but regardless, this demonstrates thetrue exibility and simplicity MAT provides.

The main focus of MAT is to support using AMASE in the implementation and testing of cooperativecontrol algorithms, but with LMCP being the new standard communication protocol, and CMASI the

11 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359

standard message set, there are a multitude of possibilities for MAT. In fact, with LMCP, MAT is able tointerface with any other system also using LMCP. Following the sample application of the perimeter patrolalgorithm discussed in Section IV, a C++ implementation of the algorithm was created in order to be ighttested at Vandenberg Air Force Base, CA. While AMASE provides an e�ective and open environment forsimulation testing, AFRL uses a separate software package for applying new algorithms in a ight test:Vigilant Spirit Command Center.3 Vigilant Spirit now uses the very same CMASI message set as AMASE,but instead of simulating aircraft dynamics, connects to real aircraft in ight. Unfortunately, the C++implementation of the patrol algorithm su�ered from a similar fate as the MAT implementation: VigilantSpirit had no way of giving the algorithm a Patrol Task since it had no knowledge of Patrol Tasks.

To solve this problem, the MATLAB-AMASE Toolbox was called upon to provide support. The patrolalgorithm needed a controller to give it Patrol Tasks, Plan Requests to execute those tasks, and Alarm Signalsfor when an alarm had been sounded on the perimeter. Using MAT, an initial Message Sender module wascreated in one day that provided the ability to send any LMCP message that the user speci�es in the modulecon�guration. After testing and improving the module over the course of about a week, the module andMAT were being used in a ight test providing a method to control the perimeter patrol algorithm throughLMCP. Having the ability to communicate via LMCP allows MAT to extend beyond just the intended realmof cooperative control algorithm development and provide a truly exible software package for UAV research.

VI. Conclusion

In the paradigm of cooperative control algorithms for UAVs, the ability to test one’s research is at leastas important as the research itself. Traditionally, most researchers in this �eld would have been forced eitherto use a simple test simulation that would most likely produce less-than-realistic results compared to a costlyreal-world implementation and testing, or spend an excessive amount of time and e�ort to create a realisticsimulation that would become unfeasible to use for other applications. With the creation and adoptionof AMASE, this problem is assuaged. AMASE provides a realistic simulation environment for researchersto test their cooperative control algorithms, saving time and e�ort to be put back into research. WhileAMASE has the convenience of a TCP/IP interface via LMCP, lessening the e�ort required by researchersto test their developments will always be advantageous. For this reason, the MATLAB-AMASE Toolbox wascreated as a bridge between the powerful AMASE simulation and the MATLAB development environmentfamiliar to many researchers. Using MAT, researchers will be able to quickly create modules to interfacetheir algorithms with AMASE using minimal e�ort. MAT will allow researchers to develop cooperativecontrol algorithms using a familiar environment while still providing access to AMASE and without beingrequired to learn another programming language like Java or C++. The plurality of MAT’s bene�ts becomesapparent when considering that as AMASE provides for a more realistic simulation for testing than timewould likely permit one to create initially, time is also saved by not having to create a custom simulation,or even an interface to AMASE for that matter, allowing more time to be spent on actual development ofa quality algorithm instead of infrastructure to test it. To test its e�ectiveness, MAT was applied to theperimeter patrol problem. A MAT module was created for the perimeter patrol algorithm and proved toe�ectively test the algorithm with encouraging results. Furthermore, the scalability of MAT using LMCPcontinues to open new opportunities to prove its e�ectiveness and convenience to MATLAB and Simulinkprogrammers. The MATLAB-AMASE Toolbox has successfully ful�lled its objectives and will a�ord futureresearch endeavors into cooperative control algorithms the opportunities presented here.

References

1Duquette, M., \E�ects-Level Models for UAV Simulation," Proceedings of the AIAA Modeling and Simulation TechnologiesConference, Chicago, IL, 2009.

2K. Krishnamoorthy, M. Pachter, S. D. and Chandler, P., \Approximate Dynamic Programming with State AggregationApplied to Perimeter Patrol," Proceedings of the AIAA Guidance, Navigation, and Control Conference, Toronto, 2010.

3Feitshans, G. L., Rowe, A. J., Davis, J. E., Holland, M. D., and Berger, L. R., \Vigilant Spirit Control Station (VSCS):The Face of COUNTER," Proceedings of the AIAA Guidance, Navigation, and Control Conference, Honolulu, HI, 2008.

12 of 12

American Institute of Aeronautics and Astronautics

Dow

nloa

ded

by P

EN

NSY

LV

AN

IA S

TA

TE

UN

IVE

RSI

TY

on

Sept

embe

r 9,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/6.2

010-

8359