Controlling and Configuring Large UAV Teams Paul Scerri, Yang Xu, Jumpol Polvichai, Katia Sycara and Mike Lewis Carnegie Mellon University and University

Controlling and Configuring Large UAV Teams

Paul Scerri, Yang Xu, Jumpol Polvichai, Katia Sycara and Mike Lewis

Carnegie Mellon University and University of Pittsburgh

Context• Aim to build large heterogeneous teams for

complex tasks– Robots, agents, people– 10,000s of actors

• Multiagent version of Belief Desire Intention approach to autonomous behavior– Builds on key abstraction of a Team Oriented Plan

• Defines the activities that must take place and interactions between those activities

– Supported by extensive theoretical (logical) work• Key algorithms are NP-complete or worse

– Heuristics required for scalability

WWWAgent ProxiesFor People

InformationAgents

Ontology-basedMatchmakers

Specific Target Application: Wide Area Search Munitions (WASMs)

• Part munition, part unmanned aerial vehicle– Single use– Variety of sensors– Limited fuel supply, approximately 30 minutes– Communicate with each other, manned aircraft

• Concept of Operations (under development)– Small number of manned aircraft – Potentially other ground forces– 100s of WASMs, performing a variety of missions

• Attack• Search• Battle damage assessment • Decoys• Communication relays

• Flight test planned September, 2005– 1 real and 3 simulated WASMs

Token-Based Coordination• Token: self contained packet capable of being sent around

team– Information content– Control content

• Local models– Team members use receipt of tokens to create local models of

other team members• What sorts of things are they/are they not working on?• What sorts of things might they need to know?

– Local models are used to improve the routing of future tokens

• Token “flows” implement coordination• No brittle, non-scalable “message protocols”

Token Based Algorithms

• Plan instantiation

• Removal of duplicate plans

• Role allocation

• Information sharing

• Resource allocation

• Sensor fusion

• Recovering from faulty sensor readings

Token Based Algorithms• Plan instantiation• Removal of duplicate

plans• Role allocation• Information sharing• Resource allocation• Sensor fusion• Recovering from

faulty sensor readings

Information about environment passed around in tokens, when agent receives tokens matching plan pre-conditions plan is initiated

Very high probability all applicable plans are initiated

Liao et al, 2004

0 200 400 600 800 10000

1000

2000

3000

4000

5000

6000

7000

8000

Number of Team Members

Con

flict

Res

olut

ion

Mes

sage

s

AlwaysLocalRandom

Token Based AlgorithmsInformation about initiated plans shared in tokens

Very high probability some agent gets to find out about any duplicate plans

Liao et al, 2004

• Plan instantiation• Removal of duplicate



5101520

010

2030

0

0.2

0.4

0.6

0.8

1

Size of Subteam ASize of Subteam B

Pro

babi

lity

Token Based AlgorithmsResponsibility to perform a role encapsulated in token

Only team member with token can perform role

Team member must have capability > threshold to perform role

Threshold calculated from estimates of likely role allocation outcome

Okamoto, 2004




1

10

100

1000

10000

100000

1000000

10000000

10 120

230

340

450

560

670

780

890

1000

1110

1220

1330

problem size (#agents = #roles)

com

mu

nic

atio

n p

er r

ole

(m

essa

ges

)

DSA

LA-DCOP, 0.0 threshold

LA-DCOP, 0.5 threshold

Greedy

Token Based AlgorithmsLocally sensed information shared via token to get information to team members performing role effected by information

Does not require sensing agent to know who needs information

Xu et al, 2004




0

10

20

30

40

50

60

70

80

90

Mes

sage

s

Random Small World Grid Scale Free

Network type

Without efficiency enhanced Algorith With efficiency enhanced Algorith

Token Based AlgorithmsAccess to resource represented by token

Only team member with token can use resource

Team member must have need > threshold to keep resource

Threshold changes dynamically as it moves around the team, seeing resource need




Token Based AlgorithmsUncertain sensor readings encapsulated in tokens and sent around the team

Very high probability that at least one team member gets related sensor readings and fuses for higher confidence




Token Based AlgorithmsAssumptions used to make decisions are attached to tokens resulting from that decision

Very high probability agents with contradictory information see the assumptions and initiate checking process




Layered Team Member Architecture• Bottom: Local Reasoning Team

member’s local actions are restricted by the tokens they have– E.g., without resource token, cannot use

resource• Middle: Coordination Reasoning

Movement of tokens around the team implements the coordination– Flows of tokens– Local models inform token routing

• Top: Meta-reasoning Ensures token flows work effectively

Token

Local

RoutingModel

Meta

Synergies Between Token Algorithms

• Overall performance depends on how well tokens are routed– Team members use local models to improve routing

• Observation: Execution of algorithms shares information that other algorithms can exploit– E.g., if role for strike near Pittsburgh was allocated to WASM X,

then air space resources around Pittsburgh likely of interest to WASM X

• Implementation: Use all tokens to improve local models– E.g., role tokens change local models, resource tokens move

according to local models

• Result: When all algorithms are working together the system actually performs better

Implementing Synergies - Example

• When receiving an role token from neighbor i– Probability of sending information token to i

changed proportionally to similarity between “plan initiated” token and information token

– Probability of sending resource token to i changed inverse proportionally to similarity between role and resource token

Meta-Reasoning for Token Flows

• Specific tokens are meta-reasoned about– Identify tokens that are not behaving as expected

– Bring to human attention

– Examples• Tokens that “live” too long

• Tokens that travel too much

– See Scerri et al, AIAA 2004

• Overall flows can be controlled to optimize for specific criteria– By controlling the flows, we control how the

coordination works

Neural Networks for Modeling and Controlling Token Flows

• Use a simple input/output feed-forward neural networks to represent a team performance model

– Three-layer FFNN is capable of representing any arbitrary functions

• Extend to Dynamic Networks concept to cope with dynamic behaviors– This kind of network enlarges the capacity to deal with non-

deterministic problem

• Learn the model with genetic algorithms methodology– Excellent for dealing with very huge and noisy training data set

– Based on around 1,000,000 simulation runs

Configuring Algorithms with NN• Offline

– Change environment and algorithm parameters, observe expected performance

• Online– Use NN “in reverse” to find parameter settings

for specific optimization criteria• As environment changes• As requirements change

– Allows tradeoff between performance of all algorithms

Configuration Interface

Results: Token Based Coordination

0

50

100

150

200

250

300

350

400

450

500

0 100 200 300 400

Rounds

Rew

ards

(The

mor

e the

bet

ter)

Ranom Algorithm Plan Initiation Role AllocationResource Sharing Integrated Coordination

15000

20000

25000

30000

35000

40000

45000

50000

55000

60000

150 200 250 300 350 400 450

RewardsM

essa

ges

(The

less

the b

ette

r)

Random Plan Initiation Role Allocation

Resource Planning Integrated Coodination

Total Reward Messages

Fully Integrated Random

Results: Configuring Algorithms

Results: Online Control

Machinetta: Bringing it all Together• Encapsulate token-based coordination approach into

reusable software module– Called a proxy

• Proxies provide domain independent coordination algorithms– Do not provide domain specific communication and interface code– Available in the public domain

• Machinetta used to demonstrate coordination of up to 500 distributed, heterogeneous team members in several distinct domains– Demonstrates that token based coordination is feasible – Biggest teams developed to date?

Conclusions and Future Work• Token-based coordination as a feasible

alternative paradigm for large teams• Additional layer over token flows gives

high levels of control

• Future Work– Can we make more precise mathematical

models?• Markov chains?

Documents

Controlling and Configuring Large UAV Teams Paul Scerri, Yang Xu, Jumpol Polvichai, Katia Sycara and Mike Lewis Carnegie Mellon University and University