1 Multiagent Teamwork: Analyzing the Optimality and Complexity of Key Theories and Models David V....

1

Multiagent Teamwork:Analyzing the Optimality and Complexity of Key Theories and Models

David V. Pynadath and Milind Tambe

Information Sciences Institute and

Department of Computer Science

University of Southern California

2

Agent Teamwork

Agents, robots, sensors, spacecraft, etc. Performing a common task Operating in an uncertain environment Distinct, uncertain observations Distinct actions with uncertain effects Limited, costly communication

Battlefield Simulation Satellite Clusters Disaster Rescue

3

MotivationP

erfo

rman

ce

Complexity

OptimalNew algorithm

TheoreticalApproaches

No communication

Practical Systems

?

?

?

?Optimal

Outline of Results1) Unified teamwork framework

2) Complexity of optimal teamwork

3) New coordination algorithm

4) Optimality-Complexity evaluation of existing methods

4

Enemy Radar

Example Domain:Helicopter Team

Goal

Did theysee that? I destroyed the

enemy radar.

5

Communicative Multiagent Team Decision Problem (COM-MTDP)

S: states of the world e.g., position of helicopters, position of the enemy

A: domain-level actions e.g., fly below radar, fly normal altitude

P: transition probability function e.g., world dynamics, effects of actions

: communication capabilities, possible “speech acts” e.g., “I have destroyed enemy radar.”

6

COM-MTDPs (cont’d)

: observations e.g., enemy radar, position of other helicopter

O: probability (for each agent) of observation Maps state and actions into distribution over

observations (e.g., sensor noise model)R: reward (over states, actions, messages) e.g., good if we reach destination, better if we reach it earlier

e.g., saying, “I have destroyed enemy,” has a cost

Teamwork Definition: All members share same preferences (i.e., R)

7

Problem Complexity

COM-MTDPs

Free communication

Collectively Observable

Individually Observable

No communication

8

To Communicate orNot To Communicate

Local decision of one agent at a single point in time: “I have achieved a joint goal.” “Should I tell my teammate?”

Joint intentions theory: “I must attain mutual belief.” Always communicate [Jennings]

STEAM: “I must communicate if the expected cost of

miscoordination outweighs the cost of communication.” [Tambe]

Each cost is a fixed parameter specified by designer

9

Communicate if and only if: E[R | communicate] E[R | do not communicate]

Locally Optimal Criterionfor Communication

Expectation over possible histories of states and beliefs up to current time

Expected reward over future trajectories of states and beliefs WITH communication

Expected reward over future trajectories of states and beliefs WITHOUT communication

Expected cost of communicating

10

Empirical Results

Communication Cost

Observability

V_opt-V

11

Empirical Results

12

Empirical Results

13

Silent

Optimality vs. Complexity

Op

tim

alit

y

Complexity

Globally OptimalLocally Optimal

STEAM

Jennings

seconds(log)0.1 1.0 10,000

E[R]

1.46

1.43

1.40 Observability = 0.2Comm. Cost = 0.7

14

Jennings

Optimality vs. Complexity

Op

tim

alit

y

Complexity

Globally OptimalLocally Optimal

STEAM

Silent

seconds(log)0.1 1.0 10,000

E[R]

1.80

1.43 Observability = 0.2Comm. Cost = 0.3

15

Summary

COM-MTDPs provide a unified framework for agent teamwork Representation subsumes many existing agent models Policy space subsumes many existing prescriptive theories

This framework supports deeper analyses of teamwork problems Quantitative characterization of optimality-efficiency

tradeoff , for different policies, in different domains Derivation of novel coordination algorithms

http://www.isi.edu/teamcore/Teamwork Detailed proofs Source code JAIR article