Information Sharing for Distributed Planning

AAMAS 2010 - Doctoral Symposium 1

Information Sharing forDistributed Planning

Prasanna Velagapudi

AAMAS 2010 - Doctoral Symposium 2

Large Heterogeneous Teams

• 100s to 1000s of robots, agents, people

• Complex, collaborative tasks

• Dynamic, uncertain environment

• Joint planning intractable

AAMAS 2010 - Doctoral Symposium 3

Scaling Team Planning

• Independent planners: can’t account for teammates• Existing work: needs specific structure or doesn’t

scale to these sizes– DPC, Prioritized Planning– JESP, Factored MDP, ND-POMDP

AAMAS 2010 - Doctoral Symposium 4

Iterated Distributed Planning

1. Factor the problem, enumerate interactions2. Compute independent plans & potential interactions3. Exchange messages about interactions4. Use exchanged information, improve local model

AAMAS 2010 - Doctoral Symposium 5

Iterated Distributed Planning

1. Factor the problem, enumerate interactions2. Compute independent plans & potential interactions3. Exchange messages about interactions4. Use exchanged information, improve local model

?

AAMAS 2010 - Doctoral Symposium 6

Iterated Distributed Planning

1. Factor the problem, enumerate interactions2. Compute independent plans & potential interactions3. Exchange messages about interactions4. Use exchanged information, improve local model

?

AAMAS 2010 - Doctoral Symposium 7

Iterated Distributed Planning

1. Factor the problem, enumerate interactions2. Compute independent plans & potential interactions3. Exchange messages about interactions4. Use exchanged information, improve local model

AAMAS 2010 - Doctoral Symposium 8

A Tale of Two Distributed Planners

Distributed Prioritized Planning (DPP) L-TREMOR

AAMAS 2010 - Doctoral Symposium 9

Distributed Prioritized Planning

AAMAS 2010 - Doctoral Symposium 10

Multiagent Path Planning

Start

Goal

AAMAS 2010 - Doctoral Symposium 11

Multiagent Path Planning

AAMAS 2010 - Doctoral Symposium 12

Prioritized Planning

• Assign priorities to agents based on path length

[van den Berg, et al 2005]

AAMAS 2010 - Doctoral Symposium 13

Prioritized Planning

• Plan from highest priority to lowest priority• Use previous agents as dynamic obstacles

[van den Berg, et al 2005]

AAMAS 2010 - Doctoral Symposium 14

Distributed Prioritized Planning

Parallelizable& Equivalent

AAMAS 2010 - Doctoral Symposium 15

Large-Scale Path Solutions

AAMAS 2010 - Doctoral Symposium 16

Large-Scale Path Solutions

AAMAS 2010 - Doctoral Symposium 17

DPP Results

Fewer Sequential Plans

AAMAS 2010 - Doctoral Symposium 18

DPP Results

Longer Planning TimeFewer Sequential Plans

AAMAS 2010 - Doctoral Symposium 19

• Prioritized Planning

• DPP

Why does this happen?

ABCD

ABCD

Longest planning agents might replan multiple times

Individual agent planning times varied by >2 orders of magnitude

Solution 2: Incremental Planning

Solution 1: Prioritize by plan time?

AAMAS 2010 - Doctoral Symposium 20

Summary of DPP

• Observable, certain world• Only one type of interaction: collision

• Far fewer sequential planning iterations• Incremental planning may reduce execution time

AAMAS 2010 - Doctoral Symposium 21

L-TREMOR

AAMAS 2010 - Doctoral Symposium 22

A Simple Rescue Domain

Rescue Agent

Cleaner Agent

Narrow Corridor

Victim

Unsafe Cell

Clearable Debris

AAMAS 2010 - Doctoral Symposium 23

A Simple (Large) Rescue Domain

AAMAS 2010 - Doctoral Symposium 24

Distributed POMDP with Coordination Locales (DPCL)

• Often, interactions between agents are sparse

Only fits one agent Passable if

cleaned

[Varakantham, et al 2009]

AAMAS 2010 - Doctoral Symposium 25

Distributed POMDP with Coordination Locales (DPCL)

• Define coordination locales (CLs) where POMDP model functions are not independent:

[Varakantham, et al 2009]

<S, A, Ω, P, R, O> (states) (actions) (obs.) (transition)(reward)(obs. fn)

AAMAS 2010 - Doctoral Symposium 26

Distributed POMDP with Coordination Locales (DPCL)

• Define coordination locales (CLs) where POMDP model functions are not independent:

[Varakantham, et al 2009]

S1, A1 S2, A2

SglobalR1, P1, O1 R2, P2, O2

Outside CL:(typical)

AAMAS 2010 - Doctoral Symposium 27

Distributed POMDP with Coordination Locales (DPCL)

• Define coordination locales (CLs) where POMDP model functions are not independent:

[Varakantham, et al 2009]

S1, A1 S2, A2

Sglobal

R12, P12, O12

Inside CL:(interaction)

AAMAS 2010 - Doctoral Symposium 28

TREMOR

Role Allocation Policy Solution Interaction Detection Coordination

TREMOR

Branch & Bound MDP

Independent EVA[3] solvers

Joint policy evaluation

Reward shapingof independent

models

[Varakantham, et al 2009]

AAMAS 2010 - Doctoral Symposium 29

L-TREMOR

Role Allocation Policy Solution Interaction Detection Coordination

TREMOR

Branch & Bound MDP

Independent EVA[3] solvers

Joint policy evaluation Reward shaping

of independentmodels

L-TREMOR

DecentralizedAuction

Sampling & message passing

Distributed & Parallelizable

AAMAS 2010 - Doctoral Symposium 30

Preliminary Results – Joint Utility

N = 6 N = 10N = 100

(structurally similar to N=10)

AAMAS 2010 - Doctoral Symposium 31

Preliminary Results – Timing

AAMAS 2010 - Doctoral Symposium 32

Preliminary Results – Model Accuracy

R = 0.804

AAMAS 2010 - Doctoral Symposium 33

Current Issues

• Oscillations in solutions

• Discovery of relevant locales

?

AAMAS 2010 - Doctoral Symposium 34

Summary of L-TREMOR

• Partially-observable, uncertain world• Multiple types of interactions• Role-allocation of tasks

• Improvement over independent planning• Handles large problems• Next steps: improving convergence

AAMAS 2010 - Doctoral Symposium 35

Conclusions

• Two approaches to distributed planning– DPP: approaching centralized performance– L-TREMOR: exceeding joint tractability

• Analogous strategies for distributing planning– Both iterate independent planners– Both exchange messages about states, actions

AAMAS 2010 - Doctoral Symposium 36

Future Work

• Generalized framework for distributed planning through iterative message exchange

• Reduce necessary communication• Better search over task allocations• Scaling to larger team sizes