1 Abstract Reasoning for Multiagent Coordination and Planning Bradley J. Clement

1

Abstract Reasoning for Multiagent Coordination and

Planning

Bradley J. Clement

2

Overview• Problem description• Summary of approach• Related work• Representations and supporting algorithms

– CHiPs– Metric resources– Summary information

• Coordination algorithm– Complexity analyses– Decomposition search techniques– Applications and experiments

• Planning– Concurrent hierarchical refinement and local search planners– Scheduling complexity– Mars rovers experiments

• Conclusion

3

Manufacturing Example

Production, Inventory, andFacility Managers

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E

transport2transport1

7

• Managers must coordinate or risk failure.• Managers develop plans independently.• Managers need sound and complete coordination

algorithm.• Managers may need to make coordination decisions

quickly.• Managers must reason about concurrent

action to use resources efficiently.

• Managers may need plans that handle unexpected events.

Problem Characteristics

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E


8

Problem

• Coordination (or planning) should be sound & complete.

• Agents should not coordinate (reason about subgoals) where there are no conflicts.

• Agents should act as soon as possible.• Agents should accomplish goals efficiently.

– Agents should act concurrently.– Agents should maximize utility.

• Agents should be able to handle unexpected events.

Find preferable elaborations or modifications to a group of agents’ plans that achieve their goals while striking a balance among the following objectives:

9





• Conclusion

10

Approach• Reason about plans at abstract levels to reduce the

information needed to make efficient coordination and planning decisions– concurrent hierarchical plan representation– summarize constraints of abstract tasks from those of tasks in their

decompositions– use this summary information to reason about interactions of abstract

plans

• Construct sound and complete coordination & planning algorithms

• Explore techniques and heuristics for decomposition search based on summary information

• Analyze complexity of abstract reasoning• Evaluate in different domains

11

Approach

• Complete at high level using summary information to gain flexibility in execution

• Better solutions may exist at lower levels

• Summary information aids in pruning subplans to resolve threats

coordinationlevels

crispercrispersolutionssolutionslowerlower

coordinationcoordinationcostcost

moremoreflexibilityflexibility

12

How Approach Addresses Problem• Coordination (or planning) decisions should be sound & complete.

Formalize summary information and algorithms• Agents should not coordinate (reason about subgoals) where there are no

conflicts. Use decomposition techniques and heuristics to focus search

• Agents should act as soon as possible. Find solutions efficiently at multiple levels of abstraction

• Agents should accomplish goals efficiently.– Agents should act concurrently. Reason about concurrent interactions at abstract levels– Agents should maximize utility. Use decomposition techniques and heuristics to guide search to better

solutions

• Agents should be able to handle unexpected events. Preserve decomposition choices by finding abstract solutions

15

Approach - Limitations• Do not offer algorithms/protocols that determine

optimal balancing of problem objectives– do give mechanisms that enable tradeoffs

• Do not investigate alternative coordination/negotiation protocols– instead, identify who needs to coordinate, what needs

to be coordinated, and alternative settlements

• Planning language– Only grounded, propositional states formalized

• mention how uninstantiated variables are implemented

– Metric resource usage is instantaneous

18

Contributions• Algorithms for deriving and reasoning about

summary information• Sound and complete concurrent hierarchical

coordination & planning algorithms• Integration of summary information in a local search

planner• Search techniques and heuristics that efficiently

guide decomposition and prune the search space• Complexity analyses and experiments that show

where abstract reasoning exponentially reduces cost of computation and communication

19





• Conclusion

20

Related ApproachesApproaches

Needs

Summary Info

Plan merging

HTN planning

Social Laws

Hierarchical behavior

space search

TÆMS GPGP

DPOCL OICR Distributed NOAH

Sound & complete

Avoid unnecessary coordination

Minimize coordination time

Concurrent action

Maximize utility

Handle unexpected events

24





• Conclusion

25

produce G

produce Gproduce Gon M1 on M2

build Gmove A&Bto M2

move A to M2 move B to M2

move Gto M2 build H

move Hto bin1

produce H

produce H from G

Concurrent Hierarchical Plans (CHiPs)

• pre, in, & postconditions - sets of literals for a set of propositions

• type - and, or, primitive• subplans - execute all for and, one for or; empty for

primitive• orderorder - conjunction of point or interval relations B - beforeB - before

BB

BB

BB BB

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E


32

produce G

produce Gproduce Gon M1 on M2

build Gmove A&Bto M2

move A to M2 move B to M2

move Gto M2 build H

move Hto bin1

produce H

produce H from G

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E


• existence: must, may• timing: always, sometimes, first, last• external preconditions• external postconditions

pre: available(A), available(M2)



pre: available(A), available(M1), available(M2)

Summary Conditions

pre: available(A)

pre: available(A)

36

Deriving Summary Conditions

• Can be run offline for a domain• Recursive algorithm bottoming out at primitives• Derived from those of immediate subplans• O(n2c2) for n non-primitive plans in hierarchy and

c conditions in each set of pre, in, and postconditions

• Properties of summary conditions are proven based on procedure

• Proven procedures for determining must/may - achieve/undo/clobber

37

Metric Resource Usage

• Depletable resource– usage carries over after end of

task– gas = gas - 5

• Non-depletable– usage is only local– zero after end of task– machines = machines - 2

• Replenishing a resource– negative usage– gas = gas + 10– can be depletable or

non-depletable

interval of task

40

summarized resource usage

< local_min_range, local_max_range, persist_range >

• Captures uncertainty of decomposition choices and temporal uncertainty of partially ordered actions

• Can be used to determine if a resource usage may, must, or must not cause a conflict

Summarizing Resource Usage

0

40

30

-7

-20

2010

< [-20, -7],[30, 40],[10, 20] >

41

Resource Summarization Algorithm

• Can be run offline for a domain model• Run separately for each resource• Recursive from leaves up hierarchy• Summarizes parent from summarizations of

immediate children• Considers all legal orderings of children• Considers all subintervals where upper and lower

bounds of children’s resource usage may be reached• Exponential with number of immediate children, so

summarization is really constant for one resource and O(r) for r resources

46





• Conclusion

47

Determining Temporal Relations• CanAnyWay({relations}, {psum, qsum}) - relations can

hold for any way p and q can be executed

• MightSomeWay({relations}, {psum, qsum}) - relations might hold for some way p and q can be executed

CanAnyWay({before}, {produce_H, maintenance})CanAnyWay({overlaps}, {produce_H, maintenance})

MightSomeWay({overlaps}, {produce_H, maintenance})

B - before

O - overlaps

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E


produce H

maintenance

48

Concurrent Hierarchical Plan Coordination

• Agents individually derive summary information for their plan hierarchies

• Coordinator requests summary information for expansions of agents’ hierarchies from the top down

• After each expansion, try to resolve threats by adding ordering constraints

• Algorithm shown to be sound and complete

49

Search for Coordinated Plan

• search state– set of expanded plans – set of blocked subplans– set of temporal constraints

• search operators– expand– block– constrain

blocked

blocked

temporal constraints

56

• Number of plan steps per level grows exponentially down the hierarchy O(bi)

• In worst case, summary information for each plan grows exponentially up the hierarchy O(bd-ic)

• Number of orderings of plans grows exponentially down hierarchy O(bi!)

• Resolving threats is NP-complete (reduced from Hamiltonian Path)

• In worst case, search space reduced by O(kbd-bi).

• In best case, O(kbd-bib2(d-i)).

Easier to Coordinate at Higher Levels

b - branching factori - leveld - depthc - conditions per plan

59

Search Techniques

• Prune inconsistent global plans • Branch & bound - abstract solutions

help prune space where cost is higher• “Expand most threats first” (EMTF)

– expand subplan involved in most threats– focuses search on driving down to source of

conflict

• “Fewest threats first” (FTF)– search plan states with fewest threats first– or subplans involved in most threats are blocked

first

60

Evacuation Domain Experiments

• Compare different strategies of ordering search states and ordering expansions– FAF-FAF– DFS-ExCon– FTF-EMTF– FTF-ExCon

• 4 - 12 locations• 2 - 4 transports• no, partial, & complete overlap in locations

visited

66


Summary information decomposition techniques outperform previous state-of-the-art by orders of magnitude

Search States Expanded

1

10

100

1000

10000

1 10 100 1000 10000

FTF-EMTF

DF

S-E

xCo

n

67


Decomposition techniques using summary information dominate previous heuristics in finding optimal solutions– FTF especially effective compared to

random, DFS, and FAF– EMTF not especially more effective than

ExCon but finds solutions more regularly– Overall performance differs by orders of

magnitude

68

Communication in Manufacturing Domain

• Centralized coordinator• Measure delay with varying bandwidth and latency:

(n-2)l + s/bn = number of messagess = total size of messagesl = latencyb = bandwidth

bin2 bin3 bin4bin1

M1 M2

B C

dock

A tool

D

E


74

Communication in Manufacturing Domain• Agents can minimize communication by sending summary

information at intermediate levels with a particular granularity.

• Sending all plan information at once can be exponentially more expensive: O(bd-i).

• Sending summary information one task at a time can cause exponentially greater latency: O(bi).

• However, if summary information does not collapse up the hierarchy, and coordination must occur at primitive levels, sending all at once is best.

• Domain modeler can perform similar experiment to determine appropriate granularity to send summary information.

75

Multi-Level Coordination Agent (MCA)

• Centrally coordinates plans of requesting agents in episodes

• Requests summary information as needed or summarizes given hierarchies

• Displays discovered solutions that are “better” or Pareto-optimal

• Sends synchronization and decomposition choice constraints to agents upon selection of a solution

W E

N

S

CapeVincent

CapeAmstado

Cac

a

KasoLagoon

Amisa

Jacal

Pra

Ankobra

Tana

Ofin

Afr

am

Daka

Black Caca

Kapowa

White C

aca

Mawli

LAKE CACA

Forces separated by Firestorm

AGADEZ

GAO

BinniBinni

Laki SafariPark

Gaoforces

AgadezForces

False Agadezforces

FIRESTORM

False Gaoforces

82

Multi-Level Coordination of Military Coalitions

83





• Conclusion

84

Concurrent Hierarchical Refinement Planner

• Simple modification to coordination algorithm– discover whether potential internal conflicts exist during summarization– must expand any task with potential internal conflicts

• Derive summary information for hierarchy expanded to primitive level (iteratively expand for infinite recursion of methods)

• Expand hierarchy from the top down, selecting or blocking or decomposition choices

• After each expansion, try to resolve threats– add ordering constraints– check CAW and MSW

• Sound and complete• Same complexity benefits as coordination algorithm

86

Summary Information in Local Search Planners

• Local plan-space search involves modifying (e.g. deleting, moving, adding, etc.) tasks in an existing plan.

• Hierarchy is used to pass parameters, specify temporal constraints, and explore alternative decompositions for subtasks.

• Planners like ASPEN fix the start times and durations of activities and track states and resources within a time horizon.

• Algorithms for reasoning about summary states and resources are used to track uncertain states/resources for abstract tasks.

• Using summary information results in more efficient planning and scheduling.

planninglevels

crispercrispersolutionssolutions

lowerlowerplanningplanning

costcost

moremoreflexibilityflexibility

90

Complexity Analyses: Local Search

• Moving an activity hierarchy is a factor of O(b2(d-i)) more complex at level d than i if summary information fully collapses up the hierarchy.

• If no information collapses, moving a hierarchy has the same complexity at all levels O(vnb2d).

• The number of potential temporal constraint conflicts is a factor of O(bd-i) greater at level d than i.

• Thus, reasoning at abstract levels can resolve conflicts exponentially faster.

. . .

level01

d1 2 n

branchingfactor b

c constraintsper hierarchy

vvariables

91

Decomposition Strategies• Level expansion

– repair conflicts at current level of abstraction until conflicts cannot be further resolved

– then decompose all activities to next level and begin repairing again

• Expand most threats first (EMTF)– instead of moving activity to resolve conflict,

decompose with some probability (decomposition rate)

– expands activities involved in greater numbers of conflicts (threats)

• FTF (fewest-threats-first) heuristic tests each decomposition choice and picks those with fewer conflicts with greater probability.

94

Multi-Rover Domain• 2 to 5 rovers• Triangulated field of 9 to 105 waypoints• 6 to 30 science locations assigned

according to a multiple travelling salesman algorithm

• Rovers’ plans contain 3 shortest path choices to reach next science location

• Paths between waypoints have capacities for a certain number of rovers

• Rovers cannot be at same location at the same time

• Rovers cannot cannot cross a path in opposite directions at the same time

• Rovers communicate with the lander over a shared channel for telemetry--different paths require more bandwidth than others

95

Experiments using ASPEN for a Multi-Rover Domain

Performance improves greatly when activities share a common resource.

0

1000

2000

3000

4000

5000

6000

0 1000 2000 3000 4000 5000 6000

Summary Information + Aggregation CPU Seconds

Ag

gre

gat

ion

CP

U s

eco

nd

s

0

1000

2000

3000

4000

5000

6000

0 1000 2000 3000 4000 5000 6000

Summary Information + Aggregation CPU seconds

Ag

gre

gat

ion

CP

U s

eco

nd

s

0

1000

2000

3000

4000

5000

6000

0 1000 2000 3000 4000 5000 6000

Summary Information + Aggregation CPU seconds

Ag

gre

gat

ion

CP

U s

eco

nd

s

Rarely shared resources (only path variables) Mix of rarely shared (paths) and often shared(channel) resources

Often shared (channel) resource only

99





• Conclusion

100

Contributions

• Algorithms for deriving and reasoning about summary information for propositional state and metric resources– must/may assert, achieve, clobber, undo– CAW & MSW to determine whether abstract plans

are conflict free or unresolvable– toolbox of sound and complete algorithms for

constructing efficient coordination and planning algorithms

101

Contributions

• Coordination and planning algorithms– sound, complete concurrent hierarchical

coordination– sound, complete concurrent hierarchical

planner– iterative repair planner employing abstract

reasoning with summary information– evaluated in manufacturing, evacuation,

military operations, and Mars rovers domains

102

Contributions• Complexity analyses and experiments

– Finding solutions at abstract levels is exponentially less complex O(kbd-bi) in number of tasks for both refinement and local search.

– Finding abstract solutions is exponentially less complex when summarization collapses constraints O(b2(d-i)) for both refinement and local search.

– Experiments support the analyses in evacuation and Mars rovers domains.

– Communication delay can be reduced exponentially by• gradually sending summary information O(bd-i) and• sending at an appropriate granularity O(bi).

– Extension of work by Korf ’87 and Knoblock ‘91 showing how hierarchical coordination/planning can obtain exponential speedups when subgoals interact

103

Contributions

• Decomposition search techniques– EMTF, FTF (for refinement and local search)– Pruning of inconsistent and costlier search

space– Evaluation against prior heuristics showing

stronger ability to find optimal solutions at lower abstraction levels

104

• Applying summary information to other classes of coordination/planning– state-based search– complex resources– more expressive temporal models

• Summarizing other information– constraint hierarchies (in addition to task hierarchies)– reasoning about uncertainty and risk

• Coordination protocols based on summary information– organization and scaling of agent groups– BDI-based multiagent mental models

• Coordinating continuously• Interfacing deliberative and reactive coordination• Exploiting synergy while coordinating• Case-based coordination

Future Directions

Documents

1 Abstract Reasoning for Multiagent Coordination and Planning Bradley J. Clement