Introduction

Characterizing the Behavior of a Multi-Agent-Based Search by Using it to Solve a Tight, Real-world Resource Allocation

Problem

Hui Zou and Berthe Y. Choueiry

Constraint Systems LaboratoryDepartment of Computer Science and Engineering

University of Nebraska-Lincoln{hzou|choueiry}@cse.unl.edu

Introduction

Search algorithms: systematic or iterative repair

Complex, real-world optimization problems– Systematic search thrashes– Local search gets stuck in ‘local optima’– Remedial: random walk, breakout, restart strategies, etc.

Multi-agent-based search [Liu & al. AIJ 02]

– Also an iterative repair technique– provides us with a new way– Advantages & shortcomings via a practical application

Graduate Teaching Assistants (GTA) problem:

In a semester, given

– a set of courses

– a set of graduate teaching assistants

– a set of constraints that specify allowable assignments

Find a consistent and satisfactory assignment of GTAs to courses

Background - GTA

Detailed modeling in [Glaubius & Choueiry ECAI 02 WS on Modeling]

Types of constraints: unary, binary, non-binary

– Each course has a load, indicates weight of the course– Each GTA has a (hiring) capacity, limits max. load

Background - GTA (cont’)

Problem size:

Date set Mark # variables Domain size

Problem size

Spring2001bB 69 35 3.5×10106

O 69 26 4.3×1097

Fall2001bB 65 35 2.3×10100

O 65 34 3.5×1099

Fall2002B 31 33 1.2×1047

O 31 28 7.7×1044

Spring2003B 54 36 1.1×1084

O 54 34 5.0×1082

B – boosted to make problem solvable

O – original, not necessary solvable

In practice, this problem is tight, even over-constrained Our goal: ensure GTA support to as many courses as possible

Background - GTA (cont’)

Optimization criteria:

1. Maximize the number of courses covered

2. Maximize the geometric average of the assignments wrt the GTAs’ preference values (between 0 and 5).

Problem:

– Constraints are hard, must be met

– Maximal consistent partial-assignment problem (MPA-CSP?)

– Not a MAX-CSP (which maximizes #constraints satisfied)

Background - MAS for CSPs

Multi-Agent System: agents interact & cooperate in order to achieve a set of goals

– Agents: autonomous (perceive & act), goal-directed, can communicate

– Interaction protocols: governing communications among agents

– Environment: where agents live & act

ERA [Liu & al. AIJ 2002] – Environment, Reactive rules, and Agents

– A multi-agent approach to solving a general CSP

– Transitions between states when agents move

Background - ERA’s components

Environment: a n×m two-dimensional array – n: the number of variables (agents)

– m: the maximum domain size, |Dmax|

– e(i, j).value: domain value of agent i at position j

– e(i, j).violation: violation value of agent i at position j

– Zero position: where e(i, j).violation=0When all agents are in zero position, we have a complete solution

ERA=Environment + Reactive rules + Agents

Example:


Reactive rules:– Least-move: choose a position with the min. violation value– Better-move: choose a position with a smaller violation value– Random-move: randomly choose a positionCombinations of these basic rules form different behaviors.


R e a ct iv e ru le s B e h a v io rde s ig n e r

LR le a s t-m o v e with 1 -p and ra n d o m -m o v e with p

BR b e tte r -m o v e with 1 -p and ra n d o m -m o v e with p

BLR f i r s t b e tte r -m o v e , i f f ai l the n apply LR

rBLR f i r s t apply b e tte r -m o v e r t im e s , i f f ai l the n apply LR

F rBLR apply rBLR in the f i r s t r i te rat io ns , the n apply LR


Agents: a variable is represented by an agent


At each state, an agent chooses a position to move to, following the reactive rules. The agents keep moving until all have reached zero position, or a certain time period has elapsed.

All agents in zero position Some agents in zero position

Assignments are made only for agents in zero position

Background - ERA vs local search

ERA operates by local repairs, how different is it from local search?

ERA– Each agent has an evaluation function– At each state, any agent moves wherever it desires to moveControl is localized: Each agent is in pursuit of its own happiness

Local search with min-conflict– One evaluation function for the whole state (cost), summarizes the

quality of the state– At each state, few agents are allowed to move (most unhappy ones)Control is centralized: towards one common good

Background - Example ( ERA )

4-queen problem

2

2

2

0

Init

2 02 2

Eval (agent Q1)

0

Move (agent Q1)

1 23 2

Eval (agent Q2)

2 1 2 1

Eval (agent Q3)

1

Move (agent Q3)

1 0 13

Eval (agent Q4)

0

Move(agent4)

ERA – any agent can kick any other agent from its position

Local search with min-conflict – cannot repair a variable without violating a previously repaired variable

Background - Example (ERA vs. Local search)

2

2

2

0

0

1

1

0

0

1

0

1

0

0

0

0

2

2

2

0

0

1

1

0

Empirical study - In general

Apply ERA on GTA assignment problem:

0. (Test & understand the behavior of ERA)

1. Compare performance of: – ERA: FrBLR

– LS: hill-climbing, min-conflict & random walk

– BT: B&B-like, many orderings (heuristic, random)

2. Observe behavior of ERA on solvable vs. unsolvable problems

3. Observe behavior of individual agents in ERA4. Identify a limitation of ERA: deadlock phenomenon

8 instances of the GTA assignment problem

Empirical study 1- Performance comparison

Date set Systematic Search (BT) Local Search (LS)Multi-agent Search

(ERA)

Spring2001bB √ 35 69 35 29.6 1.18 6 4.05 2 6.5 3.77 5 3.69 0 6.4 0.87 0 3.20 0 5.3 0.18

O × 26 69 26 29.6 0.88 16 3.79 0 2.5 4.09 13 3.54 0 0.9 0.39 24 2.55 8 8.3 7.39

Fall2001bB √ 35 65 31 29.3 1.06 2 3.12 0 2.5 1.71 4 3.01 0 3.8 0.33 0 3.18 1 1.9 2.68

O √ 34 65 30 29.3 1.02 2 3.12 0 1.5 2.46 4 3.04 1 3.7 0.10 0 3.27 0 0.8 1.15

Fall2002B √ 33 31 16.5 13 1.27 1 3.93 0 3.5 2.39 2 3.40 0 5.0 0.85 0 3.62 2 3.0 0.02

O × 28 31 11.5 13 0.88 4 3.58 0 1.8 2.56 4 3.61 0 2.0 0.16 8 3.22 1 2.0 0.51

Spring2003B √ 36 54 29.5 27.4 1.08 3 4.49 2 4.2 1.17 3 3.62 0 3.9 0.32 0 3.03 1 2.8 0.49

O √ 34 54 27.5 27.4 1.00 3 4.45 0 2.2 1.53 4 3.63 0 3.3 1.42 0 3.26 0 0.8 0.14

Ori

gin

al/B

oo

ste

d

So

lva

ble

?

# G

TA

s

# C

ou

rse

s

To

tal

ca

pa

cit

y (C

)

To

tal

load

(L

)

Rat

io=

C \

L

Un

as

sig

ne

d C

ou

rse

s

So

luti

on

Qu

ali

ty

Un

us

ed

GT

As

Ava

ila

ble

Re

sou

rce

CC

(×

10

8 )

Un

as

sig

ne

d C

ou

rse

s

So

luti

on

Qu

ali

ty

Un

us

ed

GT

As

Ava

ila

ble

Re

sou

rce

CC

(×

10

8 )

Un

as

sig

ne

d C

ou

rse

s

So

luti

on

Qu

ali

ty

Un

us

ed

GT

As

Ava

ila

ble

Re

sou

rce

CC

(×

10

8 )

Observations:- Only ERA finds complete solutions to all solvable instances- On unsolvable problems, ERA leaves too many unused GTAs- LS and BT exhibit similar behaviors

Empirical study 2- Solvable vs unsolvable

15

20

25

30

35

40

45

50

55

60

65

70

1 20 39 58 77 96 115 134 153 172 191

iteration

# ag

ents

in z

ero

posi

tion

Spring2001b (B)

Fall2002 (B)

Fall2001b

Spring2003

10

15

20

25

30

35

40

45

1 20 39 58 77 96 115 134 153 172 191

iteration

# ag

ents

in z

ero

posi

tion

Spring2001b (O)

Fall 2002 (O)

ERA performance on solvable problems

ERA performance on unsolvable problems

Observation:

- Number of agents in zero-position per iteration

- ERA behavior differs on solvable vs. unsolvable instances

Empirical study 3- Behavior of individual agents

0

20

40

1 51 101 151 201 251 301 351 401 451

0

10

20

30

1 51 101 151 201 251 301 351 401 451

0

10

20

1 51 101 151 201 251 301 351 401 451

va ria ble

s ta ble

c o ns ta nt

ind

ex

of

po

sit

ion

iteration

Instances• solvable • unsolvable

Motion of agents• variable• stable• constant

Observations:

Solvable Unsolvable

Variable None Most

Stable A few A few

Constant Most None

Empirical study 4- Deadlock

– Each circle corresponds to a given GTA – Each square represents an agent– A blank squares indicate that an agent is on a zero-position– The squares with same color indicate agents involved in a deadlock

Observation:

ERA is not able to avoid deadlocks and yields a degradation of the solution on unsolvable CSPs.

Discussion

Goal Actions

Control Schema Undoing assignments Conflict resolution

ERALocal

+ Immune to local optima

– May yield instability

√+ Flexible

+ Solves tight CSPs

Non-committal– Deadlock

– Shorter solutions

LS

Global+ Stable behavior

– Liable to local optima

×+ Quickly stabilizes

– Fails to solve tight CSPs even with randomness & restart strategies

Heuristic

+ Longer solutions

BT

Systematic+ Stable behavior

– Thrashes

~+ Quickly stabilizes

– Fails to solve tight CSPS even with backtracking & restart strategies

+ advantages – shortcomings

Dealing with the deadlock

Possible approaches:— Direct communications, negotiation mechanisms— Hybrids of search Global control Conflict resolution

Experiments:— Enhancing ERA with global control

– Don’t accept a move that deteriorates the global goal– Lead to local-search-like behavior (i.e., local optima)

— ERA with conflict resolution– add dummy resources– find a complete solution when LS and BT fail – remove dummy assignments, solutions are still better

Future research directions

– Enhance ERA to handle optimization problems – Test approach using other search techniques

– BT search: Randomized, credit-based– Other local repair: squeaky-wheel method – Market-based techniques, etc.

– Validate conclusions on other CSPs– random instances, real-world problems

– Try search-hybridization techniquesReferences:R. Glaubius and B.Y. Choueiry, Constraint Modeling and Reformulation in the Context of Academic Task Assignment. In Workshop Modeling and Solving Problems with Constraints, ECAI 2002.

J. Liu, H. Jing, and Y.Y. Tang. Multi-Agent Oriented Constraint Satisfaction. Artificial Intelligence, 136:101-144, 2002.

Questions

Documents

Introduction