View
20
Download
0
Category
Tags:
Preview:
DESCRIPTION
Characterizing the Behavior of a Multi-Agent-Based Search by Using it to Solve a Tight, Real-world Resource Allocation Problem Hui Zou and Berthe Y. Choueiry Constraint Systems Laboratory Department of Computer Science and Engineering University of Nebraska-Lincoln - PowerPoint PPT Presentation
Citation preview
Characterizing the Behavior of a Multi-Agent-Based Search by Using it to Solve a Tight, Real-world Resource Allocation
Problem
Hui Zou and Berthe Y. Choueiry
Constraint Systems LaboratoryDepartment of Computer Science and Engineering
University of Nebraska-Lincoln{hzou|choueiry}@cse.unl.edu
Introduction
Search algorithms: systematic or iterative repair
Complex, real-world optimization problems– Systematic search thrashes– Local search gets stuck in ‘local optima’– Remedial: random walk, breakout, restart strategies, etc.
Multi-agent-based search [Liu & al. AIJ 02]
– Also an iterative repair technique– provides us with a new way– Advantages & shortcomings via a practical application
Graduate Teaching Assistants (GTA) problem:
In a semester, given
– a set of courses
– a set of graduate teaching assistants
– a set of constraints that specify allowable assignments
Find a consistent and satisfactory assignment of GTAs to courses
Background - GTA
Detailed modeling in [Glaubius & Choueiry ECAI 02 WS on Modeling]
Types of constraints: unary, binary, non-binary
– Each course has a load, indicates weight of the course– Each GTA has a (hiring) capacity, limits max. load
Background - GTA (cont’)
Problem size:
Date set Mark # variables Domain size
Problem size
Spring2001bB 69 35 3.5×10106
O 69 26 4.3×1097
Fall2001bB 65 35 2.3×10100
O 65 34 3.5×1099
Fall2002B 31 33 1.2×1047
O 31 28 7.7×1044
Spring2003B 54 36 1.1×1084
O 54 34 5.0×1082
B – boosted to make problem solvable
O – original, not necessary solvable
In practice, this problem is tight, even over-constrained Our goal: ensure GTA support to as many courses as possible
Background - GTA (cont’)
Optimization criteria:
1. Maximize the number of courses covered
2. Maximize the geometric average of the assignments wrt the GTAs’ preference values (between 0 and 5).
Problem:
– Constraints are hard, must be met
– Maximal consistent partial-assignment problem (MPA-CSP?)
– Not a MAX-CSP (which maximizes #constraints satisfied)
Background - MAS for CSPs
Multi-Agent System: agents interact & cooperate in order to achieve a set of goals
– Agents: autonomous (perceive & act), goal-directed, can communicate
– Interaction protocols: governing communications among agents
– Environment: where agents live & act
ERA [Liu & al. AIJ 2002] – Environment, Reactive rules, and Agents
– A multi-agent approach to solving a general CSP
– Transitions between states when agents move
Background - ERA’s components
Environment: a n×m two-dimensional array – n: the number of variables (agents)
– m: the maximum domain size, |Dmax|
– e(i, j).value: domain value of agent i at position j
– e(i, j).violation: violation value of agent i at position j
– Zero position: where e(i, j).violation=0When all agents are in zero position, we have a complete solution
ERA=Environment + Reactive rules + Agents
Example:
Background - ERA’s components
Reactive rules:– Least-move: choose a position with the min. violation value– Better-move: choose a position with a smaller violation value– Random-move: randomly choose a positionCombinations of these basic rules form different behaviors.
ERA=Environment + Reactive rules + Agents
R e a ct iv e ru le s B e h a v io rde s ig n e r
LR le a s t-m o v e with 1 -p and ra n d o m -m o v e with p
BR b e tte r -m o v e with 1 -p and ra n d o m -m o v e with p
BLR f i r s t b e tte r -m o v e , i f f ai l the n apply LR
rBLR f i r s t apply b e tte r -m o v e r t im e s , i f f ai l the n apply LR
F rBLR apply rBLR in the f i r s t r i te rat io ns , the n apply LR
Background - ERA’s components
Agents: a variable is represented by an agent
ERA=Environment + Reactive rules + Agents
At each state, an agent chooses a position to move to, following the reactive rules. The agents keep moving until all have reached zero position, or a certain time period has elapsed.
All agents in zero position Some agents in zero position
Assignments are made only for agents in zero position
Background - ERA vs local search
ERA operates by local repairs, how different is it from local search?
ERA– Each agent has an evaluation function– At each state, any agent moves wherever it desires to moveControl is localized: Each agent is in pursuit of its own happiness
Local search with min-conflict– One evaluation function for the whole state (cost), summarizes the
quality of the state– At each state, few agents are allowed to move (most unhappy ones)Control is centralized: towards one common good
Background - Example ( ERA )
4-queen problem
2
2
2
0
Init
2 02 2
Eval (agent Q1)
0
Move (agent Q1)
1 23 2
Eval (agent Q2)
2 1 2 1
Eval (agent Q3)
1
Move (agent Q3)
1 0 13
Eval (agent Q4)
0
Move(agent4)
ERA – any agent can kick any other agent from its position
Local search with min-conflict – cannot repair a variable without violating a previously repaired variable
Background - Example (ERA vs. Local search)
2
2
2
0
0
1
1
0
0
1
0
1
0
0
0
0
2
2
2
0
0
1
1
0
Empirical study - In general
Apply ERA on GTA assignment problem:
0. (Test & understand the behavior of ERA)
1. Compare performance of: – ERA: FrBLR
– LS: hill-climbing, min-conflict & random walk
– BT: B&B-like, many orderings (heuristic, random)
2. Observe behavior of ERA on solvable vs. unsolvable problems
3. Observe behavior of individual agents in ERA4. Identify a limitation of ERA: deadlock phenomenon
8 instances of the GTA assignment problem
Empirical study 1- Performance comparison
Date set Systematic Search (BT) Local Search (LS)Multi-agent Search
(ERA)
Spring2001bB √ 35 69 35 29.6 1.18 6 4.05 2 6.5 3.77 5 3.69 0 6.4 0.87 0 3.20 0 5.3 0.18
O × 26 69 26 29.6 0.88 16 3.79 0 2.5 4.09 13 3.54 0 0.9 0.39 24 2.55 8 8.3 7.39
Fall2001bB √ 35 65 31 29.3 1.06 2 3.12 0 2.5 1.71 4 3.01 0 3.8 0.33 0 3.18 1 1.9 2.68
O √ 34 65 30 29.3 1.02 2 3.12 0 1.5 2.46 4 3.04 1 3.7 0.10 0 3.27 0 0.8 1.15
Fall2002B √ 33 31 16.5 13 1.27 1 3.93 0 3.5 2.39 2 3.40 0 5.0 0.85 0 3.62 2 3.0 0.02
O × 28 31 11.5 13 0.88 4 3.58 0 1.8 2.56 4 3.61 0 2.0 0.16 8 3.22 1 2.0 0.51
Spring2003B √ 36 54 29.5 27.4 1.08 3 4.49 2 4.2 1.17 3 3.62 0 3.9 0.32 0 3.03 1 2.8 0.49
O √ 34 54 27.5 27.4 1.00 3 4.45 0 2.2 1.53 4 3.63 0 3.3 1.42 0 3.26 0 0.8 0.14
Ori
gin
al/B
oo
ste
d
So
lva
ble
?
# G
TA
s
# C
ou
rse
s
To
tal
ca
pa
cit
y (C
)
To
tal
load
(L
)
Rat
io=
C \
L
Un
as
sig
ne
d C
ou
rse
s
So
luti
on
Qu
ali
ty
Un
us
ed
GT
As
Ava
ila
ble
Re
sou
rce
CC
(×
10
8 )
Un
as
sig
ne
d C
ou
rse
s
So
luti
on
Qu
ali
ty
Un
us
ed
GT
As
Ava
ila
ble
Re
sou
rce
CC
(×
10
8 )
Un
as
sig
ne
d C
ou
rse
s
So
luti
on
Qu
ali
ty
Un
us
ed
GT
As
Ava
ila
ble
Re
sou
rce
CC
(×
10
8 )
Observations:- Only ERA finds complete solutions to all solvable instances- On unsolvable problems, ERA leaves too many unused GTAs- LS and BT exhibit similar behaviors
Empirical study 2- Solvable vs unsolvable
15
20
25
30
35
40
45
50
55
60
65
70
1 20 39 58 77 96 115 134 153 172 191
iteration
# ag
ents
in z
ero
posi
tion
Spring2001b (B)
Fall2002 (B)
Fall2001b
Spring2003
10
15
20
25
30
35
40
45
1 20 39 58 77 96 115 134 153 172 191
iteration
# ag
ents
in z
ero
posi
tion
Spring2001b (O)
Fall 2002 (O)
ERA performance on solvable problems
ERA performance on unsolvable problems
Observation:
- Number of agents in zero-position per iteration
- ERA behavior differs on solvable vs. unsolvable instances
Empirical study 3- Behavior of individual agents
0
20
40
1 51 101 151 201 251 301 351 401 451
0
10
20
30
1 51 101 151 201 251 301 351 401 451
0
10
20
1 51 101 151 201 251 301 351 401 451
va ria ble
s ta ble
c o ns ta nt
ind
ex
of
po
sit
ion
iteration
Instances• solvable • unsolvable
Motion of agents• variable• stable• constant
Observations:
Solvable Unsolvable
Variable None Most
Stable A few A few
Constant Most None
Empirical study 4- Deadlock
– Each circle corresponds to a given GTA – Each square represents an agent– A blank squares indicate that an agent is on a zero-position– The squares with same color indicate agents involved in a deadlock
Observation:
ERA is not able to avoid deadlocks and yields a degradation of the solution on unsolvable CSPs.
Discussion
Goal Actions
Control Schema Undoing assignments Conflict resolution
ERALocal
+ Immune to local optima
– May yield instability
√+ Flexible
+ Solves tight CSPs
Non-committal– Deadlock
– Shorter solutions
LS
Global+ Stable behavior
– Liable to local optima
×+ Quickly stabilizes
– Fails to solve tight CSPs even with randomness & restart strategies
Heuristic
+ Longer solutions
BT
Systematic+ Stable behavior
– Thrashes
~+ Quickly stabilizes
– Fails to solve tight CSPS even with backtracking & restart strategies
+ advantages – shortcomings
Dealing with the deadlock
Possible approaches:— Direct communications, negotiation mechanisms— Hybrids of search Global control Conflict resolution
Experiments:— Enhancing ERA with global control
– Don’t accept a move that deteriorates the global goal– Lead to local-search-like behavior (i.e., local optima)
— ERA with conflict resolution– add dummy resources– find a complete solution when LS and BT fail – remove dummy assignments, solutions are still better
Future research directions
– Enhance ERA to handle optimization problems – Test approach using other search techniques
– BT search: Randomized, credit-based– Other local repair: squeaky-wheel method – Market-based techniques, etc.
– Validate conclusions on other CSPs– random instances, real-world problems
– Try search-hybridization techniquesReferences:R. Glaubius and B.Y. Choueiry, Constraint Modeling and Reformulation in the Context of Academic Task Assignment. In Workshop Modeling and Solving Problems with Constraints, ECAI 2002.
J. Liu, H. Jing, and Y.Y. Tang. Multi-Agent Oriented Constraint Satisfaction. Artificial Intelligence, 136:101-144, 2002.
Questions
Recommended