Scheduling Manufacturing Systems with Agents: A Game Theory Based Negotiation Approach

Scheduling Manufacturing Systems Scheduling Manufacturing Systems with Agents: A Game Theory Based with Agents: A Game Theory Based Negotiation ApproachNegotiation Approach

By

Astrid Babayan

Department of Mathematics and EngineeringHarold Washington College

January 2005

Harold Washington College - Math Colloquium 2005 2

What is IE?What is IE?

• Industrial Engineering started early in the twentieth century with the applications of the scientific methods in factories. Because of its initial factory orientation, this engineering discipline became known as industrial production or management engineering. Industrial Engineering is frequently defined as the integration of systems of machines, people, materials, money, and methods. While these key components still play an extremely important role in I.E., the meanings of these terms have became far more general in scope. A common misconception is that all I.E.'s work in factories. Information, energy and computers have become equally important components, causing the applications of I.E. to expand beyond factories to hospitals and other health-care operations, transportation, consulting companies, food processing companies, computer/computer software companies, organizations, media operations, service companies, such as banking and utilities, and divisions of local, regional, and national governments. I.E.'s are often suited to hold positions traditionally held by Business Administration and Mechanical Engineering majors.


Research ObjectivesResearch Objectives

• Develop multi-agent based large scale scheduling problem solving framework with incorporated cooperation mechanism ruled by the game theoretic regulations.

• Develop objective measures of effectiveness for evaluating the developed model.

• Verified and validated the effectiveness of the developed methodology by comparing the results with those provided by other researchers.


Scheduling ProblemScheduling Problem

Given:• a set of activities that must be executed;• a set of resources with which to perform the activities;• a set of constraints which must be satisfied; and• a set of objectives with which to judge a schedule’s

performance.Find:• The best way to assign the resources to the activities at

specific times such that all of the constraints are satisfied and the best objective measures are produced.


Scaling Issues - The Size of the ProblemScaling Issues - The Size of the Problem

Using the nomenclature of Van Dyke Parunak, scheduling problems consist of asking what must be done where and when.

• Tasks (what) to complete.• Resources (where) operate on.• Specific periods of time (when).


Assembly Subdivision Representation of a Assembly Subdivision Representation of a ProductProduct

Par

ts p

rod

uct

ion

Sub-assembly

Pro

du

ct

Production progress in assembly

Pro

du

ct c

omp

lexi

ty

Sub-assembly

Sub-assembly

Sub-assembly

Sub-assembly

Sub-assembly


Classification of Scheduling ProblemsClassification of Scheduling Problems

Simple Digraph (Gs):one assembly operation at each assembly level

P 3

P 2

P 1

P

1

P

2

P

3

A

1

A

2



Complex Digraph (Gc):more than one assembly operation at least in one assembly level

P 1

P 2

P 3

P 4

P 5

P

1

P

2

P 5

A

1

A

3

P

3

P

4

A

2



N-product scheduling problem: N multiple products are produced in the system. In solving the N-product scheduling problem, the assembly sequence of a product could be either a simple digraph or a complex digraph.

Product 1 Product N. . .


NN-Product Scheduling -Product Scheduling GcGc Scheduling Scheduling

Construct a complex digraph by connecting the assembly nodes of N products to a dummy final assembly node, Ad. Let t(Ad)=0.

Product 1

.

.

.

Product N

t(Ad)=0

.

.

. Ad


General Structure of {General Structure of {mm, , qq} Manufacturing } Manufacturing SystemSystem


ConstraintsConstraints

Precedence relations among the activities represented by the digraph should not be violated.


Scheduling Problem FormulationScheduling Problem Formulation

Given the manufacturing system structure and product

assembly structure representation by a digraph assign

parts and subassemblies/assemblies to the machines at

the machining and assembly stages and determine the

processing sequences on the machines so that the

makespan), i.e., the maximum completion time Cmax, is

minimized.


Solution Approaches Developed for Solution Approaches Developed for Solving the Scheduling ProblemsSolving the Scheduling Problems

Exact Solution Methods• Mathematical Programming

• Linear Programming• Integer Programming• Dynamic Programming • Bounded Enumeration

(modifications of BB)

Heuristic Solution Methods• Dispatching Rules• Artificial Intelligence

Techniques• Simulated Annealing• Neural Networks• Genetic Algorithm • Fuzzy Logic

• Hybrid Methods


Problem Instances SolvedProblem Instances Solved

• Machining-Driven Product Differentiation Strategy

• Assembly-Driven Product Differentiation Strategy

Assembly Machine

Raw material Parts Finished products

Machine 1

Machine 2

Machine m

.

.

.

Machining

Assembly


NP-Completeness ResultsNP-Completeness Results

• System structure {m=2, q=1}• Jobs:

Lee et al. (1993) showed that 3MAF scheduling with makespan minimization is NP-complete by reducing it to 3-PARTITION problem.

Our problem was reduced to 3MAF by constructing an instance for any {m, q} system where either m > 1 or q > 1. Hence our problem is NP-complete too.


Why Agent-Based Approach?Why Agent-Based Approach?

• can simplify problem solving by splitting the problem into simple tasks;

• can tolerate uncertain data and knowledge; • offer conceptual clarity and simplicity of design; • allow incremental modification of the system boundary;

and • suit well to distributed problems.


Designing Agent-Based SystemsDesigning Agent-Based Systems

Basic strategy: Decompose & Distribute Reduces complexity of a task less capable agents, and fewer resources needed.

Decomposition can be done: •Physically: based on the layout of the info sources or decision points. •Functionally: according to the expertise of available agents.

Distribution can be done according to:•Avoid overloading critical resources.•Assign tasks to agents with matching capabilities.•Assign resources to agents with matching capabilities.


Properties of Agents Used in the System Properties of Agents Used in the System

• autonomy: agents operate without the direct intervention of human operators, and have high degree of computational capabilities;

• sociability: agents interact with other agents via some kind of agent communication language;

• reactivity: agents perceive their acting environment, and respond to the changes that occur there;

• pro-activity: agents do not simply act in response to the environment, they are able to exhibit goal oriented behavior by taking initiatives;

• veracity: agent will not knowingly communicate false information;• benevolence: agents do not have conflicting goals, and every agent will

therefore try to do what is asked of it; and• rationality: agent will act in order to achieve its goals, and will not in

such a way as to prevent its goals being achieved.


Agent-Based Scheduling FrameworkAgent-Based Scheduling Framework

Digraph representation of product assembly

structure

Manufacturing system structure

Final schedule Game theory based agent cooperation

Scheduling subproblems of individual agents

Product assembly structure decomposition and agent

identification


Digraph Decomposition Digraph Decomposition


Scheduling Problem DecompositionScheduling Problem Decomposition

SAJ0={J3, J4, J5}.

NSJ0={J1, J2}

SJ0={}

P3

P4

P5

P6

P7

A3

A4

A2

A5

P1

P2

A1

Agent 4Job 4

Agent 2Job 2

Agent 3Job 3

Agent 5Job 5

Agent 1Job 1


Definitions and NotationsDefinitions and Notations

Define:J = (J1, J2, J3, …, Jn) the set of the jobs to be scheduled;SAJ = set of schedulable jobs. NSJ = set of non-schedulable jobs. SJ = set of scheduled jobs.

Let stage be a step in the scheduling process, when a change in the set of schedulable/scheduled jobs occurs.

The sets SAJ, NSJ, SJ will be defined for stage t as: SAJt, NSJt, SJt


Definitions and Notations (cont.)Definitions and Notations (cont.)

At any point of a time the followings hold:

JSJNSJSAJ ttt

tt NSJSAJ

tt SJSAJ

tt NSJSJ


Agents Interaction in the Scheduling Agents Interaction in the Scheduling ProcessProcess

Outer game Inner game

Set of

candidate feasible jobs

Set of

infeasible jobs

Set of scheduled jobs

Schedule Manager

mig

rati

on

relevant info sharing

migration

relevant info sharing


What Is a Game?What Is a Game?

“A game is a description of strategic interaction that includes the constraints on the actions that the players can take and the players’ interests, but does not specify the actions that the players do take.”

- Osborne and Rubenstein


What Is Game Theory?What Is Game Theory?

“Game theory is a bag of analytical tools designed to help us understand the phenomena that we observe when decision-makers interact.”

– Osborne and Rubenstein


Elements of Game TheoryElements of Game Theory

• Players with similar or different interests.• Actions available to each player.• Consequences related to players’ choices.

Examples:• Chess, go, tic-tac-toe, poker, spades.• Presidential elections, wars, political negotiations• Airline fare setting, business decisions• Channel allocation, bandwidth allocation, other engineering

problems


Two Major Branches of Game TheoryTwo Major Branches of Game Theory

• Non-cooperative game theory includes a detailed study of the strategies available to the players.

• Cooperative game theory is concerned with those situations in which players can negotiate about what to do in the game.


Cooperative Game TheoryCooperative Game Theory

• Players are going to play a game (in the future).

• Before the game, the players are able to negotiate and sign a binding contract regarding which strategies they will play.

• The result of their negotiations is the subject of cooperative game theory.


Two Classes of Cooperative GamesTwo Classes of Cooperative Games

• N person transferable utility games: N-TU• N person non-transferable utility games: N-NTU

Depending weather or not players have comparable units of utility.

applied to problem


How a N-TU Game is Defined? How a N-TU Game is Defined?

• Set of Players/Agents:

N={1, 2, …n}

• Characteristic Function:

Payoff v(S) for each coalition S N of the players’ set N


Solution Concepts in Cooperative Game Solution Concepts in Cooperative Game TheoryTheory

• Core

• Shapley Value

Efficiency:

Symmetry:

Dummy:

Additivity:

}NS ,)( and )(:),...,,({ 21 Si

iNi

in SvxNvxxxxxC

)()(!

!||!1||)( iSvSv

n

SnSv

SiNS

i

Ni

i Nvv )()(

}){(}){(, and , if )()( jSviSvSjiNSvv ji

0)( then , )(}){( if vSiNSSviSv i)()()( vuvu


Solution Concepts in Cooperative Game Solution Concepts in Cooperative Game Theory (cont.)Theory (cont.)

• Banzhaf-Coleman Power Index

Average of player i’s marginal contribution

SiNS

n

i iSvSvv })]{()([2

1)(

1

• Nucleolus

sallocationefficient ofset )}(:{1

NvxxXn

ii

)()(, if nucleolus is xOvOXxXv L

NSxSvxSeSi

i

where)(),(


Shapley ValueShapley Value

General formula for calculating Shapley value:

For our problem reduces to:

Percentage contribution:

)()(!

!||!1||)( iSvSv

n

SnSv

SiNS

i

})({)(}){(2

1)( ivSviSvvi

)(}){()( viSvv iS

}){(

)(

)()(

)(

iSv

v

vv

v i

is

i


Defining Outer GameDefining Outer Game

Initializationt=0

SAJt, NSJt, and SJt.

Agent in SAJ make their

individual schedules: Cmax,i

k=index(max i SAJ

{Cmax,i})

agent k enters

inner game

t=t+1

SAJt, NSJt, and SJt.

SAJ= End of outer game

k=index(max i SAJ ShVi/Cmax)

yes

no


Defining Inner gameDefining Inner game

Coalition: SN

Payoff: v(S)

i S

Calculate v((S-{i} )i*)

i* is a new strategy by i

k=index(max i S ShVi/Cmax)

Agent k reschedules job k

Terminate rescheduli

ng

Inner game is stabilized

yes

no


Lower Bound on MakespanLower Bound on Makespan

LB = solution obtained from MIP relaxation of the original problem

Since the optimization is MIN problem, the following holds:

maxCCLB


Agent-Based Scheduling - CodingAgent-Based Scheduling - Coding

Agent Class properties:• Knowledge of the system structure• Knowledge of his/her own job/task structure• Knowledge of system state

• Completion time on each assembly/fabrication machine

• Knowledge of the state of sets SAJ, NSJ, SJ• Knowledge of his/her preceding agents completion time• Knowledge of formulating and solving MIP formulations

• Solver DLL by Frontline Systems


Example : Agent-Based Scheduling Example : Agent-Based Scheduling

Assembly structure and machining and assembly times of a product.

Agent 1 Job 1

A2

P3

A4

P1

P2

A1

P4

P5

P6

A3Agent 2Job 2

Agent 3Job 3

Part # P1 P2 P3 P4 P5 P6

Machining Time 6 4 6 1 6 9

Assembly # A1 A2 A3 A4 - -

Assembly Time 3 1 3 5 - -

Set of jobs is defined: J = {J1, J2, J3}


Example : Agent-Based Scheduling (Cont.) Example : Agent-Based Scheduling (Cont.)

t=0

SAJ0={J2, J3}

NSJ0={J1=J2&J3}

SJ0={}

a3 is the winner t=1

SAJ1={J3}

NSJ1={J1=J2&J3}

SJ1={J2}

v(3)=12, v(S)=11, v(S+3)=19ShV(3)=(v(S+3)-v(S)+v(3)) /2=10Percentage contribution=10/19=0.555


Test Problem Generation – {1, 1} SystemTest Problem Generation – {1, 1} System

(i) MT ~ U(7, 25) ; AT ~ U(10, 30) (ii) MT ~ U(6, 14) ; AT ~ U(6, 19) (iii) MT ~ U(3, 12) ; AT ~ U(4, 12). (iv) MT ~ U(5, 16) ; AT ~ U(6, 20)

The average size of the problems corresponds to 37 part nodes, 26 assembly nodes, and 7 assembly levels.

The data was generated based on the real assembly application information from industrial assembly handbooks (e.g., Nof et al., 1996; Boothroyd, 1992; Lotter, 1989).


Analysis of Results – {1, 1}Analysis of Results – {1, 1}

Average over 10 instances of each problem type.


Test Problem Generation – {Test Problem Generation – {mm, , qq} System} System

(i) MT ~ U(3, 12) ; AT ~ U(4, 5)(ii) MT ~ U(4, 9) ; AT ~ U(6, 11) (iii) MT ~ U(2, 14) ; AT ~ U(7, 18)

The average size of the problems corresponds to 45 part nodes, 30 assembly nodes, and 5 assembly levels.

{m, q}={3, 3}; {7, 7}; {20, 20}.


Analysis of Results – {Analysis of Results – {mm, , qq}}

Average over 10 instances of each problem type.

{3,3} {7,7} {20,20} Problem CAB/CLB CAB/CLB CAB/CLB Type 1 1.152 1.394 1.651 Type 2 1.408 1.676 1.317 Type 3 1.395 1.520 1.232


N-Job 3-Machine Flexible Flowshop N-Job 3-Machine Flexible Flowshop SchedulingScheduling

• System Structure

jobs jobs finished jobs

Machine 11

Machine 21

Machine m11

Machine 12

Machine 22

Machine m23

Machine 13

Machine 23

Machine m33

Stage 1 Stage 2 Stage 3

jobs


NN-Job 3-Machine Flexible Flowshop -Job 3-Machine Flexible Flowshop SchedulingScheduling

• Task Structure


Test Problem Generation - ComparisonTest Problem Generation - Comparison

(i) balanced load:

Pj,1, Pj,2, Pj,3 ~ U(1, 100)

(ii) light load on stage 1:

Pj,1~U(1, 20), Pj,2, Pj,3 ~ U(1, 100)

(iii) light load on stage 2:

Pj,1, Pj,3 ~ U(1, 100), Pj,2~U(1, 20)


Comparison of ResultsComparison of ResultsEqual load Light load at stage 1 Light load at stage 2

n m1, m2, m3 CAB/CLB SEmin CAB/CLB SEmax CAB/CLB SEmin 2,2,2 1.15 1.20 1.08 1.16 1.07 1.13 2,2,3 1.10 1.16 1.04 1.17 1.03 1.09 2,3,2 1.13 1.14 1.11 1.15 1.07 1.12 2,3,3 1.05 1.10 1.12 1.25 1.03 1.08 3,2,2 1.18 1.16 1.08 1.10 1.10 1.10 3,2,3 1.13 1.17 1.04 1.13 1.11 1.17 3,3,2 1.16 1.10 1.11 1.11 1.10 1.09

30

3,3,3 1.21 1.27 1.11 1.18 1.10 1.20 2,2,2 1.12 1.22 1.04 1.18 1.04 1.10 2,2,3 1.08 1.16 1.02 1.20 1.02 1.05 2,3,2 1.08 1.13 1.07 1.17 1.04 1.13 2,3,3 1.03 1.06 1.07 1.28 1.01 1.05 3,2,2 1.13 1.15 1.05 1.12 1.07 1.07 3,2,3 1.08 1.17 1.02 1.13 1.07 1.15 3,3,2 1.12 1.05 1.07 1.10 1.07 1.06

50

3,3,3 1.16 1.27 1.06 1.19 1.18 1.16


Definitions and NotationsDefinitions and Notations


Definitions and Notations (cont.)Definitions and Notations (cont.)


Mixed-Integer Programming FormulationMixed-Integer Programming Formulation

Min CT(An) (1) Subject to:

m

lkj

N

kjj

jklii PtPtqAtACT1 1

)()()()( for iki APPAPA , (2)

1)(1

jik

m

kijk qq for i=1,…, N, j=1,…, N, ji (3)

)()()( iji AtACTACT for i=1,…, n, )( ij AIPA (4)

11

q

jijx i=1,…, n (5)


Mixed-Integer Programming Formulation Mixed-Integer Programming Formulation (cont.)(cont.)

)()(1

i

q

jij ACTACT

i = 1 , … , n - 1 ( 6 )

)1( ijiji xM)(ACT)CT(A f o r i = 1 , … , n - 1 , j = 1 , … , q ( 7 )

ikjkkjkjij My)A(tx)A(CT)A(CT ( 8 )

)y(M)A(CT)A(tx)A(CT ikjijiijkj 1 ( 9 )

f o r i = 1 , … , n - 1 , j = 1 , … , q , )A(NAA ik

1or 0 and , , ikjijkij yqx ( 1 0 )

a l l o t h e r v a r i a b l e s a r e n o n n e g a t i v e . ( 1 1 )

Documents

Scheduling Manufacturing Systems with Agents: A Game Theory Based Negotiation Approach