Spring 2002 IE 514 1

IE 514 Production Scheduling

Introduction

Spring 2002 IE 514 2

Contact Information

Siggi Olafsson3018 Black Engineering294-8908olafsson@iastate.eduhttp://www.public.iastate.edu/

~olafssonOH: MW 10:30-12:00

Spring 2002 IE 514 3

Administration (Syllabus)

TextPrerequisitesAssignments

Homework 35% Project 40% Final Exam 35%

Computing

Spring 2002 IE 514 4

What is Scheduling About?

Applied operations research Models Algorithms

Solution using computers Implement algorithms Draw on common databases Integration with other systems

Spring 2002 IE 514 5

Application Areas

Procurement and production

Transportation and distribution

Information processing and communications

Spring 2002 IE 514 6

Manufacturing Scheduling

Short product life-cyclesQuick-response manufacturingManufacture-to-order

More complex operations must be scheduled in shorter amount of time with less room for errors!

Spring 2002 IE 514 7

Scope of Course

Levels of planning and scheduling Long-range planning (several years), middle-range planning (1-2 years), short-range planning (few months), scheduling (few weeks), and reactive scheduling (now)

These functions are now often integrated

Spring 2002 IE 514 8

Scheduling Systems

Enterprise Resource Planning (ERP) Common for larger businesses

Materials Requirement Planning (MRP) Very common for manufacturing companies

Advanced Planning and Scheduling (APS) Most recent trend Considered “advanced feature” of ERP

Spring 2002 IE 514 9

Scheduling Problem

Allocate scarce resources to tasksCombinatorial optimization problem

Maximize profitSubject to constraints

Mathematical techniques and heuristics

Spring 2002 IE 514 10

Our Approach

Scheduling Problem

Model

Conclusions

Problem Formulation

Solve with Computer Algorithms

Spring 2002 IE 514 11

Scheduling Models

Project schedulingJob shop schedulingFlexible assembly systems

Lot sizing and schedulingInterval scheduling, reservation,

timetablingWorkforce scheduling

Spring 2002 IE 514 12

General Solution Techniques

Mathematical programming Linear, non-linear, and integer programming

Enumerative methods Branch-and-bound Beam search

Local search Simulated annealing/genetic

algorithms/tabu search/neural networks.

Spring 2002 IE 514 13

Scheduling System Design

Databases

Schedule generation

User interfaces

Database Management

Automatic Schedule Generator

Schedule EditorPerformance Evaluation

Graphical Interface

User

Order master file

Shop floordata collection

Spring 2002 IE 514 14

LEKIN

On disk with bookGeneric job shop scheduling systemUser friendly windows environmentC++ object oriented designCan add own routines

Spring 2002 IE 514 15

Advanced Topics

Uncertainty, robustness, and reactive scheduling

Multiple objectives

Internet scheduling

Spring 2002 IE 514 16

Topic 1

Setting up the Scheduling Problem

Spring 2002 IE 514 17

Modeling

Three components to any model: Decision variables

This is what we can change to affect the system, that is, the variables we can decide upon

Objective functionE.g, cost to be minimized, quality measure to be

maximized

ConstraintsWhich values the decision variables can be set

to

Spring 2002 IE 514 18

Decision “Variables”

Three basic types of solutions:

A sequence: a permutation of the jobs

A schedule: allocation of the jobs in a more

complicated setting of the environment

A scheduling policy: determines the next

job given the current state of the system

Spring 2002 IE 514 19

Model Characteristics

Multiple factors: Number of machine and resources, configuration and layout, level of automation, etc.

Our terminology:Resource = machine (m)Entity requiring the resource = job (n)

Spring 2002 IE 514 20

Notation

Static data: Processing time (pij)

Release date (rj)

Due date (dj)

Weight (wj)

Dynamic data: Completion time (Cij)

Spring 2002 IE 514 21

Machine Configuration

Standard machine configurations: Single machine models Parallel machine models Flow shop models Job shop models

Real world always more complicated.

Spring 2002 IE 514 22

Constraints

Precedence constraintsRouting constraintsMaterial-handling constraintsStorage/waiting constraintsMachine eligibilityTooling/resource constraintsPersonnel scheduling constraints

Spring 2002 IE 514 23

Other Characteristics

Sequence dependent setupPreemptions

preemptive resume preemptive repeat

Make-to-stock versus make-to-order

Spring 2002 IE 514 24

Objectives and Performance Measures

Throughput (TP) and makespan (Cmax)

Due date related objectives

Work-in-process (WIP), lead time

(response time), finished inventory

Others

Spring 2002 IE 514 25

Throughput and Makespan

Throughput Defined by bottleneck machines

Makespan

Minimizing makespan tends to maximize throughput and balance load

niCCCC

CCCC

imiii

n

,...,1,,...,,max

,...,,max

21

21max

Spring 2002 IE 514 26

Due Date Related Objectives

LatenessMinimize maximum lateness (Lmax)

TardinessMinimize the weighted tardiness

jjj dCL

0,max jjj dCT

n

jjjTw

1

Spring 2002 IE 514 27

Due Date Penalties

jdjC

jL

jdjC

jT

jdjC

jU1

jdjC

In practice

Tardiness

Late or Not

Lateness

Spring 2002 IE 514 28

WIP and Lead Time

Work-in-Process (WIP) inventory costMinimizing WIP also minimizes average

lead time (throughput time)Minimizing lead time tends to minimize

the average number of jobs in systemEquivalently, we can minimize sum of

the completion times:

n

ijC

1

n

jjjCw

1

Spring 2002 IE 514 29

Other Costs

Setup cost

Personnel cost

Robustness

Finished goods inventory cost

Spring 2002 IE 514 30

Topic 2

Solving Scheduling Problems

Spring 2002 IE 514 31

Classic Scheduling Theory

Look at a specific machine environment with a specific objective

Analyze to prove an optimal policy or to show that no simple optimal policy exists

Thousands of problems have been studied in detail with mathematical proofs!

Spring 2002 IE 514 32

Example: single machine

Lets say we have Single machine (1), where the total weighted completion time

should be minimized (wjCj)

We denote this problem as

jjCw||1

Spring 2002 IE 514 33

Optimal Solution

Theorem: Weighted Shortest Processing time first - called the WSPT rule - is optimal for

Note: The SPT rule starts with the job that has the shortest processing time, moves on the job with the second shortest processing time, etc.

jjCw||1

Spring 2002 IE 514 34

Proof (by contradiction)

Suppose it is not true and schedule S is optimal

Then there are two adjacent jobs, say job j followed by job k such that

Do a pairwise interchange to get schedule S ’

k

k

j

j

p

w

p

w

j k

k j

kj ppt

kj ppt

t

t

Spring 2002 IE 514 35

Proof (continued)

The weighted completion time of the two jobs under S is

The weighted completion time of the two jobs under S ‘ is

Now:

Contradicting that S is optimal.

jkjkk wpptwpt )()(

kkjjj wpptwpt )()(

jjkkk

kkjkjj

kkkjjjkkjjj

wpptwpt

wptwpwpt

wptwpwptwpptwpt

)()(

)()(

)()()()(

Spring 2002 IE 514 36

Complexity Theory

Classic scheduling theory draws heavily on complexity theory

The complexity of an algorithm is its running time in terms of the input parameters (e.g., number of jobs and number of machines)

Big-Oh notation, e.g., O(n2m)

Spring 2002 IE 514 37

Polynomial versus NP-Hard

)(log nO

)(nO)( 2nO)2( nO

)( size Problem n

Time

Spring 2002 IE 514 38

Scheduling in Practice

Practical scheduling problems cannot be solved this easily!

Need: Heuristic algorithms Knowledge-based systems Integration with other enterprise functions

However, classic scheduling results are useful as a building block

Spring 2002 IE 514 39

General Purpose Scheduling Procedures

Some scheduling problems are easy Simple priority rules Complexity: polynomial time

However, most scheduling problems are hard Complexity: NP-hard, strongly NP-hard Finding an optimal solution is infeasible

in practice heuristic methods

Spring 2002 IE 514 40

Types of Heuristics

Simple Dispatching RulesComposite Dispatching RulesBranch and BoundBeam SearchSimulated AnnealingTabu SearchGenetic Algorithms

ConstructionMethods

ImprovementMethods

Spring 2002 IE 514 41

Topic 3

Dispatching Rules

Spring 2002 IE 514 42

Dispatching Rules

Prioritize all waiting jobs job attributes machine attributes current time

Whenever a machine becomes free: select the job with the highest priority

Static or dynamic

Spring 2002 IE 514 43

Release/Due Date Related

Earliest release date first (ERD) rule variance in throughput times

Earliest due date first (EDD) rule maximum lateness

Minimum slack first (MS) rule

maximum lateness 0,max tpd jj

CurrentTime

ProcessingTime

Deadline

Spring 2002 IE 514 44

Processing Time Related

Longest Processing Time first (LPT) rule balance load on parallel machines makespan

Shortest Processing Time first (SPT) rule sum of completion times WIP

Weighted Shortest Processing Time first (WSPT) rule

Spring 2002 IE 514 45

Processing Time Related

Critical Path (CP) rule precedence constraints makespan

Largest Number of Successors (LNS) rule precedence constraints makespan

Spring 2002 IE 514 46

Other Dispatching Rules

Service in Random Order (SIRO) ruleShortest Setup Time first (SST) rule

makespan and throughputLeast Flexible Job first (LFJ) rule

makespan and throughputShortest Queue at the Next

Operation (SQNO) rule machine idleness

Spring 2002 IE 514 47

Discussion

Very simple to implementOptimal for special casesOnly focus on one objectiveLimited use in practiceCombine several dispatching rules

Composite Dispatching Rules

Spring 2002 IE 514 48

Example

Single Machine with Weighted Total Tardiness

Spring 2002 IE 514 49

Setup

Problem: No efficient algorithm (NP-Hard)Branch and bound can only solve

very small problems (<30 jobs)

Are there any special cases we can solve?

jjTw||1

Spring 2002 IE 514 50

Case 1: Tight Deadlines

Assume dj=0 Then

We know that WSPT is optimal for this problem!

jjjj

jj

jjj

CwTw

CC

dCT

,0max

,0max

Spring 2002 IE 514 51

Conclusion

The WSPT is optimal in the extreme case and should be a good heuristic whenever due dates are tight

Now lets look at the opposite

Spring 2002 IE 514 52

Case 2: “Easy” Deadlines

Theorem: If the deadlines are sufficiently spread out then the MS rule

is optimal (proof a bit harder)Conclusion: The MS rule should be a

good heuristic whenever deadlines are widely spread out

0,maxarg tpdj jjselect

Spring 2002 IE 514 53

Composite Rule

Two good heuristics Weighted Shorted Processing Time (WSPT )

optimal with due dates zero

Minimum Slack (MS)Optimal when due dates are “spread out”

Any real problem is somewhere in betweenCombine the characteristics of these

rules into one composite dispatching rule

Spring 2002 IE 514 54

Apparent Tardiness Cost (ATC) Dispatching RuleNew ranking index

When machine becomes free: Compute index for all remaining jobs Select job with highest value

Scaling constant

)(

0,maxexp)(

tpK

tpd

p

wtI jj

j

jj

Spring 2002 IE 514 55

Special Cases (Check)If K is very large:

ATC reduces to WSPT

If K is very small and no overdue jobs: ATC reduces to MS

If K is very small and overdue jobs: ATC reduces to WSPT applied to overdue jobs

Spring 2002 IE 514 56

Choosing K

Value of K determined empiricallyRelated to the due date tightness

factor

and the due date range factor

max

1C

d

max

minmax

C

ddR

Spring 2002 IE 514 57

Choosing K

Usually 1.5 K 4.5Rules of thumb:

Fix K=2 for single machine or flow shop. Fix K=3 for dynamic job shops.

Adjusted to reduce weighted tardiness cost in extremely slack or congested job shops

Statistical analysis/empirical experience

Spring 2002 IE 514 58

Topic 4

Branch-and-Bound& Beam Search

Spring 2002 IE 514 59

Branch and Bound

Enumerative methodGuarantees finding the best scheduleBasic idea:

Look at a set of schedules Develop a bound on the performance Discard (fathom) if bound worse than

best schedule found before

Spring 2002 IE 514 60

Example

Single Machine with Maximum Lateness Objective

Spring 2002 IE 514 61

Classic Results

The EDD rule is optimal forIf jobs have different release dates,

which we denote

then the problem is NP-HardWhat makes it so much more

difficult?

max||1 L

max||1 Lrj

Spring 2002 IE 514 62

max||1 Lrj

Jobs 1 2 3

jp 4 2 5

jr 0 3 5

jd 8 14 10

1 2 3

1r 2r 3r

0 5 10 15

1d 3d 2d

55,4,4max1015,1410,84max

,,max,,max 332211321max

dCdCdCLLLL

EDD

Can weimprove?

Spring 2002 IE 514 63

Delay Schedule

1 23

1r 2r 3r

0 5 10 15

1d 3d 2d

30,2,4max1010,1416,84max

,,max,,max 332211321max

dCdCdCLLLL

Add a delay

What makes this problem hard is thatthe optimal schedule is not necessarilya non-delay schedule

Spring 2002 IE 514 64

Final Classic Result

The preemptive EDD rule is optimal for the preemptive (prmp) version of the problem

Note that in the previous example, the preemptive EDD rule gives us the optimal schedule

max|,|1 Lprmprj

Spring 2002 IE 514 65

Branch and Bound

The problem

cannot be solved using a simple dispatching rule so we will try to solve it using branch and bound

To develop a branch and bound procedure: Determine how to branch Determine how to bound

max||1 Lrj

Spring 2002 IE 514 66

Data

Jobs 1 2 3 4

jp 4 2 6 5

jr 0 1 3 5

jd 8 12 11 10

Spring 2002 IE 514 67

Branching

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

Spring 2002 IE 514 68

Branching

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

Discard immediately because

5

3

4

3

r

r

Spring 2002 IE 514 69

Branching

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

Need to develop lower bounds onthese nodes and do further branching.

Spring 2002 IE 514 70

Bounding (in general)

Typical way to develop bounds is to relax the original problem to an easily solvable problem

Three cases: If there is no solution to the relaxed problem there

is no solution to the original problem If the optimal solution to the relaxed problem is

feasible for the original problem then it is also optimal for the original problem

If the optimal solution to the relaxed problem is not feasible for the original problem it provides a bound on its performance

Spring 2002 IE 514 71

Relaxing the Problem

The problem

is a relaxation to the problem

Not allowing preemption is a constraint in the original problem but not the relaxed problem

We know how to solve the relaxed problem (preemptive EDD rule)

max||1 Lrj

max|,|1 Lprmprj

Spring 2002 IE 514 72

Bounding

Preemptive EDD rule optimal for the preemptive version of the problem

Thus, solution obtained is a lower bound on the maximum delay

If preemptive EDD results in a non-preemptive schedule all nodes with higher lower bounds can be discarded.

Spring 2002 IE 514 73

Lower Bounds

Start with (1,•,•,•): Job with EDD is Job 4 but Second earliest due date is for Job 3

54 r

Job 1

Job 2

Job 3

Job 4

0 10 20

5max L

Spring 2002 IE 514 74

Branching

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

(1,2,•,•) (1,3,•,•)

5max L

7max L

6max L5max L(1,3,4,2)

Spring 2002 IE 514 75

Beam Search

Branch and Bound: Considers every node Guarantees optimum Usually too slow

Beam search Considers only most promising nodes (beam width) Does not guarantee convergence Much faster

Spring 2002 IE 514 76

Single Machine Example

(Total Weighted Tardiness)

Jobs 1 2 3 4

jp 10 10 13 4

jd 4 2 1 12

jw 14 12 1 12

Spring 2002 IE 514 77

Branching(Beam width = 2)

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

(1,4,2,3) (2,4,1,3) (3,4,1,2) (4,1,2,3)

ATC Rule

408 436 814 440

Spring 2002 IE 514 78

Beam Search

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

(1,2,•,•) (1,3,•,•) (1,4,•,•) (2,1,•,•) (2,3,•,•) (2,4,•,•)

(2,4,3,1)(2,4,1,3)(1,4,3,2)(1,4,2,3)

Spring 2002 IE 514 79

Discussion

Implementation tradeoff: Careful evaluation of nodes

(accuracy) Crude evaluation of nodes (speed)

Two-stage procedure Filtering

crude evaluationfilter width > beam width

Careful evaluation of remaining nodes

Spring 2002 IE 514 80

Topic 5

Random Search

Spring 2002 IE 514 81

Construction versus Improvement Heuristics

Construction Heuristics Start without a schedule Add one job at a time Dispatching rules and beam search

Improvement Heuristics Start with a schedule Try to find a better ‘similar’ schedule

Can be combined

Spring 2002 IE 514 82

Local Search

0. Start with a schedule s0 and set k = 0.

1. Select a candidate schedule

from the neighborhood of sk

2. If acceptance criterion is met, let

Otherwise let3. Let k = k+1 and go back to Step 1.

ksNs

.1 ssk

.1 kk ss

Spring 2002 IE 514 83

Defining a Local Search Procedure

Determine how to represent a schedule.

Define a neighborhood structure.

Determine a candidate selection

procedure.

Determine an acceptance/rejection

criterion.

Spring 2002 IE 514 84

Neighborhood Structure

Single machine A pairwise interchange of adjacent jobs

(n-1 jobs in a neighborhood) Inserting an arbitrary job in an arbitrary

position( n(n-1) jobs in a neighborhood)

Allow more than one interchange

Spring 2002 IE 514 85

Neighborhood Structure

Job shops with makespan objective Neighborhood based on critical paths Set of operations start at t=0 and end at

t=Cmax

Machine 1

Machine 2

Machine 3

Machine 4

Spring 2002 IE 514 86

Neighborhood Structure

The critical path(s) define the neighborhood Interchange jobs on a critical path Too few better schedules in the

neighborhood Too simple to be useful

Spring 2002 IE 514 87

One-Step Look-Back Interchange

Machine h

Machine i

Machine h

Machine i

Jobs on critical path

Interchange jobson critical path

Machine h

Machine i

One-step look-backinterchange

Spring 2002 IE 514 88

Neighborhood Search

Find a candidate schedule from the neighborhood: Random Appear most promising

(most improvement in objective)

Spring 2002 IE 514 89

Acceptance/Rejection

Major difference between most methods Always accept a better schedule? Sometimes accept an inferior schedule?

Two popular methods: Simulated annealing Tabu search

Same except the acceptance criterion!

Spring 2002 IE 514 90

Topic 6

Simulated Annealing

Spring 2002 IE 514 91

Simulated Annealing (SA)

Notation S0 is the best schedule found so far

Sk is the current schedule

G(Sk) it the performance of a schedule

Note that)()( 0SGSG k

Aspiration Criterion

Spring 2002 IE 514 92

Acceptance Criterion

Let Sc be the candidate schedule

If G(Sc) < G(Sk) accept Sc and let

If G(Sc) G(Sk) move to Sc with probability

ck SS 1

k

ck SGSG

ck eSSP )()(

1 ),(

Temperature > 0

Always 0

Spring 2002 IE 514 93

Cooling Schedule

The temperature should satisfy

Normally we let

but we sometimes stop when some predetermined final temperature is reached

If temperature is decreased slowly convergence is guaranteed

0...321

0k

Spring 2002 IE 514 94

SA AlgorithmStep 1:

Set k = 1 and select the initial temperature 1.

Select an initial schedule S1 and set S0 = S1.

Step 2:Select a candidate schedule Sc from N(Sk).

If G(S0) < G(Sc) < G(Sk), set Sk+1 = Sc and go to Step 3.

If G(Sc) < G(S0), set S0 = Sk+1 = Sc and go to Step 3.

If G(Sc) < G(S0), generate Uk Uniform(0,1);

If Uk P(Sk, Sc), set Sk+1 = Sc; otherwise set Sk+1 = Sk; go to Step 3.

Step 3:Select k+1 k.

Let k=k+1. If k=N STOP; otherwise go to Step 2.

Spring 2002 IE 514 95

Discussion

SA has been applied successfully to many industry problems

Allows us to escape local optimaPerformance depends on

Construction of neighborhood Cooling schedule

Spring 2002 IE 514 96

Topic 7

Tabu Search

Spring 2002 IE 514 97

Tabu Search

Similar to SA but uses a deterministic acceptance/rejection criterion

Maintain a tabu list of schedule changes A move made entered at top of tabu list Fixed length (5-9) Neighbors restricted to schedules not

requiring a tabu move

Spring 2002 IE 514 98

Tabu List

Rational Avoid returning to a local optimum

Disadvantage A tabu move could lead to a better

scheduleLength of list

Too short cycling (“stuck”) Too long search too constrained

Spring 2002 IE 514 99

Tabu Search AlgorithmStep 1:

Set k = 1. Select an initial schedule S1 and set S0 = S1.

Step 2:Select a candidate schedule Sc from N(Sk).

If Sk Sc on tabu list set Sk+1 = Sk and go to Step 3.

Enter Sc Sk on tabu list.

Push all the other entries down (and delete the last one).If G(Sc) < G(S0), set S0 = Sc.

Go to Step 3.

Step 3:Select k+1 k.

Let k=k+1. If k=N STOP; otherwise go to Step 2.

Spring 2002 IE 514 100

Single Machine Total Weighted Tardiness Example

Jobs 1 2 3 4

jp 10 10 13 4

jd 4 2 1 12

jw 14 12 1 12

Spring 2002 IE 514 101

Initialization

Tabu list empty L={}Initial schedule S1 = (2,1,4,3)

0 10 20 30 40

Deadlines

20 - 4 = 16

10 - 2 = 8

37 - 1 = 3624 - 12 = 12

50012123618121614jjTw

Spring 2002 IE 514 102

Neighborhood

Define the neighborhood as all schedules obtain by adjacent pairwise interchanges

The neighbors of S1 = (2,1,4,3) are

(1,2,4,3)(2,4,1,3)(2,1,3,4)

Objective values are: 480, 436, 652

Select the bestnon-tabu schedule

Spring 2002 IE 514 103

Second Iteration

Let S0 =S2 = (2,4,1,3). G(S0)=436.Let L = {(1,4)}New neighborhood

Sequence (4,2,1,3) (2,1,4,3) (2,4,3,1)

Twj 460 500 608

Spring 2002 IE 514 104

Third Iteration

Let S3 = (4,2,1,3), S0 = (2,4,1,3). Let L = {(2,4),(1,4)}New neighborhood

Sequence (2,4,1,3) (4,1,2,3) (4,2,3,1)

Twj 436 440 632

Tabu!

Spring 2002 IE 514 105

Fourth Iteration

Let S4 = (4,1,2,3), S0 = (2,4,1,3). Let L = {(1,2),(2,4)}New neighborhood

Sequence (1,4,2,3) (4,2,1,3) (4,1,3,2)

Twj 402 460 586

Spring 2002 IE 514 106

Fifth Iteration

Let S5 = (1,4,2,3), S0 = (1,4,2,3). Let L = {(1,4),(1,2)}This turns out to be the optimal

scheduleTabu search still continuesNotes:

Optimal schedule only found via tabu list We never know if we have found the optimum!

Spring 2002 IE 514 107

Topic 8

Genetic Algorithm

Spring 2002 IE 514 108

Genetic Algorithms

Schedules are individuals that form populations

Each individual has associated fitness

Fit individuals reproduce and have children in the next generation

Very fit individuals survive into the next generation

Some individuals mutate

Spring 2002 IE 514 109

Individual(sequence)

(2,1,3,4) (3,4,1,2) (4,1,3,2)

Fitness jjTw 652 814 586

Total Weighted Tardiness Single Machine Example

First Generation Initial populationselected randomly

Spring 2002 IE 514 110

Mutation Operator

Individual (2,1,3,4) (3,4,1,2) (4,1,3,2)

Fitness 652 814 586

(4,3,1,2)

Reproductionvia mutation

FittestIndividual

Spring 2002 IE 514 111

Cross-Over Operator

Individual (2,1,3,4) (4,3,1,2) (4,1,3,2)

Fitness 652 758 586

(4,1,3,2)

(2,1,3,4)

Reproductionvia cross-over

FittestIndividuals

(2,1,3,2)

(4,1,3,4)

Infeasible!

Spring 2002 IE 514 112

Representing Schedules

When using GA we often represent individuals using binary strings

(0 1 1 0 1 0 1 0 1 0 1 0 0 )(1 0 0 1 1 0 1 0 1 0 1 1 1 )

(0 1 1 0 1 0 1 0 1 0 1 1 1 )

Spring 2002 IE 514 113

Discussion

Genetic algorithms have been used very successfully

Advantages: Very generic Easily programmed

Disadvantages: A method that exploits special structure

is usually faster (if it exists)

Spring 2002 IE 514 114

Topic 9

Nested Partitions Method

Spring 2002 IE 514 115

The Nested Partitions MethodThe Nested Partitions Method(Shameless self-promotion!)(Shameless self-promotion!)

Partitioning of all schedules Similar to branch-and-bound & beam search

Random sampling & local search used to find which node is most promising Similar to beam search

Retains only one node (beam width = 1)Allows for possible backtracking

Spring 2002 IE 514 116

Single Machine Total Weighted Tardiness Example

Jobs 1 2 3 4

jp 10 10 13 4

jd 4 2 1 12

jw 14 12 1 12

Spring 2002 IE 514 117

First Iteration

(•,•,•,•)

(1,•,•,•) (2,•,•,•) (3,•,•,•) (4,•,•,•)

(1,4,2,3) 402

Random Samples (uniform sampling, ATC rule, genetic algorithm)

(1,2,4,3) 480(1,4,3,2) 554

I(1,•,•,•) = 402Promising Index

Most promising region

Spring 2002 IE 514 118

Notation

We let(k) be the most promising region in the k-th

iterationj(k) be the subregions, j=1,2,…,M

M+1(k) be the surrounding region

denote all possible regions (fixed set)0 all regions that contain a single schedule

I() be the promising index of

Spring 2002 IE 514 119

Second Iteration

(•,•,•,•)

(1,•,•,•) {(2,•,•,•),(3,•,•,•), (4,•,•,•)}

(1,3,•,•) (1,4,•,•)(1,2,•,•) (4,2,1,3) 460

Random Samples

(2,4,1,3) 436(4,1,3,2) 586

Most promising region (2)

Best Promising IndexI(4(2)) = 436

4(2)

1(2) 2(2) 3(2)

s(2))

Spring 2002 IE 514 120

Third Iteration

(•,•,•,•)

(1,•,•,•) {(1,2,•,•), (1,4,•,•),(2,•,•,•),(3,•,•,•), (4,•,•,•)}

(1,3,•,•) (4,2,3,1) 632

Random Samples

(2,4,1,3) 436(1,4,2,3) 402

(1,3,2,4) (1,3,4,2) I(3(3)) = 402

1(3) 2(3)

3(3)s(3))

(3)

Best Promising Index

Spring 2002 IE 514 121

Fourth Iteration

(•,•,•,•)

(1,•,•,•) {(2,•,•,•),(3,•,•,•), (4,•,•,•)}

(1,3,•,•) (1,4,•,•)(1,2,•,•)

4(4)

1(4) 2(4) 3(4)

s(4))

(4)

Backtracking!

Spring 2002 IE 514 122

Finding the Optimal Schedule

The sequence

is a Markov chainEventually the singleton region opt

containing the best schedule is visited Absorbing state (never leave opt) Finite state space visited after finitely many iterations.

1)( kk

Spring 2002 IE 514 123

DiscussionFinds best schedule in finite iterations

Only branch-and-bound can also guarantee

If we can calculate/estimate/guarantee the probability of moving in right direction: Expected number of iterations Probability of having found optimal schedule Stopping rules that guarantee performance

(unique)

Works best if incorporates dispatching rules and local search (genetic algorithms, etc)

Spring 2002 IE 514 124

Topic 10

Mathematical Programming

Spring 2002 IE 514 125

Review of Mathematical Programming

Many scheduling problems can be formulated as mathematical programs: Linear Programming (IE 534) Nonlinear Programming (IE 631) Integer Programming (IE 632)

See Appendix A in book.

Spring 2002 IE 514 126

Linear Programs

Minimizesubject to

nn xcxcxc ...2211

njx

bxaxaxa

bxaxaxa

bxaxaxa

j

nmnmm

nn

nn

,...,1for 0

...

...

...

12211

12222121

11212111

Spring 2002 IE 514 127

Solving LPs

LPs can be solved efficiently Simplex method (1950s) Interior point methods (1970s)

Polynomial timeHas been used in practice to solve

huge problemsIE 534

Spring 2002 IE 514 128

Nonlinear Programs

Minimize

subject to),...,,( 21 nxxxf

0),...,(

0),...,(

0),...,(

2,1

2,12

2,11

nm

n

n

xxxg

xxxg

xxxg

Spring 2002 IE 514 129

Solving Nonlinear Programs

Optimality conditions Karush-Kuhn-Tucker Convex objective, convex constraints

Solution methods Gradient methods (steepest descent,

Newton’s method) Penalty and barrier function methods Lagrangian relaxation

Spring 2002 IE 514 130

Integer Programming

LP where all variables must be integer

Mixed-integer programming (MIP)Much more difficult than LPMost useful for scheduling

Spring 2002 IE 514 131

Example: Single Machine

One machine and n jobsMinimize

Define the decision variables

n

j jjCw1

otherwise.0

at time starts job if1 tjx jt

Spring 2002 IE 514 132

IP Formulation

Minimize

subject to

n

j

C

tjtjj xptw

1

1

0

max

)(

tjx

tx

jx

jt

n

j

t

ptsjs

C

tjt

j

, 1,0

1

1

1

1

}0,max{

1

0

max

Spring 2002 IE 514 133

Solving IPs

Solution Methods Cutting plane methods

Linear programming relaxationAdditional constraints to ensure integers

Branch-and-bound methodsBranch on the decision variablesLinear programming relaxation provides

bounds

Generally difficult

Spring 2002 IE 514 134

Disjunctive Programming

Conjuctive All constraints must be satisfied

Disjunctive At least one of the constraints must be

satisfiedZero-one integer programsUse for job-shop scheduling