USC Viterbi School of Engineering

Ewa Deelman

Resource Management

USC Viterbi School of Engineering

Runtime-based Scheduling

• Priority Queue Phase– Eligible jobs are determined – based on fairness, system

limitations, job state, etc...

– The list of eligible jobs is then sorted by job priority and jobs are scheduled to run until a job cannot be scheduled due to resource limitations

– Resources and start/end times are reserved for the first job that could not be scheduled

• Backfill Scheduling Phase– Jobs following the reserved job which would not delay

the reserved job are scheduled on the remaining idle rsources

USC Viterbi School of Engineering

Single Queue Backfilling

• A job is allowed to jump in the queue ahead of jobs that are delayed due to lack of resources

• Non-preemptive • Conservative backfilling

– A job is allowed to jump ahead provided it does not delay any previous job in the queue

• Aggressive backfilling– A job is allowed to jump ahead if the first job is not

affected

– Better performing then conservative backfilling

USC Viterbi School of Engineering

Aggressive backfilling

• Job (arrival time, number of procs, expected runtime)

• Any job that executes longer than expected runtime is killed

• Define pivot—first job in the queue• If enough resources for pivot, execute pivot and

define a new pivot• Otherwise sort all currently executing jobs in

order of their expected completion time– Determine pivot time—when sufficient resources for

pivot are available

USC Viterbi School of Engineering

Aggressive backfilling

• At pivot time, any idle processors not required for pivot job are defined as extra processors

• The scheduler searches for the first queued job that– Requires no more than the currently idle processors and

will finish by the pivot time, or

– Requires no more than the minimum currently idle processors and the extra processors

• Once a job becomes a pivot, it cannot be delayed• Possible that a job will end sooner and pivot starts

before pivot time

USC Viterbi School of Engineering

Aggressive backfilling example

Job A(10 procs,40mins)

8

time

QueueJob B (12 procs, 1 hour)Job C (20 procs, 2 hours)Job D (2 procs, 50 mins)Job E (6 procs, 1 hour)Job F (4 procs, 2 hours)

10

processors

Running Job A(10 procs, 40 mins)

USC Viterbi School of Engineering

Aggressive backfilling example

Job A(10 procs,40mins)

8

time

queueJob B (12 procs, 1 hour)Job C (20 procs, 2 hours)Job E (6 procs, 1 hour)Job F (4 procs, 2 hours)

10

processorsPIVOTJob B

JOB D

Running Job A(10 procs, 40 mins)Job D (2 procs, 50 mins)

USC Viterbi School of Engineering

Aggressive backfilling example

QueueJob B (12 procs, 1 hour)Job C (20 procs, 2 hours)Job E (6 procs, 1 hour)

Job A(10 procs,40mins)

8

time

10

processorsPIVOTJob B

Job D

Running Job A(10 procs, 40 mins)Job D (2 procs, 50 mins)Job F (4 procs, 2 hours)

Job F

USC Viterbi School of Engineering

Aggressive backfilling example

QueueJob C (20 procs, 2 hours)Job E (6 procs, 1 hour)

Job A(10 procs,40mins)

8

time

10

processorsJob B

Job DJob F

Running Job D (2 procs, 50 mins)Job F (4 procs, 2 hours)Job B (12 procs, 1 hour)

USC Viterbi School of Engineering

Aggressive backfilling example

QueueJob C (20 procs, 2 hours)Job E (6 procs, 1 hour)

Job A(10 procs,40mins)

8

time

10processors

Job B

Job DJob F

Running Job D (2 procs, 50 mins)Job F (4 procs, 2 hours)Job B (12 procs, 1 hour)

PivotJob C (20 procs, 2 hours)