Exploiting Deadline Flexibility in Grid Workflow Rescheduling Wei Chen Alan Fekete Young Choon Lee

Exploiting Deadline Flexibility in Grid Workflow Rescheduling

Wei Chen

Alan Fekete

Young Choon Lee

Agenda

• IntroductionIntroduction

• Deadline Guaranteed Rescheduling

• Workflow Scheduling

• Task Rescheduling

• Performance Study

• Conclusion

Computational Grid and Workflow Application

• Computational Grid:– Heterogeneous Computing Site (Resource Instance)– Advance Reservation

5

4

32

1

• Workflow Application– Directed Acyclic Graph (DAG)– Job (V, E), where V is the set of tasks and

E is directed edges represent precedence constraints between corresponding tasks

}......,,{ 21 nRRRGrid

Grid Workflow Scheduling

• List scheduling heuristics

• Heterogeneous Earliest-Finish-Time (HEFT)– Greedy Best-First Strategy– It lacks an overall consideration in scheduling different

workflow jobs

Agenda

• Introduction

• Deadline Guaranteed ReschedulingDeadline Guaranteed Rescheduling




• Conclusion

The Approach we build on: Deadline Guaranteed Rescheduling (DGR)

• Deadline-based scheduling: it allows each job to come with a deadline, and from this, each task of the job can be placed more flexibly (not only at the earliest possible timeslot)

• A rescheduling mechanism: the tasks of an earlier job might be rearranged to other time slots or resource instances, giving extra resource availability for more urgent tasks

3

4

An Example of Scheduling and Rescheduling Workflow Jobs

Deadline (B)

Deadline (A)4

32

1

5

2

(A) 1

(B)

(a)

A1

A2A

3

A4

A5

R1 R2 R3

(b)

A1

A2

B3

A3

A4

A5

B1

B2

B4

R1 R2 R3

(c)

A1

A2

B3

A3

A4

A5

B1

B2

B4

R1 R2 R3

A2

The Key Points of Our Approach

• First, our approach loosely distributes tasks along the time axis according to the deadline of the workflow job, but not squeezes them on the earliest finish time. It is more flexible in rescheduling to allow urgent tasks get required resource availability.

• Second, our approach is not to reconsider schedules of the whole job again. Each task is rescheduled within a time slot boundary so that it does not affect the current schedules of all its predecessors and successors. This simplifies the complexity of our algorithm.

• Third, our rescheduling can be made not only in time dimension (another time slot), but also in space dimension (different resource instances). This increases the flexibility in rescheduling.

• Our rescheduling is to rearrange advance reservations of tasks before they are submitted for execution. This approach does not incur the cost in task migration.

Agenda

• Introduction


• Workflow SchedulingWorkflow Scheduling



• Conclusion

• Weighted DAG

Task Deadlines

(6) )}(')(')(min{)( ,)( ijjjiiji eDTvETvdeadlinevdeadlinevsuccessorvVv

(5) ' ' /

(4) )}(max{

(3) )()}()(max{)( ),( ,

(2) /)()(D ,

(1) /)()( ,

bandwidthnetwork average the:

speedn computatio average the:

ratioDTDTratioETETmakespandeadlineratio

VvvEFTmakespan

vETeDTvEFTvEFTvrpredecessovVv

BedataeTEe

SvworkloadvETVv

ii

iijjiiji

Bijijij

Siii

• An advisable deadline for each task

• The deadline of a workflow job can be guaranteed if all of its tasks are finished before their deadlines. These advisable deadlines reasonably balance the time for each task based on their workload proportions.

Scheduling Algorithm

Input a DAG Output scheduling of the job

calculate deadlines for each task; rank tasks into a priority list

for each task in the list do

schedule task within its deadline

if it fails then

schedule task in the earliest finish time

if this finish time > job’s deadline then break the loop

end if

end for

if scheduling is not done then

rollback schedules have been made

for each task in the list do

schedule task in the earliest finish time

if this finish time > job’s deadline then reject the job

end for

end if

Agenda

• Introduction



• Task ReschedulingTask Rescheduling


• Conclusion

Time Slot Boundary

• The time slot boundary is calculated when a task tries to be rescheduled on a specific resource instance

• At the moment, the actual schedules of the task’s predecessors and successors are known

• Since the target resource is specified, the actual network bandwidths between the resource instance and that of the task’s predecessors or successors are also known

})()()(min{),(

})()()(max{),(

)( ),( ,

ikikki

ijijji

ikiji

ebandwidthedatavASTRvLFT

ebandwidthedatavAFTRvEST

vsuccessorvvrpredecessovVv

TTT

Bipartite Graph Matching

• We make all tasks one part of nodes T (no matter which workflow job the task belongs to), and all resource instances the other part R.

• Every task is linked with all its satisfiable resources. The arrow of the line shows whether the task has been scheduled on (or matched with) a resource instance, which is represented by an arrow pointing to the task.

1

(a)

R

2

3

1

2

1

(b)

R

2

3

1

2

1

(c)

R

2

3

1

2

Rescheduling Algorithm

Input a task Output scheduling of the task

push the task into an empty stack S

while S is not empty pop a task from S for each satisfiable resource of the task do calculate EST and LFT if it can be scheduled in the boundary then return: the scheduling else if a task can be removed then push it into S end if end forend while

return: scheduling fails

Agenda

• Introduction




• Performance StudyPerformance Study

• Conclusion

Experiment Setup

• Heterogeneous Grid– 1,000 heterogeneous computing sites– Different setting in resource properties, computation

capacity and speed– Computing sites are fully connected by varying

network bandwidths• Workflow Jobs

– various sizes and parallelism degrees– both computation intensive and communication

intensive ones– some are more urgent than others

Acceptance Rate

Overall Acceptance Rate

20%

40%

60%

80%

100%

0 100 200 300

Number of Jobs Submitted

HEFTDGRDGR-L

Resource Utilization

Resource Utilization

0%

20%

40%

60%

80%

100%

0 100 200 300

Number of Submitted Jobs

HEFTDGRDGR-L

Running Time of Algorithms

Running Time

0

50

100

150

200

250

300

0 100 200 300

Number of Submitted Jobs

Tim

e (m

s)

HEFTDGRDGR-L

Agenda

• Introduction





• ConclusionConclusion

Conclusion

• A deadline-based strategy to schedule and reschedule workflow jobs; individual tasks can be rescheduled, based on the requirements of later jobs as they arrive.

• The approach satisfies Grid users as more jobs can be finished before their deadlines, and it also benefits the Grid owner by improving resource utilization.

• By using appropriate heuristics, the cost of the scheduling decision-making is quite acceptable and scalable to a large number of tasks scheduled in the system.

Thanks

Questions

Documents

Exploiting Deadline Flexibility in Grid Workflow Rescheduling Wei Chen Alan Fekete Young Choon Lee