Scheduling Parameter Scheduling Parameter Sweep ApplicationsSweep Applications

Sathish VadhiyarSathish VadhiyarSources/Credits: Papers on survey of Sources/Credits: Papers on survey of heuristis and APST papers. Figures heuristis and APST papers. Figures

taken from the paperstaken from the papers

BackgroundBackground

Tasks of a job do not have Tasks of a job do not have dependenciesdependenciesA machine executes a single task at a A machine executes a single task at a time – space shared machinestime – space shared machinesCollection of tasks and machines are Collection of tasks and machines are known aprioriknown aprioriMatching of tasks to machines done Matching of tasks to machines done offlineofflineEstimates of execution time for each Estimates of execution time for each task on each machine is knowntask on each machine is known

Scheduling ProblemScheduling Problem

ETC – Expected time to compute matrixETC – Expected time to compute matrix

ETC(i,j) – estimated execution time of task ETC(i,j) – estimated execution time of task i on machine ji on machine j

Notations:Notations: mat(j) – machine availability time for machine mat(j) – machine availability time for machine

j, i.e., earliest time at which j has completed all j, i.e., earliest time at which j has completed all tasks that were previously assigned to ittasks that were previously assigned to it

Completion time, ct(i,j) = mat(j)+ETC(i,j)Completion time, ct(i,j) = mat(j)+ETC(i,j)

Objective – find max ct(i, j), makespan, Objective – find max ct(i, j), makespan, and find heuristic with minimum makespanand find heuristic with minimum makespan

Scheduling HeuristicsScheduling Heuristics

1.1. Opportunistic Load Balancing (OLB)Opportunistic Load Balancing (OLB)• Assign next task (arbitrary order) to the next Assign next task (arbitrary order) to the next

available machineavailable machine• Regardless of task’s ETC on that machineRegardless of task’s ETC on that machine

2.2. User Directed Allocation (UDA)User Directed Allocation (UDA)• Assign next task (arbitrary order) to the Assign next task (arbitrary order) to the

machine with lowest ETCmachine with lowest ETC• Regardless of machine availabilityRegardless of machine availability

3.3. Fast greedyFast greedy• Assign each task (arbitrary order) to the Assign each task (arbitrary order) to the

machine with minimum ctmachine with minimum ct

Scheduling HeuristicsScheduling Heuristics

4.4. Min-MinMin-Min• Start with a list of Unmapped tasks, U.Start with a list of Unmapped tasks, U.• Determine the set of minimum completion times for U.Determine the set of minimum completion times for U.• Choose the next task that has min of min completion Choose the next task that has min of min completion

times and assign to the machine that provides the min. times and assign to the machine that provides the min. completion time.completion time.

• The new mapped task is removed from U and the The new mapped task is removed from U and the process is repeated.process is repeated.

• Theme - Map as many tasks as possible to their first Theme - Map as many tasks as possible to their first choice of machinechoice of machine

• Since short jobs are mapped first, the percentage of Since short jobs are mapped first, the percentage of tasks that are allocated to their first choice is hightasks that are allocated to their first choice is high

Scheduling HeuristicsScheduling Heuristics

5. 5. Max-MinMax-Min• Start with a list of Unmapped tasks, U.Start with a list of Unmapped tasks, U.• Determine the set of minimum completion times for U.Determine the set of minimum completion times for U.• Choose the next task that has max of min completion Choose the next task that has max of min completion

times and assign to the machine that provides the min. times and assign to the machine that provides the min. completion time.completion time.

• The new mapped task is removed from U and the The new mapped task is removed from U and the process is repeated.process is repeated.

• Avoid starvation of long tasksAvoid starvation of long tasks• Long tasks executed concurrently with short tasksLong tasks executed concurrently with short tasks• Better machine-utilization Better machine-utilization

6.6. GreedyGreedy• Combination of max-min and min-minCombination of max-min and min-min• Evaluates both and finds the better solutionEvaluates both and finds the better solution

Scheduling HeuristicsScheduling Heuristics

Genetic AlgorithmGenetic Algorithm

GAGA

Operates 200 chromosomes. A chromosome Operates 200 chromosomes. A chromosome represents a mapping of task to machines, a represents a mapping of task to machines, a vector of size t.vector of size t.Initial population – 200 chromosomes randomly Initial population – 200 chromosomes randomly generated with 1 Min-Min seedgenerated with 1 Min-Min seedEvaluation – initial population evaluated based on Evaluation – initial population evaluated based on fitness value (makespan)fitness value (makespan)Selection – Selection –

Roulette wheel – probabilistically generate new Roulette wheel – probabilistically generate new population, with better mappings, from previous population, with better mappings, from previous populationpopulation

Elitism – guaranteeing that the best solution (fittest) is Elitism – guaranteeing that the best solution (fittest) is carried forwardcarried forward

GA - Roulette wheel schemeGA - Roulette wheel scheme

ChromosomesChromosomes 11 22 33 44

ScoreScore 44 1010 1414 22

Probability ofProbability of 0.130.13 0.33 0.33 0.47 0.07 0.47 0.07

selectionselection

Select a random number, r, between 0 and Select a random number, r, between 0 and 1.1.

Progressively add the probabilities until the Progressively add the probabilities until the sum is greater than rsum is greater than r

GAGA

CrossoverCrossover Choose pairs of chromosomes.Choose pairs of chromosomes. For every pairFor every pair

Choose a random pointChoose a random pointexchange machine assignments from that point till the end exchange machine assignments from that point till the end of the chromosomeof the chromosome

Mutation. For every chromosome:Mutation. For every chromosome: Randomly select a taskRandomly select a task Randomly reassign it to new machineRandomly reassign it to new machine

EvaluationEvaluationStopping criterion:Stopping criterion:

Either 1000 iterations orEither 1000 iterations or No change in elite chromosome for 150 iterationsNo change in elite chromosome for 150 iterations

Simulated AnnealingSimulated AnnealingPoorer solutions accepted Poorer solutions accepted with a probability that with a probability that depends on temperature depends on temperature valuevalueInitial mappingInitial mappingInitial temperature – initial Initial temperature – initial makespanmakespanEach iteration:Each iteration:

Generate new mapping Generate new mapping based on mutation of prev. based on mutation of prev. mapping. Obtain new mapping. Obtain new makespanmakespan

If new makespan better, If new makespan better, acceptaccept

If new makespan worse, If new makespan worse, accept if a random number accept if a random number z in [0,1] > y wherez in [0,1] > y where

Reduce temperature by Reduce temperature by 10%10%

Genetic Simulated Annealing Genetic Simulated Annealing (GSA)(GSA)

Almost same as GAAlmost same as GADuring selection, SA is employed to form During selection, SA is employed to form new populationnew populationInitial system temperature – average Initial system temperature – average makespan of the populationmakespan of the populationEach iteration of GAEach iteration of GA Post-mutation or post-crossover, during a Post-mutation or post-crossover, during a

comparison of chromosome with the previous comparison of chromosome with the previous chromosomechromosome

if (new makespan) < (old makespan + if (new makespan) < (old makespan + temperature), new chromosome becomes part temperature), new chromosome becomes part of the populationof the population

Temperature decreased by 10%Temperature decreased by 10%

Tabu searchTabu search

Keeps track of regions of solution space that have already Keeps track of regions of solution space that have already been searchedbeen searchedStarts with a random mappingStarts with a random mappingGenerate all possible pairs of tasks, (i,j), t in (0, t-1) and j in Generate all possible pairs of tasks, (i,j), t in (0, t-1) and j in (i+1, t)(i+1, t)i and j’s machine assignments are exchanged (short hop) i and j’s machine assignments are exchanged (short hop) and makespan evaluatedand makespan evaluatedIf makespan better (successful short hop), search begins If makespan better (successful short hop), search begins from i=0, else search continues from previous (i,j)from i=0, else search continues from previous (i,j)Continue until 1200 successful short hops or all pairs have Continue until 1200 successful short hops or all pairs have been evaluatedbeen evaluatedAdd final mapping to tabu list. The list keeps track of Add final mapping to tabu list. The list keeps track of solution space searchedsolution space searchedA new random mapping generated that differs from solution A new random mapping generated that differs from solution space by atleast half the machine assignments (long hop)space by atleast half the machine assignments (long hop)Search continued until fixed number of short and long hopsSearch continued until fixed number of short and long hops

A Comparison Study of Static A Comparison Study of Static Mapping Heuristics for a Class of Mapping Heuristics for a Class of

Meta-Tasks on Heterogeneous Meta-Tasks on Heterogeneous Computing SystemsComputing Systems Tracy D. Tracy D.

Braun et. alBraun et. al

Simulation ResultsSimulation Results

ETC matrix randomly generated using uniform ETC matrix randomly generated using uniform distributiondistributionETC matrices may beETC matrices may be

Consistent – m/c i faster than m/c j for all tasks. Each Consistent – m/c i faster than m/c j for all tasks. Each row is sorted across all columns.row is sorted across all columns.

Inconsistent – not sortedInconsistent – not sorted Semi-consistent – sorted across even columns.Semi-consistent – sorted across even columns.

t=512, m=16t=512, m=16OLB, UDA, Fast Greedy and Greedy – few secondsOLB, UDA, Fast Greedy and Greedy – few secondsSA, tabu – 30 secondsSA, tabu – 30 secondsGA, GSA – 60 secondsGA, GSA – 60 secondsA* - 20 minutesA* - 20 minutes

Task execution resultsTask execution results

Consistent casesConsistent cases UDA performs worst in consistent cases – why?

Inconsistent casesInconsistent cases UDA improves because “best” machines are distributed UDA improves because “best” machines are distributed

avoiding load imbalanceavoiding load imbalance Fast greedy and Min-Min improve upon UDA since they Fast greedy and Min-Min improve upon UDA since they

consider MCTs and MCTs are also evenly distributedconsider MCTs and MCTs are also evenly distributed Tabu search gave poor performance for inconsistent Tabu search gave poor performance for inconsistent

cases since there are more successful short hops than cases since there are more successful short hops than long hops.long hops.

In both cases, Min-Min performed better than Max-In both cases, Min-Min performed better than Max-Min due to the nature of the task mixMin due to the nature of the task mixIn both cases, GA’s performed the best.In both cases, GA’s performed the best.

All tasks were assigned the same machine

AppLeS Parameter AppLeS Parameter Sweep Template (APST)Sweep Template (APST)

APSTAPST

For efficient deployment of parameter For efficient deployment of parameter sweep applications on the Gridsweep applications on the GridDistinct experiments share large input files Distinct experiments share large input files and produce large output filesand produce large output filesShared data files must be co-located with Shared data files must be co-located with experimentsexperimentsPSA – set of independent tasksPSA – set of independent tasks Input – set of files, a single file can be input to Input – set of files, a single file can be input to

more than 1 taskmore than 1 task Output – each task produces exactly one outputOutput – each task produces exactly one output Number of computational tasks in PSA orders of Number of computational tasks in PSA orders of

magnitude greater than number of processorsmagnitude greater than number of processors

Scheduling HeuristicsScheduling Heuristics

Self-scheduled workqueueSelf-scheduled workqueue

Adaptive scheduling algorithm – assigns Adaptive scheduling algorithm – assigns tasks to nodes as soon as they are tasks to nodes as soon as they are available in a greedy fashionavailable in a greedy fashion

Suitable:Suitable: If no large input filesIf no large input files Large clusters interconnected with high-speed Large clusters interconnected with high-speed

clustersclusters Computation to data-movement times are highComputation to data-movement times are high

AlgorithmAlgorithm

Gantt ChartGantt Chart

Step 4Step 4

HeuristicsHeuristicsMin-minMin-min

f – minimum of CTf – minimum of CTi,ji,j Best – minimumBest – minimum

Max-minMax-min f – minimum of CTf – minimum of CTi,ji,j Best – maximumBest – maximum

SufferageSufferage f – ratio between second minimum and minimumf – ratio between second minimum and minimum Best – maximumBest – maximum a host should be given to a task that would “suffer” the a host should be given to a task that would “suffer” the

most if not given the hostmost if not given the host

XSufferageXSufferage Site-levelSite-level Cluster-level MCTs and cluster-level sufferageCluster-level MCTs and cluster-level sufferage Avoids deficiencies of sufferage when a task’s input file Avoids deficiencies of sufferage when a task’s input file

is in a cluster and 2 hosts in the cluster have identical is in a cluster and 2 hosts in the cluster have identical performanceperformance

Sufferage and XSufferageSufferage and XSufferageHost k in cluster j

Sufferage and XSufferageSufferage and XSufferage

Sample APST SetupSample APST Setup

Impact of Quality of Information on Impact of Quality of Information on SchedulingScheduling

Random noise [-p,+p] ; p – 0%-100% Random noise [-p,+p] ; p – 0%-100% added to accuracy of estimatesadded to accuracy of estimates

ResultsResults

ResultsResults

Robust Static Allocation of Resources for Independent Robust Static Allocation of Resources for Independent Tasks…-Sugavanam et. al., JPDC 2007Tasks…-Sugavanam et. al., JPDC 2007

ETC numbers are just estimates and inaccurateETC numbers are just estimates and inaccurate

Need to map tasks to maximize the robustness Need to map tasks to maximize the robustness of makespan against estimation errorsof makespan against estimation errors

Robustness?Robustness? Degradation in makespan is within acceptable limits Degradation in makespan is within acceptable limits

when estimates are perturbedwhen estimates are perturbed

The goal is to maximize the collective allowable The goal is to maximize the collective allowable error in estimation without makespan exceeding error in estimation without makespan exceeding the constraintthe constraint

Problem FormulationProblem Formulation

CCestest – vector of estimated execution times on a – vector of estimated execution times on a machinemachineC – vector of actual timesC – vector of actual timesPerformance feature, Ø that determines the Performance feature, Ø that determines the robustness of the makespan is the finish times of robustness of the makespan is the finish times of machines = {Fmachines = {Fjj, 1<j<M}, 1<j<M}

Robustness radius for machine j and for Robustness radius for machine j and for mapping mapping μμ: minimum Euclidean distance : minimum Euclidean distance between C and Cbetween C and Cestest within which the finish time within which the finish time of machine j can reach tolerable variationof machine j can reach tolerable variation

Robustness RadiusRobustness Radius

Within this robustness radius, the finish Within this robustness radius, the finish time of machine j will be atmost the time of machine j will be atmost the makespan constraint, tau.makespan constraint, tau.

The above equation can be interpreted as The above equation can be interpreted as the perpendicular distance from cthe perpendicular distance from cestest to the to the hyperplane, tau-Fhyperplane, tau-Fjj(C) = 0(C) = 0

Rewritten as:Rewritten as:

Robustness MetricRobustness Metric

Robustness metric for the mapping:Robustness metric for the mapping:

If Euclidean distance between actual and If Euclidean distance between actual and estimated is no larger than the above metric, the estimated is no larger than the above metric, the makespan will be atmost the constraint, taumakespan will be atmost the constraint, tau

The larger the robustness metric, better the The larger the robustness metric, better the mappingmapping

Thus the problem is to maximize the metric such Thus the problem is to maximize the metric such that the makespan is within the time constraintthat the makespan is within the time constraint

Heuristics – Max-MaxHeuristics – Max-Max

Greedy Iterative Maximization Greedy Iterative Maximization (GIM)(GIM)

Greedy Iterative Maximization Greedy Iterative Maximization (GIM)(GIM)

Sum Iterative Maximization (SIM)Sum Iterative Maximization (SIM)

“Robustness improvement” – change in the sum of the robustness radii of the machines after task reassignment or swapping

Sum Iterative Maximization (SIM)Sum Iterative Maximization (SIM)

GenitorGenitor

GenitorGenitor

Memetic AlgorithmMemetic Algorithm

Combines global search using genetic algorithm Combines global search using genetic algorithm and local search using hill climbingand local search using hill climbing

MemeticMemetic

HereBoy Evolutionary AlgorithmHereBoy Evolutionary Algorithm

Combines GA and SACombines GA and SA

HereBoy Evolutionary AlgorithmHereBoy Evolutionary Algorithm

User defined maximum mutation rate

Reduces the mutation as the current robustness reaches upper bound

Probability of accepting a poorer solution

User defined maximum probability

UB calculation for HereboyUB calculation for Hereboy

Assumes a homogeneous MET system – Assumes a homogeneous MET system – execution time for a task on all machines execution time for a task on all machines is equal to minimum execution time on the is equal to minimum execution time on the original set of machinesoriginal set of machines

The tasks are arranged in ascending orderThe tasks are arranged in ascending order

N=T/M. The first N tasks are stored in a N=T/M. The first N tasks are stored in a set Sset S

GIM and SIM had low makespan and high robustness

ReferencesReferencesThe AppLeS Parameter Sweep Template: User-Level Middleware for the The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid by Grid by Henri Casanova, Graziano Obertelli, Francine BermanHenri Casanova, Graziano Obertelli, Francine Berman and and Rich Rich wolski wolski Proceedings of the Super Computing Conference (SC'2000).Proceedings of the Super Computing Conference (SC'2000).Heuristics for Scheduling Parameter Sweep applications in Grid Heuristics for Scheduling Parameter Sweep applications in Grid environments by environments by Henri Casanova, Arnaud Legrand, Dmitrii ZagorodnovHenri Casanova, Arnaud Legrand, Dmitrii Zagorodnov and and Francine Berman Francine Berman in Proceedings of the 9th Heterogeneous Computing in Proceedings of the 9th Heterogeneous Computing workshop (HCW'2000), pp349-363.workshop (HCW'2000), pp349-363.A Comparison Study of Static Mapping Heuristics for a Class of Meta-Tasks A Comparison Study of Static Mapping Heuristics for a Class of Meta-Tasks on Heterogeneous Computing Systems on Heterogeneous Computing Systems Tracy D. Braun, Howard Jay Siegel, Tracy D. Braun, Howard Jay Siegel, Noah Beck, Ladislau L. Bölóni, Albert I. Reuther, Mitchell D. Theys, Bin Yao, Noah Beck, Ladislau L. Bölóni, Albert I. Reuther, Mitchell D. Theys, Bin Yao, Richard F. Freund, Muthucumaru Maheswaran, James P. Robertson, Richard F. Freund, Muthucumaru Maheswaran, James P. Robertson, DDebra ebra HensgenHensgen, Eighth Heterogeneous Computing Workshop , April 12 - 12, 1999, , Eighth Heterogeneous Computing Workshop , April 12 - 12, 1999, San Juan, Puerto Rico Page 15San Juan, Puerto Rico Page 15Robust static allocation of resources for independent tasks under makespan Robust static allocation of resources for independent tasks under makespan and dollar cost constraints. and dollar cost constraints. Journal of Parallel and Distributed Journal of Parallel and Distributed ComputingComputing, , Volume 67, Issue 4Volume 67, Issue 4, , April 2007April 2007, , Pages 400-416. Pages 400-416. Prasanna Prasanna Sugavanam, H.J. Siegel, Anthony A. Maciejewski, Mohana Oltikar, Ashish Sugavanam, H.J. Siegel, Anthony A. Maciejewski, Mohana Oltikar, Ashish Mehta, Ron Pichel, Aaron Horiuchi, Vladimir Shestak, Mohammad Al-Otaibi, Mehta, Ron Pichel, Aaron Horiuchi, Vladimir Shestak, Mohammad Al-Otaibi, Yogish Krishnamurthy, Yogish Krishnamurthy, et al.et al.

Simulation BackgroundSimulation Background

ETC(i,j) = B(i)*xETC(i,j) = B(i)*xrrijij

B(i) = xB(i) = xbbii; x; xbb

ii ЄЄ [1, [1, ФФbb]]

xxrrijij ЄЄ [1, [1, ФФrr]]

ФФrr, , ФФb b can control machine and task can control machine and task heterogeneityheterogeneity

ETC matrix randomly generated ETC matrix randomly generated using uniform distributionusing uniform distribution

Models used in APSTModels used in APST

SchedulingScheduling

Goal:Goal: Minimize application’s makespan - time between when Minimize application’s makespan - time between when

11stst input file submitted and when last output file is input file submitted and when last output file is obtainedobtained

Sched():Sched(): Takes into account resource performance estimates to Takes into account resource performance estimates to

generate a plan for assigning file transfers to links and generate a plan for assigning file transfers to links and tasks to hoststasks to hosts

Call sched repeatedly at various point of time (called Call sched repeatedly at various point of time (called scheduling events) to make it more adaptivescheduling events) to make it more adaptive

At each scheduling event, sched() knows:At each scheduling event, sched() knows: Grid topologyGrid topology Number and locations of copies of input filesNumber and locations of copies of input files List of computations and file transfers currently List of computations and file transfers currently

underway or already completedunderway or already completed

APST ArchitectureAPST Architecture

NWS also used for forecasting ETC values