Upload
tania-monfort
View
216
Download
0
Embed Size (px)
Citation preview
Divisible Load Scheduling Divisible Load Scheduling
A TutorialA Tutorial
Thomas Robertazzi
University at Stony Brook
What is a Divisible Load?What is a Divisible Load?
A computational & networkable A computational & networkable load that is arbitrarily partitionable load that is arbitrarily partitionable (divisible) amongst processors and (divisible) amongst processors and links. links.
There are no precedence relations.There are no precedence relations.
Simple Application Simple Application ExampleExample
Problem: Sum 1000 trillion numbersProblem: Sum 1000 trillion numbers
Approach: Partition the numbers Approach: Partition the numbers among 100 processorsamong 100 processors
But how?But how?
Simple Application Simple Application ExampleExample
To optimize solution time (maximize To optimize solution time (maximize speedup) one needs to take into speedup) one needs to take into account heterogeneous link and account heterogeneous link and processor speeds, computation and processor speeds, computation and communication intensities, communication intensities, interconnection topology and interconnection topology and scheduling policy.scheduling policy.
Divisible Load Scheduling Theory Can Divisible Load Scheduling Theory Can Do This!Do This!
Applications (Generic)Applications (Generic)
Grid Computing/Meta-computingGrid Computing/Meta-computing Data Intensive ComputingData Intensive Computing Sensor Processing Sensor Processing Image ProcessingImage Processing Scientific/Engineering ComputingScientific/Engineering Computing Financial ComputingFinancial Computing
Applications (Specific)Applications (Specific)
Pattern SearchingPattern Searching Database ComputationDatabase Computation Matrix-Vector ComputationMatrix-Vector Computation E&M Field Calculation (CAD)E&M Field Calculation (CAD) Edge DetectionEdge Detection
DLT Modeling AdvantagesDLT Modeling Advantages
Linear and Deterministic Modeling Linear and Deterministic Modeling Tractable Recursive/Linear Tractable Recursive/Linear
Equation SolutionEquation Solution Schematic LanguageSchematic Language Equivalent ElementsEquivalent Elements Many ApplicationsMany Applications
Interconnection Interconnection TopologiesTopologies
Linear Daisy ChainLinear Daisy Chain BusBus Single Level and Multilevel TreesSingle Level and Multilevel Trees MeshMesh HypercubeHypercube
Directions: ScalabilityDirections: Scalability
1 32
1 11
Sequential Distribution(Saturation)
Simultaneous Distribution(Scalable)
Hung & RobertazziCheng & Robertazzi
An ExampleAn Example
Model Specifications:Model Specifications:• A star network( single level tree A star network( single level tree
network), and multi-level tree.network), and multi-level tree.• Computation and transmission time is Computation and transmission time is
a linear function of the size of load.a linear function of the size of load.• Level to Level: Store and Forward Level to Level: Store and Forward
SwitchingSwitching• Same Level: Concurrent Load Same Level: Concurrent Load
Distribution.Distribution.
Children without Front End: Children without Front End: After receiving the assigned data, each After receiving the assigned data, each
child proceeds to process the data.child proceeds to process the data.
Timing Diagram (single level tree) : Timing Diagram (single level tree) : Children without Front End Children without Front End
m+1 unknows vs. m+1 unknows vs. m+1 Eqs.m+1 Eqs.
Recursive equations:Recursive equations:
Normalization equation:Normalization equation:
miTwTzTwTz
TwTzTw
cpiicmiicpiicmii
cpcmcp
,...,3,2 1111
111100
1...210
m
Distribution Solution:Distribution Solution:
11
0
1101
2 21
1
11
12
1
Here
11
1
Here
,...,3,2
k
TzTwTwk
qk
TzTw
TzTwq
miq
cmcpcp
m
i
i
ll
cmicpi
cmicpii
i
lli
The load distribution solution is The load distribution solution is similar to the solution of the state-similar to the solution of the state-dependent M/M/1 queuing system.dependent M/M/1 queuing system.
Similarities to Queueing Similarities to Queueing TheoryTheory
Linear model and tractable solutionsLinear model and tractable solutions
Schematic LanguageSchematic Language
Equivalent ElelementsEquivalent Elelements
Infinite Size NetworksInfinite Size Networks
Speedup AnalysisSpeedup Analysis
m
i
i
ll
cp
cp
mf
f
qk
k
Tw
Tw
T
TSpeedup
2 21
10
00
0
,
0,
11
11
1
Speedup Analysis (continued)Speedup Analysis (continued)
11
10
01
1
1
12
1
11
Here
11
1 Here
,...,3,2
kmk
zTwTTwk
mk
zTwT
zTwTq
miq
cmcpcp
cmcp
cmcpi
i
lli
10
00
0
,
0,
11
1
km
Tw
Tw
T
T
Speedup
cp
cp
mf
f
Tree Network Tree Network (Children without Front Ends)(Children without Front Ends)
Collapsing single level Collapsing single level treestrees
Bandwidth of Fat TreeBandwidth of Fat Tree
Definition: The bandwidth of level j Definition: The bandwidth of level j in a fat tree can be defined as pin a fat tree can be defined as p j-1j-1z. z.
zmzpj
l
lj
11
01
Directions: Directions: Sequencing and InstallmentsSequencing and Installments
Daisy Chain SurpriseDaisy Chain Surprise
Efficiency RuleEfficiency Rule
Ghose, Mani & Bharadwaj
Directions: Directions: Sequencing and InstallmentsSequencing and Installments
Multi-installment for Sequential Multi-installment for Sequential DistributionDistribution
12
3
45
6
Ghose, Mani & Bharadwaj
Directions: Directions: Sequencing and InstallmentsSequencing and Installments
Diminishing returns in using multi-installment distribution.Ghose, Mani & Bharadwaj
Directions: Directions: Sequencing and InstallmentsSequencing and Installments
Drozdowski
Directions: Time Varying Directions: Time Varying ModelingModeling
Can be solved with integral calculus. Sohn & Robertazzi
Directions: Monetary Cost Directions: Monetary Cost OptimizationOptimization
MinMin C CTotal Total nnccnnwwnnTTcpcpn=1
N
Bus
Processors
Optimal Sequential Distribution if:
cn-1wn-1 less than cnwn for all n
Sohn, Luryi & Robertazzi
Directions: Monetary Cost Directions: Monetary Cost OptimizationOptimization
2 US Patents:2 US Patents:
Patent 5,889,989 (1999): Processor CostPatent 5,889,989 (1999): Processor Cost
Patent 6,370,560 (2001): Processor and Patent 6,370,560 (2001): Processor and
Link CostLink Cost
Enabling technology for an open e-Enabling technology for an open e-commerce commerce
market in leased proprietary computing. market in leased proprietary computing. Sohn, Charcranoon, Luryi & Robertazzi
Directions: Database Directions: Database ModelingModeling
Expected time to find multiple records in flat file database
Ko & Robertazzi
Directions: RealismDirections: Realism
Finite Buffers (Bharadwaj)Finite Buffers (Bharadwaj)
Job Granularity (Bharadwaj)Job Granularity (Bharadwaj)
Queueing Model IntegrationQueueing Model Integration
Directions: Experimental Directions: Experimental WorkWork
Database Join (Drozdowski)
Directions: Future Directions: Future ResearchResearch
Operating Systems:Operating Systems:
Incorporate divisible load scheduling Incorporate divisible load scheduling
into (distributed) operating systemsinto (distributed) operating systems
Measurement Process Modeling:Measurement Process Modeling:
Integrate measurement process Integrate measurement process
modeling into divisible scheduling modeling into divisible scheduling
Directions: Future Directions: Future ResearchResearch
Pipelining (Dutot)Pipelining (Dutot)
Concept: Distribute load to Concept: Distribute load to
further processors first further processors first for for
speedup improvementspeedup improvement
Improvement reported for daisy chainsImprovement reported for daisy chains
Directions: Future Directions: Future ResearchResearch
System Parameter Estimation (Ghose):System Parameter Estimation (Ghose):
Concept: Send small “probing” loads across Concept: Send small “probing” loads across
links and to processors to links and to processors to estimate estimate
available effortavailable effort
Challenge: Rapid change in link & processor Challenge: Rapid change in link & processor
statestate
DLT has a Good FutureDLT has a Good Future
Many Applications including Many Applications including
wireless sensor wireless sensor networksnetworks
Tractable (Modeling & Computation)Tractable (Modeling & Computation)
Rich Theoretical BasisRich Theoretical Basis
Thank you!Thank you!
Questions???Questions???