View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Problem• Co-selection of resources
– Resource Selection
– Resource configuration
– Mapping workload to resources
Outline
• Problem• > Related work• Set Match• Resource Selection Service• Experiments• Summary• Future work
Related work• Networked Batch Queue System
– PBS, LSF, NQE
• Application-specific scheduler– AppLeS, etc.
• Condor
Outline
• Problem• Related work• > Set Match• Resource Selection Service• Experiments• Summary• Future work
Set Match• Resource Set:
– Several resources with similar characteristics that may be located in different sites.
• Set Match– A Set Match occurs between a resource request and a
resource set. If the resource set fulfill the resource request and the resource request fulfill resource’s management policy, they match each other.
Set Match
• ClassAds Language
The classads language is a language for expressing and evaluating attributes. An attribute is a named expression. A classad is a set of named expressions. Expression similar to those found in C/C++.
B=[ name=”foo”; type="machine"; cpuspeed=800M; memory=512M]
A=[ Requirements = other.type = = "machine" && other.cpuspeed > 500M; Rank = other.cpuspeed ]
Set Match
• Set-Extended ClassAds language– Aggregation functions
• Max(), Min(), Sum(), SizeOfSet()
– Other functions• suffix(S, L), Prefix(S, L)
Set Match
An Example of the Set Extended ClassAds [ Type=”Set”; Domainlist={ucsd.edu, cs.uiuc.edu}; requirements = Sum(other.memory) > 1G && other.cpuspeed > 200M && suffix(other.Hostname, Domainlist); rank = Min(other.cpuspeed)*SizeOfSet() ]
Resource Selection mechanism
Get all available resources and their status
Construct a Resource Set
Match request with the Resource Set andcalculate the quality of the match
Any other candidateResource Set ?
No
Yes
Output the Resource Set with highest quality
Resource Selection mechanism
Get all available resources and their status
Filter Resources based on the resource request
Construct a Resource Set
Match request with the Resource Set andcalculate the quality of the match
Any other candidateResource Set ?
No
Yes
Output the Resource Set with highest quality
Set BestSet and CandidateSet as empty
Choose X from Resource Pool such that for any Y inResource Pool
Rank(CandiateSet + X) > Rank(CandidateSet + Y)
Move X from Resource Pool to CandidateSet
Yes
No Match(CandidateSet, Request)
Resource Pool is Empty?Yes
No
Rank(CandidateSet) > Rank(BestSet)
Replace BestSet with CandidateSet
No
Yes
OUTPUT
Outline
• Problem• Related work• Set Match• > Resource Selection Service• Experiments• Summary• Future work
Resource Selection Service
• Help applications to choose a good Resource Set in Grid environment.
– Synchronous and asynchronous service
• Mapping application workload to resources if needed.
Resource Selection Service
• Framework
NWS
ResourceMonitor
SetMatcher
Mapper
RSS
App
ResourceRequest
Result
GRISes
GIIS
MDSResourceInformation
Resource Request• Owner: The sender of this request. • Type of Service: Synchronous or asynchronous.• Job description: The characteristics of the job to
be run, for example, problem size, problem specification and the performance model.
• Job resource requirements: User resource requirements, for example, memory capability, type of operating system, software packages installed, etc.
• Mapper: The kind of mapper algorithm to be used.• Rank: The criteria to rank the matched resources.
Resource Request: An example1. [ Service = "InstantService";
2. iter=100; alpha=100; x=100; y=100; z=100;
3. computetime = x*y*alpha/other.cpuspeed*370;
4. comtime= ( other.RLatency+ y*x*254/other.RBandwidth
+other.LLatency+y*x*254/other.Lbandwidth);
5. exectime=(computetime+comtime)*iter+startup;
6. Mapper = [type ="dll";
libraryname="cactus"; function="mapper"];
7. requirements = Sum(other.MemorySize) >= (1.757 + 0.0000138*z*x*y) && suffix(other.machine, domains);
8. domains={ cs.utk.edu, ucsd.edu};
9. rank=Min(1/exectime) ]
Job description
Resource Requirements
Type of Service
Mapper
Rank
Resource Selection Result
<virtualMachine>
<result statusCode="200" statusMessage="OK"/>
<machineList>
<machine dns="torc2.cs.utk.edu" processor= 2 x= 20>
<machine dns="torc3.cs.utk.edu" processor= 2 x= 15>
<machine dns="torc6.cs.utk.edu" processor= 2 x= 15>
</machineList>
</virtualMachine>
Outline
• Problem• Related work• Set Match• Resource Selection Service• > Experiments• Summary• Future work
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2 R 3
Cactus application
• Application performance model– ExecTime= (Communication time + Computation time) * slowdown
• Application mapping algorithm– 1 Dimension mapping
R 1 R 2 R 3
Execution Time Prediction
0
100
200
300
400
500
600
700
100*100*100 140*140*280 180*180*360 220*220*420
Predicted time Execution time
Problem Size
Time (S)
torc1Torc1Torc5
Torc1Torc5
Torc1Torc5 Torc1
,3, 5
Torc1,3, 5
Torc1,3, 5, o.ucsd.edu
Resource Selection Algorithm Test
• 1. o.ucsd.edu, mystere.ucsd.edu, saltimbanco.ucsd.edu
• 2: mystere.ucsd.edu, o.ucsd.edu
• 3: o.ucsd.edu, Saltimbanco.ucsd.edu 4. o.ucsd.edu
• 5: saltimbanco.ucsd.edu 6: mystere.ucsd.edu
0
50
100
150
200
250
300
1 2 3 4 5 6 7
Execution time
Resource Selection Algorithm Test
• 1: torc6.cs.utk.edu 2: o.ucsd.edu 3: Saltimbanco.ucsd.edu
• 4. Torc6.cs.utk.edu + o.ucsd.edu 5: o.ucsd.edu + saltimbanco.ucsd.edu
• 6. o.ucsd.edu, mystere.ucsd.edu, torc6.cs.utk.edu
0
50
100
150
200
250
300
350
1 2 3 4 5 6 7
Execution time
Outline
• Problem• Related work• Set Match• Resource Selection Service• Evaluation of the implementation• > Summary• Future work
Summary
• Extended the ClassAds language to describe the requirement for a Resource Set
• Implement a Set Matchmaker• Create a general framework for the Resource
Selection Service• Validate our implementation by the Cactus
application
Future Work
• Extend the resource selection algorithm to support other applications.– Gang match
– Resource with particular topology
• Provide fault tolerance to incomplete or error information
• A general matchmaking mechanism