Resource Selector
Chuang Liu
What do we want to do?
• A smart Resource Selector
App
R S
Resource requirement
What do we want to do?
App
R SThese Resources seem fit your requirement best
Goal
• A Resource Selector for general purpose• Matching between application’s requirement and a
set of resources• Adaptability to dynamic status of distributed
environment• Support ownership of resource • Support performance model of application• The interface between application and RS should
be simple
RS--Structure
Resource
Selector GRIS
GIISMDSAp
p
RS
MatchMaker
RequirementResource
A subset of resource
App
Challenges
• How to specify resource and request and how to match request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
Challenges
• How to specify the request and resource and how to match the request with resource
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
ClassAd—Mechanism to specify resources and request
• The classad mechanism is a language for expressing and evaluating attribute
• A classad is a set of named expressions
• Each named expression is called an attribute
• Expression similar to those found in C/C++
ClassAd : example(resource)
• [• OpSys=“LINUX”;• Name= “trapezius.cs.uchicago.edu”;• LoadAvg= 0.03;• Friends = (“foster”, “dave”);• Untrusted = (“evil”, “rival”);• Constraints= !member(other.Owner, Untrusted) &&
(LoadAvg < 0.3);• ]
ClassAd : example(request)
• [
• Owner=“chliu”;
• Requirements= other.LoadAvg < 0.3 && other.opSys=“LINUX”;
• Rank = 1/other.LoadAvg;
• ]
Challenges
• How to specify the resource and request and how to match the request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
Consistency
• Several threads in RS update information about system status based TTL value.– Update information about available resource by
access GIIS– Update information about status of every
resource by access GRIS or GIIS• Tradeoff between Performance and Consistency
Challenges
• How to specify the resource and request and how to match the request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources which fit application’s requirement best. (N <=M)
Resource Selection
• How to select N resource from M available resources efficiently.
• How to judge which one is best among several matched results
Resource Selection
• How to select N resource from M available resources efficiently.
• How to judge which one is best among several matched results
Criteria to judge the desirability of resource
• Performance model– F(resource Info, application info)
– [ minCPUSpeed > 10 MIPS
– minMemSize > 100 MB
– Rank= minCPUSpeed * NumOfResource
– ]
– Embed a program or function call in Classad• ? Classad don’t support function call in expression.
Resource Selection
• How to Select N resources from M available resources
• How to judge which one is best among several matched results
Bilateral match- Clique
• Organize all available resources into several cliques.
Classads for resource
Classads for clique
Classads for requirement
Match maker
Clique- How to organize clique
Methods to organize clique– Manually– Automatically– Pros:
• Easy and useful
– Cons:• Not flexibility
Clique- Naive
• Naive method– For example:
• Resource: { a, b, c}
• Cliques {a}, {b}, {c}, {a, b}, {b,c}, {a,c}, {a, b, c}
– Cons• The number of clique is 2 to N
Gang Match
• Gang Matching 1 N
Classads for resource
Classads for requirement
Match maker
Gang match- Greedy Algorithm
• Greedy Algorithm– Clique, candidate = null– Match the requirement with every resource– Choose resource with highest ranking as the first number of clique– For(;;) {– If (clique match requirement) and (performance of application increase)– Candidate=clique– Match the requirement with resource which is not in clique– Add node with highest ranking in the left nodes to clique– Else– Return Candidate – }
Gang match- Greedy Algorithm
• Pros:– High performance– Give pretty good optimal result to loosely
coupled application
• Cons:– Locally optimal choice does not always lead to
globally optimal solution
Gang match- Port and docking
• Ports and docking• [ Ports = {• [ Label = Host1;• Requirements= MemorySize > 17.2M;• Constraint = Host1.Arch == "INTEL" && Host1.OpSys == "LINUX";• Rank = Host1.MIPS• ],• [ Label = Host2;• Requirements= MemoryReqs > 18 M;• Constraint = Host2.Arch == "INTEL" && Host2.OpSys == "LINUX"
&&• Host1.Subnet == Host2.Subnet• Rank = Host2.KFlops• ]• Rank = 1 * Host1.MIPS + 8 * Host2.Kflops;• } ]
Gang match- Port and docking
• Pros:– Internal mechanism provided by Classad– ? Available in Classad package
• Cons:– Match performance– Application need to specify how many nodes it
wanted
Gang match- Dynamic programming
• Limited resource use– System administrator control how many
resources user can use– User’s requirement
• [ performance > threadhold value && number of resource is as little as possible]
– Application’s requirement • App specify how many CPU it wanted in request• A 4 X 4 X 4 Grid calculation, numOfCPU < 64
Gang match Dynamic programming
• Multi-dimensions knapsack problem– Knapsack problem : “There are M items, every
item has a Weight and a Value. Try to choose items from these items such that their value is maximum and their total weight is less than W.”
Gang match- Dynamic programming
• Pros:– More flexible– Always provide good resource to application– polynomial time algorithm o(N)
• Cons:– Performance is bad than greedy algorithm
Project Status
• A RS is running, but not smart enough.– Two match strateges have been implemented
• Clique
• Greedy algorithm
Open problem
• How well does RS work?– Performance of application on selected resource– Cost of RS
• Evaluate different strategy in match making
Welcome comment and suggestion
Thank you!