Resource Selector Chuang Liu

What do we want to do? A smart Resource Selector App R S Resource requirement

What do we want to do? App R S These Resources seem fit your requirement best

Goal A Resource Selector for general purpose Matching between application’s requirement and a set of resources Adaptability to dynamic status of distributed environment Support ownership of resource Support performance model of application The interface between application and RS should be simple

RS--Structure Resourc e Selector GRI S GIIS MDS Ap p RS MatchMaker Requirement Resource A subset of resource App

Challenges How to specify resource and request and how to match request with resource. Consistency of data in RS and system status. How to choose N resources from M available resources. (N <=M)

Challenges How to specify the request and resource and how to match the request with resource Consistency of data in RS and system status. How to choose N resources from M available resources. (N <=M)

ClassAd— Mechanism to specify resources and request The classad mechanism is a language for expressing and evaluating attribute A classad is a set of named expressions Each named expression is called an attribute Expression similar to those found in C/C++

ClassAd : example(resource) [ OpSys=“LINUX”; Name= “trapezius.cs.uchicago.edu”; LoadAvg= 0.03; Friends = (“foster”, “dave”); Untrusted = (“evil”, “rival”); Constraints= !member(other.Owner, Untrusted) && (LoadAvg < 0.3); ]

ClassAd : example(request) [ Owner=“chliu”; Requirements= other.LoadAvg < 0.3 && other.opSys=“LINUX”; Rank = 1/other.LoadAvg; ]

Challenges How to specify the resource and request and how to match the request with resource. Consistency of data in RS and system status. How to choose N resources from M available resources. (N <=M)

Consistency Several threads in RS update information about system status based TTL value. –Update information about available resource by access GIIS – Update information about status of every resource by access GRIS or GIIS Tradeoff between Performance and Consistency

Challenges How to specify the resource and request and how to match the request with resource. Consistency of data in RS and system status. How to choose N resources from M available resources which fit application’s requirement best. (N <=M)

Resource Selection How to select N resource from M available resources efficiently. How to judge which one is best among several matched results

Resource Selection How to select N resource from M available resources efficiently. How to judge which one is best among several matched results

Criteria to judge the desirability of resource Performance model –F(resource Info, application info) –[ minCPUSpeed > 10 MIPS – minMemSize > 100 MB – Rank= minCPUSpeed * NumOfResource –] –Embed a program or function call in Classad ? Classad don’t support function call in expression.

Resource Selection How to Select N resources from M available resources How to judge which one is best among several matched results

Bilateral match- Clique Organize all available resources into several cliques. Classads for resource Classads for clique Classads for requirement Match maker

Clique- How to organize clique Methods to organize clique –Manually –Automatically –Pros: Easy and useful –Cons: Not flexibility

Clique- Naive Naive method –For example: Resource: { a, b, c} Cliques {a}, {b}, {c}, {a, b}, {b,c}, {a,c}, {a, b, c} –Cons The number of clique is 2 to N

Gang Match Gang Matching 1  N Classads for resource Classads for requirement Match maker

Gang match- Greedy Algorithm Greedy Algorithm –Clique, candidate = null – Match the requirement with every resource –Choose resource with highest ranking as the first number of clique –For(;;) { – If (clique match requirement) and (performance of application increase) – Candidate=clique – Match the requirement with resource which is not in clique – Add node with highest ranking in the left nodes to clique – Else – Return Candidate –}

Gang match- Greedy Algorithm Pros: –High performance –Give pretty good optimal result to loosely coupled application Cons: –Locally optimal choice does not always lead to globally optimal solution

Gang match- Port and docking Ports and docking [ Ports = { [ Label = Host1; Requirements= MemorySize > 17.2M; Constraint = Host1.Arch == "INTEL" && Host1.OpSys == "LINUX"; Rank = Host1.MIPS ], [ Label = Host2; Requirements= MemoryReqs > 18 M; Constraint = Host2.Arch == "INTEL" && Host2.OpSys == "LINUX" && Host1.Subnet == Host2.Subnet Rank = Host2.KFlops ] Rank = 1 * Host1.MIPS + 8 * Host2.Kflops; } ]

Gang match- Port and docking Pros: –Internal mechanism provided by Classad –? Available in Classad package Cons: –Match performance –Application need to specify how many nodes it wanted

Gang match- Dynamic programming Limited resource use –System administrator control how many resources user can use –User’s requirement [ performance > threadhold value && number of resource is as little as possible] –Application’s requirement App specify how many CPU it wanted in request A 4 X 4 X 4 Grid calculation, numOfCPU < 64

Gang match Dynamic programming Multi-dimensions knapsack problem –Knapsack problem : “There are M items, every item has a Weight and a Value. Try to choose items from these items such that their value is maximum and their total weight is less than W.”

Gang match- Dynamic programming Pros: –More flexible –Always provide good resource to application –polynomial time algorithm o(N) Cons: –Performance is bad than greedy algorithm

Project Status A RS is running, but not smart enough. –Two match strateges have been implemented Clique Greedy algorithm

Open problem How well does RS work? –Performance of application on selected resource –Cost of RS Evaluate different strategy in match making

Welcome comment and suggestion Thank you!