Design and Evaluation of a Resource Selection Framework for Grid Applications University of Chicago

Problem Co-selection of resources –Resource Selection –Resource configuration –Mapping workload to resources

Outline Problem > Related work Set Match Resource Selection Service Experiments Summary Future work

Related work Networked Batch Queue System –PBS, LSF, NQE Application-specific scheduler –AppLeS, etc. Condor

Outline Problem Related work > Set Match Resource Selection Service Experiments Summary Future work

Set Match Resource Set: –Several resources with similar characteristics that may be located in different sites. Set Match –A Set Match occurs between a resource request and a resource set. If the resource set fulfill the resource request and the resource request fulfill resource’s management policy, they match each other.

Set Match Matchmaker

Set Match ClassAds Language The classads language is a language for expressing and evaluating attributes. An attribute is a named expression. A classad is a set of named expressions. Expression similar to those found in C/C++. B=[ name=”foo”; type="machine"; cpuspeed=800M; memory=512M] A=[ Requirements = other.type = = "machine" && other.cpuspeed > 500M; Rank = other.cpuspeed ]

Set Match Set-Extended ClassAds language –Aggregation functions Max(), Min(), Sum(), SizeOfSet() –Other functions suffix(S, L), Prefix(S, L)

Set Match An Example of the Set Extended ClassAds [ Type=”Set”; Domainlist={ucsd.edu, cs.uiuc.edu}; requirements = Sum(other.memory) > 1G && other.cpuspeed > 200M && suffix(other.Hostname, Domainlist); rank = Min(other.cpuspeed)*SizeOfSet() ]

Resource Selection mechanism

Outline Problem Related work Set Match > Resource Selection Service Experiments Summary Future work

Resource Selection Service Help applications to choose a good Resource Set in Grid environment. –Synchronous and asynchronous service Mapping application workload to resources if needed.

Resource Selection Service Framework

Resource Request Owner: The sender of this request. Type of Service: Synchronous or asynchronous. Job description: The characteristics of the job to be run, for example, problem size, problem specification and the performance model. Job resource requirements: User resource requirements, for example, memory capability, type of operating system, software packages installed, etc. Mapper: The kind of mapper algorithm to be used. Rank: The criteria to rank the matched resources.

Resource Request: An example 1.[ Service = "InstantService"; 2. iter=100; alpha=100; x=100; y=100; z=100; 3. computetime = x*y*alpha/other.cpuspeed*370; 4. comtime= ( other.RLatency+ y*x*254/other.RBandwidth +other.LLatency+y*x*254/other.Lbandwidth); 5. exectime=(computetime+comtime)*iter+startup; 6. Mapper = [type ="dll"; libraryname="cactus"; function="mapper"]; 7. requirements = Sum(other.MemorySize) >= ( *z*x*y) && suffix(other.machine, domains); 8. domains={ cs.utk.edu, ucsd.edu}; 9. rank=Min(1/exectime) ] Job description Resource Requirements Type of Service Mapper Rank

Resource Selection Result

Outline Problem Related work Set Match Resource Selection Service > Experiments Summary Future work

Cactus application Application performance model –ExecTime= (Communication time + Computation time) * slowdown Application mapping algorithm –1 Dimension mapping R 1

Cactus application Application performance model –ExecTime= (Communication time + Computation time) * slowdown Application mapping algorithm –1 Dimension mapping R 1R 2

Cactus application Application performance model –ExecTime= (Communication time + Computation time) * slowdown Application mapping algorithm –1 Dimension mapping R 1R 2R 3

Cactus application Application performance model –ExecTime= (Communication time + Computation time) * slowdown Application mapping algorithm –1 Dimension mapping R 1R 2R 3

Execution Time Prediction Problem Size Time (S) torc1 Torc1 Torc5 Torc1 Torc5 Torc1 Torc5 Torc1,3, 5 Torc1,3, 5, o.ucsd.edu

Resource Selection Algorithm Test 1. o.ucsd.edu, mystere.ucsd.edu, saltimbanco.ucsd.edu 2: mystere.ucsd.edu, o.ucsd.edu 3: o.ucsd.edu, Saltimbanco.ucsd.edu 4. o.ucsd.edu 5: saltimbanco.ucsd.edu 6: mystere.ucsd.edu

Resource Selection Algorithm Test 1: torc6.cs.utk.edu 2: o.ucsd.edu 3: Saltimbanco.ucsd.edu 4. Torc6.cs.utk.edu + o.ucsd.edu 5: o.ucsd.edu + saltimbanco.ucsd.edu 6. o.ucsd.edu, mystere.ucsd.edu, torc6.cs.utk.edu

Outline Problem Related work Set Match Resource Selection Service Evaluation of the implementation > Summary Future work

Summary Extended the ClassAds language to describe the requirement for a Resource Set Implement a Set Matchmaker Create a general framework for the Resource Selection Service Validate our implementation by the Cactus application

Future Work Extend the resource selection algorithm to support other applications. –Gang match –Resource with particular topology Provide fault tolerance to incomplete or error information A general matchmaking mechanism

Welcome comments and suggestions Thank you!

Problem Co-selection of resources –Dynamic nature of Grid resources –Heterogeneity of resources –The autonomy of resources –The quality of resources depends on the application’s run characteristics