Grid Application Development Software Project Outline l Resource Selection: Current Directions l Contracts: Current Directions l Current Status –Resource Selection >Request Protocol >Response Protocol –Resouce “Scheduling” –Contracts –Migration Manager

Resource Selection Current Directions

Grid Application Development Software Project Current Architecture Under Development Resource Selection Client Thorn External Resource Selection Service “Worm” Migration Module Cactus Worm Server Thorns Cactus Application Unit Cactus Flesh Performance Degradation Detection User Supplied Application Payload External Processes Migration Logic Manager GridFTP Client Thorn External GridFTP Server (Source) External GridFTP Server (Destination) Data transfer

Grid Application Development Software Project Resource Selector Architecture UCSD (UCSD) Resource Selection Client Thorn Resource Selection Library UCSD (HFA/GradsSoft) HFA/GradsSoft Translator Request in ClassAds format Response (format?) MDS NWS GRIS’s Protocol? Http? SOAP?

Grid Application Development Software Project Resource Selector Architecture ClassAds (ClassAds) Resource Selection Client Thorn ClassAds library Resource Selection Engine Request in ClassAds format Response (format?) MDS NWS GRIS’s Protocol? Http? SOAP? UTk Project Needed for recovery and timeliness?

Grid Application Development Software Project Resource Selector Architecture Other RS’s (Other) Resource Selection Client Thorn Other Resource Selection Service Request in some format Response in some format Protocol? Http? SOAP?

Contract Monitoring Current directions

Grid Application Development Software Project Contract Monitor l Driven by three user-controllable parameters –Time quantum for “time per iteration” –% degradation in time per iteration (relative to prior average) before noting violation –Number of violations before migration l Potential causes of violation –Competing load on CPU –Computation requires more processing power: e.g., mesh refinement, new subcomputation –Hardware problems

Grid Application Development Software Project Contract Monitor Details l The end user specifies several variables. l These variables can be changed during runtime by contacting the application with an HTTP interface. l These variables include: – time quantum – % degradation – number of violations before migration l The system will then calculate the average wall clock time per iteration for each time quantum. l If the average iteration in any time quantum has lower performance (by the percentage specified) than the average for all the other previous quanta, then a violation is noted.

Grid Application Development Software Project Actions Taken on Contract Violation l Occurs when more than the specified number of violations have been noted l New set of resources requested from the ResourceSelector l Checkpoints application l Moves checkpoint data to the new resources along with other data needed for restart l Restarts application on the new resources

Current Status

Grid Application Development Software Project Resource Selection l Demonstrated migration using RS with simple protocol (using raw sockets). l Working on more robust protocol over HTTP using ClassAds as request and XML as response –Robustness (error handling) critical on real grid –Important to use well known protocol l Working on incorporating performance model into ClassAds

Grid Application Development Software Project Resource Selection: Example Input [ Type="request"; Owner="dangulo"; RequiredDomains={"cs.uiuc.edu", "ucsd.edu"}; requirements= "other.opSys=="LINUX" & other.minMemSize> (100G/other.CPUCount) && Include(other.domains, RequiredDomains) "; Rank= other.minCPUSpeed * other.CPUCount / (other.maxCPULoad+1); ]

Grid Application Development Software Project Resource Selection: Input l Need to specify other user-centric informaion –Cactus is installed in user space l We’re investigating whether we can put the Performance Model equations into the ClassAds format in order to pass it to the Resource Selector. –The “Rank” value in the preceding slide shows a simple example of this.

Grid Application Development Software Project Resource Selection: Example output

Grid Application Development Software Project Resource Selection: Example output No resource is found <result statusCode="204“ statusMessage="No match Resource is Found"/>

Grid Application Development Software Project Resource Selection: Example output Bad request from client (request format error)

Grid Application Development Software Project Resource Selection: Example output MDS server is down <result statusCode="601“ statusMessage="MDS Service is not available"/>

Grid Application Development Software Project Resource “Scheduling” l What word do we use for allocating machines to data (“scheduling” seems wrong). l We’re assuming that RS does this l We need to map RS output to Cactus machine distribution

Grid Application Development Software Project Contract Monitoring l Demonstrated detection of performance degradation l Application monitors placed in Cactus scheduling –routine called once per iteration –accesses Cactus internal timing API –synchronization implies that timing on all nodes are identical >could use different Cactus scheduling times to get node dependant results

Grid Application Development Software Project Migration Manager l In initial development l Will allow RS selection to occur asynchronously l Will make intelligent choice on whether migration will actually help –Will not migrate to seemingly lower quality resources