Scalable Systems Software Center Resource Management and Accounting Working Group Face-to-Face Meeting September 11-12, 2003 Washington D.C.

Resource Management and Accounting Working Group Working group scope Progress over last quarter Next steps Topics for group consideration

Working Group Scope The Resource Management Working Group is involved in the areas of resource management, scheduling and accounting. This working group will focus on the following software components: Queue Manager Scheduler Accounting and Allocation Manager Meta Scheduler Other critical resource management components are being developed in the Process Management and Monitoring Working Group: Process Manager Cluster Monitor

Resource Management Component Architecture Queue Manager Allocation Manager Node Monitor Meta Scheduler Local Scheduler Node Manager Process Manager Security System Information Service Discovery Service Color Key Working Group Resource Management and Accounting Execution Management and Monitoring Node Configuration and Infrastructure Infrastructure Services Event Manager

Resource Management Prototype Demonstration Queue Manager Allocation Manager Node Monitor Local Scheduler Process Manager Discovery Service Color Key Working Group Resource Management and Accounting Execution Management and Monitoring Node Configuration and Infrastructure Job Submission Client 1 Submit-Job 3 Query-Node 6 Exec-Process 4 Create-Reservation 2 Query-Job 5 Run-Job 8 Delete-Job 0 Service-Lookup 7 Query-Job 9 Withdraw-Allocation This demo runs a simple end-to-end test with a job being submitted running past it’s wallclock limit

General Progress SSSRMAP v2.0 interface specification has been implemented and tested by most RMWG components [all except Silver Meta-Scheduler] –Includes both HTTP Wire Protocol and XML Message Format –Includes implementation of SSS Job Object definition v2.0 Implemented default SSSRMAP v2.0 security authentication for most components [all except Silver Meta-Scheduler] –HMAC-SHA1 digital signature with shared secret key –Canonicalizes the XML System testing nearly complete for SSSRMAP v2 (on xtorc)

General Progress Created Node Object Specification version 1.0 –Differentiates between configured, available and utilized node properties Proposed set of response/status codes drafted –Success, warning and error response codes –Allows for component-specific and application-specific error- codes –Supports multiple levels of specificity Portability testing is underway (for initial release components) –Cobbed together a bunch of machines to test on

Portability Testing Progress

General Progress Re-release of v1.0 Initial SSS Resource Management Suite –OpenPBS-SSS sss_pbs_svr –Maui Scheduler –QBank sss_qbank_svr Webpage for RMWG recreated –Relevant documentation –Software downloads (tarballs and rpms) –Mailing lists –Links –Bug tracking

Scheduler Progress Implemented SSSRMAP v2 Wire Protocol (all clients, Resource Manager and Allocation Manager interfaces) Implemented SSSRMAP v2 Message Format (75% of clients, job object for Resource Manager and Allocation Manager) Implemented SSSRMAP authentication via shared secret keys Added 64-bit support for HMAC algorithm

Scheduler Progress Added limited support for SSSRMAP error codes Improved multi-taskgroup support Added man pages and command-line usage documentation Added resource manager, allocation manager and grid scheduler diagnostics to assist in configuration and troubleshooting

Queue Manager Progress Settled on a name: "Bamboo" Implemented SSSRMAP v2 wire protocol (including authentication) Support for SSSRMAP v2 XML support for single step jobs Data storage implemented via flat files or ODBC compliant database. Many improvements in build system, run-time configuration, and logging.

Accounting and Allocation Manager Progress QBank –Portability testing has begun Linux, HP-UX, AIX and IRIX completed Gold –Implemented SSSRMAP v2.0 encryption Using 3DES & session key generated from shared secret key Uses compression algorithm –Tested and verified SSSRMAP v2.0 authentication –Support added for Role Based Access Control (fine- grained command authorization)

Accounting and Allocation Manager Progress Gold –Object interface in web-based GUI has been implemented (gives you powerful low-level access to allocation manager objects) –Reimplemented the Gold client in Perl to overcome latency issues inherent in java startup overhead –Created a suite of full-featured Perl command-line clients Manages Users, Projects, Machines, Allocations (deposits, withdrawals, refunds, transfers, balance,…), Reservations, Quotations, Jobs, Usage, Transactions –Installed Gold on PNNL 11.8TF Linux cluster in transparent mode to test coherency and stability –Slow progress on open source front Blanket approval letter came out June 24th (CC ) Decision not to commercialize Public domain vs. open source (copyright issues)

Meta-Scheduler Progress Documentation for installation, configuration, and troubleshooting

Future Work User Oriented Problem Response System Complete portability testing for initial release components –(at least Linux, AIX, +other_UNIX) Release alpha versions of new components –(Bamboo, Silver, Gold) Begin portability testing for new components Create per-component interface specification documents (binding to SSSRMAP) Complete Design Specification documents for new components

Future Work Local Scheduler Complete integration of SSSRMAP v2 for queue and node objects Support full suite of allocation manager interface calls Full support for error codes Enhance dynamic job support with queue/task manager

Future Work Local Scheduler Support multi-source resource management interface Continued progress in resource limit enforcement and tracking Full resource limit enforcement and tracking configuration Integrate with checkpoint/restart capability (when available)

Future Work Queue manager Finish prologue/epilogue support once exit codes are available from process manager Interface with Node Monitor (probably after initial release) IO staging (may need API from process manager) Full multi step job support Package code for distribution. Add support for optional site job submission verification script.

Future Work Accounting and Allocation manager User and Admin interface for Gold Web-based GUI will be developed Integration with Directory Service Open source gold (BSD license) SSL over web gui and password authentication Production testing of Gold on 11.8TF Linux cluster (side-by-side with QBank)

Future Work Meta Scheduler Implement SSSRMAP v2 Wire Protocol and Message Format Add allocation manager interface support Add threaded support for cluster (local) scheduler interface

Issues requiring inter-group discussion Response Codes SC03 User Oriented Problem Response System Need process exit codes from process manager Cluster Monitor Open source