The Moab Grid Suite CSS´ 06 – Bonn – July 28, 2006

Computation in Life Science Research High-throughput virtual screening (serial) Molecular dynamics or Monte Carlo simulations (parallel) Free energy calculations (parallel) Cheminformatics (serial) Structure prediction and design (serial/parallel) Quantum chemistry (serial/parallel) ©2006 anterio consult & researchMoab Grid Suite Multiple sites, continents Multiple applications, hardware Different policies, operating systems, security Problems a Pharma/Biotech May Face

Moab Grid Suite ©2006 anterio consult & researchMoab Grid Suite Moab Grid Suite is a policy-based grid management solution that integrates scheduling, managing, monitoring and reporting of workloads across independent clusters.

Moab Grid Suite Boehringer Ingelheim Biogen Idec Fred Hutchinson Cancer Research Center TeraGrid, WestGrid, CMA, CDAC ©2006 anterio consult & researchMoab Grid Suite Exemples of Customers

Technical Overview Message Passing SerialParallel Application Resource Manager Operating System Hardware (Cluster or SMP) AdminUsersSecurity Application Portal CLI GUI Cluster Workload Manager: Scheduler, Policy Manager, Integration Platform Grid Workload Manager: Scheduler, Policy Manager, Integration Platform ©2006 anterio consult & researchMoab Grid Suite Cluster Stack / Framework:

Architecture Technical Overview ©2006 anterio consult & researchMoab Grid Suite

What Moab does Moab Grid Suite ©2006 anterio consult & researchMoab Grid Suite Optimizes Resource Utilization with Intelligent Scheduling and Advanced Reservations Unifies Cluster Management across Varied Resources and Services Dynamically Adjusts Workload to Enforce Policies and Service Level Agreements Automates Diagnosis and Failure Response

What Moab does not do Moab Grid Suite ©2006 anterio consult & researchMoab Grid Suite Does not does do resource management (usually) Does not install the system (usually) Not a storage manager Not a license manager Does not do message passing

Cluster B Cluster B Grid Types Administrator Cluster A Local Area Grid (LAG) Cluster B Cluster C Shared User Space Shared Data Space Moab Cluster A Wide Area Grid (WAG) Cluster B Cluster C Multiple User Spaces Multiple Data Spaces Moab (Master) Moab Cluster A Cluster B Cluster C Moab (Grid Head Node) Moab Centralized Management Cluster A Cluster C Moab Centralized & Local Management All Grid Rules Moab (Grid Head Node) Shared Grid Rules Local Grid Rules Moab Local Grid Rules Moab Local Grid Rules Cluster A Cluster C Moab Local Management Peer to Peer Local Grid Rules Moab Local Grid Rules Moab Local Grid Rules Management Scenarios ©2006 anterio consult & researchMoab Grid Suite

Grid Relationship Combination Administrator Cluster A Cluster B Cluster C Shared User Space Shared Data Space Moab Multiple User Spaces Multiple Data Spaces Cluster D Cluster E Moab (Grid Head Node) Shared Grid Rules Local Area Grid Rules Moab Local Grid Rules Moab Local Grid Rules 1. Join local area grids into wide area grids --- Moab is able to facilitate virtually any grid relationship. 1 ©2006 anterio consult & researchMoab Grid Suite

Grid Relationship Combination Administrator Cluster A Cluster B Cluster C Shared User Space Shared Data Space Moab Multiple User Spaces Multiple Data Spaces Cluster D Cluster E Moab (Grid Head Node) Shared Grid Rules Local Area Grid Rules Moab Local Grid Rules Moab Local Grid Rules Cluster F Cluster G Cluster H Moab Local Grid Rules Moab Local Grid Rules Moab Local Grid Rules 2. Join wide area grids to other wide area grids (whether they be managed centrally, locally - peer to peer or mixed) 1 2 ©2006 anterio consult & researchMoab Grid Suite

Grid Relationship Combination Administrator Cluster A Cluster B Cluster C Shared User Space Shared Data Space Moab Multiple User Spaces Multiple Data Spaces Cluster D Cluster E Moab (Grid Head Node) Shared Grid Rules Local Area Grid Rules Moab Local Grid Rules Moab Local Grid Rules Cluster F Cluster G Cluster H Moab Local Grid Rules Moab Local Grid Rules Moab Local Grid Rules Hosting Site Moab Local Grid Rules 3. Resource sharing can be in one direction for use with hosting centers, or to bill out resources to other sites ©2006 anterio consult & researchMoab Grid Suite

Grid Relationship Combination Administrator Cluster A Cluster B Cluster C Shared User Space Shared Data Space Moab Multiple User Spaces Multiple Data Spaces Cluster D Cluster E Moab (Grid Head Node) Shared Grid Rules Local Area Grid Rules Moab Local Grid Rules Moab Local Grid Rules Cluster F Cluster G Cluster H Moab Local Grid Rules Moab Local Grid Rules Moab Local Grid Rules Hosting Site Moab Local Grid Rules 4. Have multiple levels of grid relationships (e.g. conglomerates within conglomerates within conglomerates) ©2006 anterio consult & researchMoab Grid Suite 4

Administrator Sovereignty: Local vs. Centralized Management Policies ©2006 anterio consult & researchMoab Grid Suite Local Admin Each Admin can manage their own cluster Submit to either: Local cluster Specified cluster(s) in the grid Generically to the grid Local Cluster A Resources Grid Allocated Resources Portion Allocated to Grid Local Admin can apply policies to manage: 1. Local user access to local cluster resources 2. Local user access to grid resources 3. Outside grid user access to local cluster resources (general or specific policies) Local Users Outside Grid Users Grid Administration Body Grid Administration Body can apply policies to manage: A. General grid policies (Sharing, Priority, Limits, etc.) A

Administrator Data Staging Manages intra-cluster and inter-cluster job data staging requirements so as to minimize resource inefficiencies and maximize system utilization Prevent the loss of compute resources due to data blocking and can significantly improve cluster performance. ©2006 anterio consult & researchMoab Grid Suite

Administrator ©2006 anterio consult & researchMoab Grid Suite Cluster A Moab Local Grid Rules 4 Start job after the file is verified to be in the right location 1 Storage System Data Manager Job Submission 3 Benefits: Prevents non-staged jobs from blocking usage of nodes Drawbacks: No job-centric prioritization takes place in the order of which data gets staged first 2 Verified Data Staging

Administrator ©2006 anterio consult & researchMoab Grid Suite Cluster A Moab Local Grid Rules 5 Priority order of data staging 1 Storage System Data Manager Job Submission 4 3 Priority Jobs First 2 Prioritized Data Staging

Administrator ©2006 anterio consult & researchMoab Grid Suite Priority order of data staging and data-staging centric scheduling Benefits: Prevents non-staged jobs from blocking usage of nodes Provides soft prioritization of data staging requests Intelligently schedule resources based on data staging information Drawbacks: Prioritization is only softly provided Cluster A Moab Local Grid Rules 6 Storage System Data Manager Job Submission 5 Priority Jobs First Fully Scheduled Data Staging: External Storage

Administrator ©2006 anterio consult & researchMoab Grid Suite Priority order of data staging and data-staging centric scheduling Benefits: Prevents non-staged jobs from blocking usage of nodes Provides soft prioritization of data staging requests Intelligently reserves resources based on data staging information Drawbacks: Prioritization is only softly provided Cluster A Moab Local Grid Rules Data Manager Job Submission S S S Storage is on Local Compute Nodes 1 3 Priority Jobs First 6 Fully Scheduled Data Staging: Local Storage

Administrator ©2006 anterio consult & researchMoab Grid Suite Monitoring performance statistics of multiple independent clusters Detecting and diagnosing failures from geographically distributed clusters Tracking cluster, storage, network, service, and application resources Generating load-balancing and resource state information for users and middleware services Information Services

Administrator ©2006 anterio consult & researchMoab Grid Suite Administer Resources and Workload Policies Through an Easy-to- Use Graphical User Interface Monitor, Diagnose and Report Resource Allocation and Usage Moab Grid Manager TM

Administrator Moab Grid Manager TM ©2006 anterio consult & researchMoab Grid Suite Service Monitoring and Management:

Administrator Moab Grid Manager TM ©2006 anterio consult & researchMoab Grid Suite RealTime Performance and Accounting Analysis:

End User Moab Access Portal TM Submit jobs from a web browser View and modify only your own workload Assist end users to self-manage behaviours Assists users in better utilizing resources: General info Job eval. Completed job failure post-mortem. Job start time estimates. Job control. Reservation control. ©2006 anterio consult & researchMoab Grid Suite

Contact Tassilo Eckerle Sales Manager Phone:+49 (6 21) Fax:+49 (6 21) Mobile:+49 (1 63) Jianxin Duan, PhD Research Scientist Phone:+49 (6 21) Fax:+49 (6 21) Thank you for your attention! For further questions please do not hesitate to contact us.