Thursday, August 21, 2008 Cyberinfrastructure for Research Teams UAB High Performance Computing Services John-Paul Robinson

UAB Cyberinfrastructure (CI) Investments Common Network User Identity (BlazerID) for consistent identity across systems Early Internet2 Member providing high bandwidth network access to other research campuses High Performance Computing (HPC) Investments to build investigative capacity for computational research On-going Model of Engagement to support Research Technology Investments

Alabama State Optical Network and National LamdaRail Alabama SON is a very high bandwidth lambda network. Operated by SLR. Connects major research institutions across state Connects Alabama to National Lambda Rail and Internet2

10GigE Campus Research Network  Connects campus HPC centers to facilitate resource aggregation  Compute clusters scheduled for connectivity  Facilitates secure network build outs  Expands access to regional and national compute resources

UAB Investments in HPC

Cyberinfrastructure Elements A Continuum of Identity lower assurance – facilitates collaboration higher assurance – facilitates trust Maximized Network Bandwidth Pools of Execution Resources A Common Data Framework Reliability and Performance Monitoring

Harnessing CI with the Grid Interconnects and coordinates resources across administrative domains Uses standard, open, and general purpose interfaces and protocols Allows resource combination to deliver high quality services built on the core utility The “grid” is the Fabric of Interconnected Resources

About UABgrid Leverages Local, Regional and National Cyberinfrastructure Components Identity, Execution, Data, Status, and Networking Integrated Technology Infrastructure to Facilitate and Encourage Collaboration Remember: It's All About the Data Sharing Information is the Motivation for Collaboration

UABgrid Overview UABgrid Pilot launched at campus HPC Boot-Camp September 2007 User-driven collaboration environment supports web and grid applications Leverages InCommon for user identification SSO for web applications and VO Management Self-service certificate generation for Globus users Provides meta-cluster to harness on- and off- campus compute power using GridWay

Cyberinfrastructure IdM Exec Data Net Info UABgrid Application 1 Application 4 Application 3 Application 2 Building Standard Service Interfaces

Infrastructure to Support Application Domains Cyberinfrastructure IdM Exec Data Net Info UABgrid Application 1 ResearchUser AdminEducation

UABgrid Provides Services to Research Applications Cyberinfrastructure IdM Exec Data Net Info UABgrid Research Applications UsersStats FilesProcesses GroupsComm

UABgrid Applications and Services Collaboration Support  VO Tools: VO Mgmt, Mail lists, Wiki's, Project Mgmt, Portals... Research Applications Support  Compute Expansion Goals Generic Model Current Focus is Workflow Migration  Science Domains Microbiology -- DynamicBLAST Statistical Genetics – R Statistical Package Cancer Center – caBIG

UABgrid VO Management: User Attributes to Apps IdP 1 IdP 2 IdP 1 IdP n App 1 App n App 2 Identity ProvidersApplications User Attributes VO Attributes myVocs System

Collaboration Support myVocs box forms the core of VO collaboration infrastructure VO resources like mailing list, wiki's, and Trac intrinsic to VO and can access common authorization information Additional web collaboration tools instantiated as needed (eg. Gridsphere) VO resources hosted in VM cloud dev.uabgrid is a working VO model for the construction and management of UABgrid

Compute Expansion Meta-scheduling: Grid Cluster Cluster Upgrades and Acquisitions Resource Aggregation  State Resources  Regional Resources via SURAgrid  National and International Resources via TeraGrid & Open Science Grid

UABgrid Compute Cluster Test Architecture

UABgrid Pilot Meta-Cluster Specifications Today 2 campus clusters + ASA resource: 912 processing cores, >5TFlops of power 2009 Targets Add all shared campus clusters: 1156 more processing cores and 10TFlops of additional power On Going Local expansion though campus HPC investments Engage SURAgrid, OSG, TeraGrid, and other grid compute suppliers for more compute power

SURAgrid Drawing Power from the Grid

Generic Grid Application Model Command LineCustom ClientWeb Portal Appllication Workflow Logic Metascheduling: GridWay, DRMAA, Swift, Pegasus, Avalon Globus Client Tools Globus Services SGE App Code Data LSF App Code Data PBS App Code Data Cluster 1Cluster nCluster 2

Grid Migration Goals Eliminate need for user-level grid technology awareness Build on grid middleware, tools, and standards to maximize portability and resource utilization Manage and leverage variable resource availability and dynamic load balancing Efficiently and transparently handle issues like application availability, fault tolerance, and interoperability

Application Containers Simplify Administration Types of Containers User Accounts Java Boxes Virtual Machines Account Containers Initial target because most common and addresses R application configuration Allows for library dependency and site dependency configuration Full continuum of deployment options from fully staged for each job to statically cached on resources

Migrating Workflows to Grid Statistical Genetics R Statistical Package Methodological Analysis Workflow Many Isolated Computations Work in Progress and Promising Results Developing Work led by John-Paul Robinson in UAB HPC Services Microbiology DynamicBLAST – Grid Version of BLAST Master Worker Type Application Maximize Throughput, Minimize Job Turn-around Leading Model for Migrations Work led by Enis Afgan and Dr. Puri Bangalore in CIS

Statistical Genetics on the Grid – MIG Grid Technology offers 100s of computers which means more Compute Power, Disk Storage Efficiency, and Faster Results Microarray The Grid Tissue Samples Data Clusters

MIG Workflow Powered by the Grid Manual job control constrains performance to the human scale (~10) Automating job control enables managing scale that significantly improves job performance and resource utilization

Dynamic BLAST Grid Workflow BLAST is a gene search algorithm Dynamic BLAST breaks application steps and search apart and spreads effort across the grid Good example of component and data parallelization

SCOOP – Coastal Ocean Observation and Prediction SURA program to advance the sciences of prediction and hazard planning for coastal populations Harvests cycles around the grid Working with MCNC/Renci to use Cheaha via SURAgrid

Research Initiative Support caBIG UAB Comprehensive Cancer Center funded to connect to caBIG Contributed to completion of Self-Assessment and Implementation Plan Deploying Life Sciences Distribution to support research workflows caBIG provides a very good model for service and infrastructure abstractions caGrid Bring BlazerID system to NIST Level 2 Exploring Integration of caGrid GAARDS AuthX Infrastructure (GridGrouper)‏

caGrid Provides Tools For Many Research Domains Taxonomy Development Traverna Workflow Management

Education and Training UAB 2007 HPC Boot Camp included sessions on grid computing and UABgrid Pilot launch 2008 HPC Bootcamp September 22, 2008 UAB 1 st Annual CIDay in conjunction with ASA campus visit CIS has taught graduate-level grid computing courses since fall 2003 Active participation in grid technology communities MardiGras08, OGF22, SURAgrid All-Hands, Internet2, caBIG

Open Development Model UABgrid development work is done openly Outside groups are actively engaged in the development of infrastructure (CIS, ENG, ASA, etc)‏ Development group relies on the same services available to all users (we eat our own dog food)‏ Virtual organizations build on infrastructure and are free to engage to their level of interest

Collaborative Development Engaging User Groups and Service Providers to leverage Infrastructure We are building our own solutions to depend on the grid In order to build a grid, you need carrots – there has to be a benefit, even if it's long term Grid services and development environment built on virtual machine foundation – key to expectation of “running from the cloud”

Engagement in a Regional Infrastructure Construction Involved in SURAgrid since it's inception as a voluntary extension the the NSF's Middleware Initiative Tesbed Have helped mold an organization that provides broad engagement across organizations in the development of infrastructure SURAgrid Governance Committee just completed strategic plan to guide the next 4 years

Technology in Service of Research IT expresses institutional initiatives IT doesn't necessarily do it but should help make it possible To have leading research you need leading infrastructure IT supports a leading edge infrastructure and services framework IT provides transparent interfaces to services and operations Implement grid interfaces and conventions for our own services – “eat our own dog food”

Trust is the Foundation for Collaboration People Use Technology They Trust  Open Communication Channels  Researchers and Infrastructure communicate as peers  Intra-organizational communication is fluid  Control Over Implementation  Application requirements lead acquisitions  Service Partnership  Researchers and Infrastructure work together to satisfy organizational commitments  Important Issues are Guaranteed Service  Researchers have authorized influence over Infrastructure because are part of same organization

On The Horizon  Data Services UABgrid Backup  Implement using technologies that satisfy needs of the user community (eg. GridFTP, REDDnet  Focus on backup of VMs: putting our valuable data on-line...just like users would be expected to do Data Stores  Dspace, Fedora, Alfresco, Subversion  Metrics increase reliability confidence and maintain a pulse on the impact of our solutions  Resource Integration Guidlines  High Speed to the Desktop

Acknowledgments UAB Office of the Vice President for Information Technology Collaborators at UAB in Computer and Information Sciences, the School of Engineering, the School of Public Health Section on Statistical Genetics, Comprehensive Cancer Center Collaborators within SURAgrid, Internet2, and other organizations

A Closing Thought... We are part of the Cyberinfrastructure The reason for CI is to empower us as individuals to engage with others as we build community at UAB and reach out to collaborate with other like minded communities around the globe