Constructing Campus Grids Experiences adapting myVocs to UABgrid John-Paul Robinson High Performance Computing Services Office of the Vice President for Information Technology University of Alabama at Birmingham Internet2 Spring Member Meeting April 2007

Overview UAB CyberInfrastructure UABgrid myVocs myVocs box myVocs box on UABgrid Setting Up a VO Future Directions

UAB CyberInfrastructure UAB HPC Resources Shared HPC Facility has 4 clusters Computer Science HPC Facility has 2 clusters UAB overall HPC computing power has been tripling approximately on a 2 year cycle during the past 4 years Optical Networks – campus & regional UABgrid – a campus computing and collaboration environment

UAB HPC Resources IBM BlueGene/L System (most recent) 2 Dell Xeon 64-bit Linux Clusters 128 nodes 4 TB disk storage Gigabit and Infiniband interconnect 2 Verari Opteron 64-bit Linux Clusters 64 and 32 nodes 2 GB RAM per node Gigabit interconnect IBM Xeon 32-bit Linux Cluster 64 Nodes, Gigabit interconnect

UAB 10GigE Research Network Build high bandwidth network linking UAB compute clusters Leverage network for staging and managing grid-based compute jobs Connect directly to high-bandwidth regional networks

UABgrid Common interface for access to HPC infrastructure Leverage UAB identity management system for consistent identity across resources Provide access to regional, national, and international collaborators using Shibboleth identity framework Support research collaboration through autonomous virtual organizations

UABgrid Architecture Leverages IdM investments via InCommon Provides collaboration environment for autonomous virtual organizations Supports integration of local, shared, and regional resources

UAB Office of the VP of IT CyberInfrastructure Vision 10 Gigabit Ethernet optical network links major research areas in state High performance computation resources distributed across state Campus grids like UABgrid provide uniform access to computational resources Regional grids like SURAgrid provide access to aggregate computational power and unique resources

Alabama Regional Optical Network Alabama RON is a very high bandwidth lambda network. Operated by SLR. Connects major research institutions across state Connects Alabama to National Lambda Rail and Internet2 – projected completion for 2007

Aggregating Resources UABgrid 2.0, powered by myVocs, to begin pilot operation Summer 2007 Exploring grid interconnection with Alabama Supercomputer Authority and UA System to aggregate resources in state Continuing participation with SURAgrid to aggregate resources in region

UABgrid Background Project grew out of NMI Testbed participation, complemented by participation in developing SURAgrid Initially an integration of campus identity with grid credentials using Pubcookie to issue certificates from UABgrid CA Initial tool integration based exclusively on identity UABgrid CA: credentials used by grid computing courses; part of SURAgrid Bridge CA

Limitations of Initial Version No virtual organization support or other authorization attributes UABgrid CA key escrow limits trust Support for non-UAB users limited Inter-domain trust via web user interface doesn't scale well

Complimentary Activities “NMI Enabled Open Source Collaboration Tools for Virtual Organization” grant explores middleware integration (2003) Mailing list system integration discussions in Internet2 Mlist working group leads to “Shibboleth Systems” insights (2004) myVocs.org developed as demonstration of Shibboleth system (2005) GridShib collaboration expands system reach to Globus-based grid resources (2006) myVocs box built to ease deployment (2006)

“Shibboleth System” Simplified, strict “federation” of one identity provider (IdP) with many resources providers reflects trust model of traditional system environments Using Shibboleth for intra-system attribute transfer supports applications distributed across domain boundaries The system can receive outside attributes from standard Shibboleth IdP federations Essentially a proxy identity provider

myVocs Demonstration virtual organization collaboration environment at myVocs.org Use Shibboleth for identity management and attribute distribution Leverage wealth of open source web applications for VO collaboration tools Globus provides distributed computation foundation GridShib binds Shibboleth and Globus for common attribute foundation

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n Identity Providers

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n Identity Providers

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n Identity Providers Univ Attributes

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n Identity Providers Univ Attributes VO Attributes

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n Identity Providers Univ Attributes VO Attributes

myVocs Solves the Attribute Puzzle IdP 1 IdP 2 IdP 1 IdP n App 1 App n App 2 Identity ProvidersApplications Univ Attributes VO Attributes

A Look Inside myVocs UAB IdP Other IdPs Open IdP UIUC IdP Shibboleth SP VO IdP with GridShib VO Attribute Store VO SP Mail List VO SP Wiki VO SP CMS VO SP Grid Apps Globus SP myVocs

myVocs is a “modern application environment” (in spirit of RL Bob's Middleware picture from this morning) Collaboration application scalability Many users, many organizations, many tools, many kinds of existing infrastructure Deployment manages application access

myVocs box A virtual machine instance of myvocs.org Instantiates working federated platform Allows stand-alone exploration of federation middleware Simplify construction of federated system environments Support development of federated applications Conceptualize complex federations as simple federations in layers

myVocs box Contents Debian GNU/Linux minimal system install Shibboleth IdM infrastructure Simplified group management with Sympa Dynamically allocated collaboration tools GridShib CA and IdP interfaces Short-circuit identity provider Basic tools to support stand-alone operation

Running myVocs box Download virtual machine image from Run it with VMware Player or Server Put myvocs-box IP in /etc/hosts Point browser at Explore VO management & sample web tools

UABgrid 2.0 Use of myVocs collaboration environment architecture resolves limitations of initial version Leverage myVocs box instance as the VO management platform UABgrid CA aligned with PKI-lite GridShib CA supports grid credential assignment without key escrow InCommon federation supplies identities and other useful attributes

UABgrid and myVocs UAB IdP Other IdPs Shibboleth SP VO IdP with GridShib VO Attribute Store VO SP Web Apps VO SP Grid Apps Globus SP

UABgrid running myVocs box Know the network profile configuration Import myVocs box into local namespace Integrate with local trust environment Hook in identity providers Establish virtual organizations Migrate existing resources Integrate new resources

Network Profile Default ports HTTP, HTTPS, SSH. OK No firewall rules. OK Public default root password. Not OK

Import into Namespace “Import” into namespace means assign appropriate local host name Host name change affects system, web server, Shibboleth, and messaging System name is standard host name change process Web server has static rule with default host name Shibboleth has host name in config and metadata Messaging requires Sendmail to masquerade as new host name and to listen on external interface

Integrate with Local Trust Environment UABgrid CA defines PKI trust environment for hosts and users on UABgrid UABgrid CA will define trust foundation for myVocs box and UABgrid metadata Migration from default myVocs box trust configuration delayed temporarily to speed exploration of other parts of implementation Default myVocs config “works” with a false sense of self

Hook in Identity Providers The goal is to make UABgrid an InCommon application InCommon will be primary identity federation for UABgrid UABgrid operating policy for InCommon is being developed Initial draft awaiting review Two levels of access with different attribute requirements: collab tools & compute resources OpenIdP.org in use for initial testing

Establish Virtual Organization VOs are easy to create by way of the Sympa interface HPC Services group has existing virtual organization called the Advanced Technology selected for migration to UABgrid VO (Drupal, mailing list, Connotea, Trac, etc) 6 core members with additional will be used to manage UABgrid using UABgrid (eat own dog food)

UABgrid Management Project cfengine for configuration management All nodes will need Globus + GridShib stack to accept “management” jobs Authorization to execute jobs comes VO role Taking system perspective provides a simplistic model to support construction of infrastructure Still early on, but grid management using the grid infrastructure is the goal

Experience: Authentication Shibboleth clearly sufficient for web applications User certs via GridShib CA interface good for non-web applications Flexible yet consistent session lifetime management needed – can be achieved for now via published practices Essentially, authentication needs can be pretty well satisfied with existing technology

Experience: Authorization Default myVocs authz roles OK for smaller groups (only 3 roles) No central PDP (each app decides meaning of roles) good for enabling integration rather than enforcing it (applications just receive consistent attributes) Managing multiple apps independently can be time consuming, use a small number

Experience: Applications Sample applications in myVocs box are OK for working groups due to scale Sample web applications dated – the current sample apps need to be updated to latest releases and modernized Management of some application features requires file system access – need owner/admin file UI for web applications Need registration UI for additional apps GridShib for Globus is for WS (ie. not SSH)

Experience: Final Thought Don't get lost in the technology. Shibboleth and Globus are just the means to building user-driven, federated system environments

Remaining Tasks Integrate myVocs box with UABgrid trust fabric Migrate existing applications used – requires some development work to address Shibboleth support Integrate additional resources – on-going evaluation of application needs for this and other VOs Migrate other existing working groups to UABgrid 2.0 (a.k.a. buy-in)

The Future UABgrid 2.0 Pilot begins summer 2007 Explore grid-based integration with UA System and Alabama Supercomputer Authority Recruiting additional manpower myVocs box Will continue to be leveraged on UABgrid for development efforts and improved as VO management platform Performance of VM analyzed Ease of administration improved Shibboleth trust management, additional attributes

Acknowledgments NSF ANI “NMI Enabled Open Source Collaboration Tools for Virtual Organization” Office of the Vice President for Information Technology, University of Alabama at Birmingham Projects: SURAgrid, GridShib, Internet2 People: Jill Gemmill, Tom Scavo, Von Welch, Jim Phelps, Michael Schiffers, David Shealy

References UAB CyberInfrastructure Planning UABgrid myVocs & myVocs box OpenIdP.org