A Vision for Core Services 2.0 Core Services 2.0 WG David L. Hart TG Quarterly Meeting, Dec. 7, 2007

What do Core Services encompass? Bringing users into the TeraGrid –Processes and procedures to add new users Getting users set up to use resources –Ensure users have the right credentials on approved resources –Permitting PIs to manage their projects. Supporting the resource ‘economy’ –Includes the Allocations process and POPS –Accounting for usage –Data for Accountability and Metrics

Why do we need Core Services 2.0? Current processes and procedures –Don’t scale –Aren’t well integrated –Haven’t evolved in parallel with new technologies Many new requirements –New capabilities within TeraGrid –General dissatisfaction with various components –A future of 15k-20k users with RPs with the same or reduced staffing but faster turnaround

The Core Services 2.0 process (so far) Assembled a Core2 group with weekly telecons –Jim Basney, John Cobb, Matt Heinzel, Laura McGinnis, Kent Milfeld, Marg Murray, JP Navarro, Steve Quinn, Tomislav Urban, Nancy Wilkins-Diehr (and Mary McIlvain for project management) Document the current baseline components Sent out Core2 members to solicit input –Try to understand the requirements across TeraGrid –Address widely known PI issues and complaints Develop an overall vision supported by a vision for the essential components of Core2

Common themes within the vision Streamlining the various authorization processes –Authorization: Permission/ability to do something Unifying TeraGrid authentication and leveraging it throughout Core Services –Authentication: Knowing who a user is Creating flexible underpinnings for handling resources, projects, and users Improving the interactions between RPs and Core Services

Keeping on target going forward Maintain what works –We can’t afford to toss everything out Improve scalability and reduce latency –Lots of opportunities here Support new requirements –Many of these as well –As new capabilities developed, don’t preclude the ability to support full spectrum of needs

Maintain what works POPS TGCDB AMIE TG User Portal None of these are fundamentally wrong, but they are fundamentally under-scoped. All will require significant enhancements to support Core2.

Improve scalability and reduce latency Allocations –Reducing latency, improving workflow, better RP support POPS –Remove NCSA site dependence Authentication –Unifying POPS/TGUP logins Authorization –Automate as many manual steps as possible Accounting –Improve scalability, reduce latency in AMIE Accountability & Metrics –Reduce dependency on ad hoc queries Resource Provisioning –Simpler interface for the RPs to Core Services 2.0

Support new requirements Allocations –Issues of scale for new resources, and new types of resources POPS –Support for project types, resource types Authorization –Automation, support for other resource types Authentication –Bridging campus enterprise authentication –Attribute-based authentication for Community users Accounting –Support for new resource types, gateway accounting Accountability & Metrics –New metrics for science impact, etc. Resource Provisioning –More consistent interface, interaction

Infrastructure requirements Site-independence –Core2 component infrastructure should be deployable at any site Geographic redundancy –The infrastructure should provide redundancy in the event of a failure at any single site. Operational parameters –Five-year outlook –15-20 RPs –15,000-20,000 users –Track 2 and Track 1 systems

Core Services 2.0 Workflow Resource Requests/POPS Authentication User Resource Advisor Resource Provisioning Authorization Compute Resource 1 Storage Resource 2 Other Type Resource n RP Accountability xRAC, DAC Process Automatic Authorization Process User Request Manual Authorization Process Show User Metrics GRAM 1 TG User Portal Authorized User Authenticated User Register/ Login Check Status/Roles Request Resources Add/Remove Users from TGCDB to TGCDB Publish Resources Informatio n Services Accounting Find Resources Send Auth Instr TGCDB Accept Usage tgusage RDR POPSdb Show Usage

Fundamental inputs for Core Services Users –Current or potential persons who wish to use TeraGrid Resources –Physical hardware deployed at RP sites Resource Descriptions –How RPs present the Resources to Core Services Requests (from users) –User: add/remove users to projects, distribute credentials –Resource: allocation proposals, transfers, etc.

Fundamental outputs for Core Services Authorized users –Users with credentials, with permission to access appropriate resources Authorization instructions/approvals –Mappings of users to projects/resources/allocations Accountability data and measures of metrics –How users, TG/RP staff, and TG/RP management understand what’s going on.

Functions of effective Core Services Establish resources as part of TeraGrid Create new users and distribute credentials Request access to resources Request authorizations for users Enforce authorizations Track and monitor usage Produce measures of metrics

Establish resources as part of TeraGrid Define resources flexibly –Types: Compute, storage, grid, visualization –Tiers: Allocated, Accounted, Authorized, Authenticated –Grids: More than just “TeraGrid Roaming” Publish resources consistently –Information Services: Already deployed –Resource Description Repository: Needed by Core Services to retain resource histories –RDR will “discover” new resources and resource changes automatically.

Create users and distribute credentials No More Postal Mail Unify on TGUP login Let users register themselves at TGUP –Can reset their own TGUP passwords automatically Support project “roles” and responsibilities –PIs, co-PIs, allocation managers Support “types” of users –Individual, Community Users, Training Users –Support Gateway needs Focus on TG SSO –Enable SSO as rapidly as possible –Support alternatives (SSH keys, passwords) with best effort

Request access to resources Allocations & POPS –Proposal submission and merit-review Non-allocated resources –Users may request creation of projects that only need non-allocated resources Designate special project types –Explicitly approved as gateways, training, staff, etc. Selective access to grid-type resources –PIs must pick those resources they plan to use Automatic handling for some requests –For others, permit RP approval without Core2 “middleman”

Allocations and POPS Better guidance for PIs through the Allocations process Deal with the ‘Rachel to Ranger’ scaling issue Resource Advisor –To help find best resource for the job –To help navigate the allocations policies Extract POPS from NCSA site-dependence –Direct linkages from POPS to TGCDB A-to-Z interface evaluation of POPS data entry and functions Easier access to POPS for metrics reporting

Request authorizations for users All users must first be Registered with TGUP Only PI, co-PIs, Allocation Managers can add/remove users –These “roles” approved during project-creation process No staff intervention needed –PIs provide the necessary review –More secure than current processes Creation of special user types –Only by PIs on approved project types –E.g., only Gateway PIs can create Community Users

Enforce authorizations Communicating authorization instructions –RP flexibility –Notifying RPs of status of multi-site allocations Providing real-time verification –To help authenticate users –To help verify authorization status –Support gateway needs –Facilitate use by campus users

Track and monitor usage Accounting++ Usage for non-compute resources tgusage

Produce measures of metrics Limit viewing per authorization level –Public/Registered –PI/Project –TG staff Metrics by resource/project/user types –With explicit types, metrics become easy to report Support for new metrics requirements –User history –Institution categories (MSIs, etc.) –Scientific impact metrics

Next steps for Core2 Implementation plan is next –Early 2008 for first timelines –Prioritizing tasks, identifying staffing levels Portal will be heavily involved, in developing and carrying out the plan Overlap some initial work with planning phase –E.g., eliminating POPS site-dependencies

Feedback? Questions? Priorities? Omissions? –Other external impacts?