TeraGrid Program Year 5 Overview John Towns Chair, TeraGrid Forum Director, Persistent Infrastructure National Center for Supercomputing Applications University of Illinois

2 Our Vision of TeraGrid Three part mission: –support the most advanced computational science in multiple domains –empower new communities of users –provide resources and services that can be extended to a broader cyberinfrastructure TeraGrid is… –an advanced, nationally distributed, open cyberinfrastructure comprised of supercomputing, storage, and visualization systems, data collections, and science gateways, integrated by software services and high bandwidth networks, coordinated through common policies and operations, and supported by computing and technology experts, that enables and supports leading­edge scientific discovery and promotes science and technology education –a complex collaboration of over a dozen organizations and NSF awards working together to provide collective services that go beyond what can be provided by individual institutions

3 Strategic Objectives Objectives determined from considering numerous inputs –user input via various mechanisms surveys, user contacts, advisory bodies, review panels, etc. –technical input from TG staff Planning for PY5 started by identifying 5 high level project strategic objectives –Enable science that could not be done without TeraGrid –Broaden the user base –Simplify users lives –Improve Operations –Enable connections to external resources

Advanced User Support – PY5 Objectives Continue to work collaboratively across TG areas to provide advanced support –activities selected based on user input, TRAC recommendations, ad hoc requests, needs indentified by staff, etc. Advanced Support for TeraGrid Applications (ASTA) –maintain ~25 ASTA projects throughout the year –contributions to Science Highlights Advanced Support for Projects (ASP) –“foundation work” to allow use of scientific codes and applications on the HPC resources. –projects with potential to benefit a number (10+) of users in a domain science or TeraGrid users in general Advanced Support for EOT (ASEOT) –content development and presentation; workshop, panel and BOF participation –outreach to new CI programs: DataNet, iPlant, etc. 4

User Services – PY5 Objectives Further improve: –the responsiveness and quality of problem resolution for our users –the feedback mechanisms by which our users become active partners in enhancing the ability of the TeraGrid By July 2010, ticket statistics will show that consultants communicate at least once per 7-day period with the user regarding the status of any ticket under investigation 2010 user satisfaction survey will show promptness and quality of user support are rated at least 85% 5

User Facing Projects and Core Services – PY5 Objectives Resource Description Repository –further consolidate, integrate resource information in central environment, IIS –simplify tasks for RPs –improve consistency of info for users Migration to integrated backend for Portal, Web, and Wiki –more monitoring, measurement for users Work toward user-created portal logins –integration with Shibboleth authentication IIS-based software catalog –combine CTSS/3 rd -party software information New access tools –job submission, metascheduling in TGUP Customization & Personalization –personalized, dynamic TGUP Home page –domain views –User Discussion Forums –$PORTAL_HOME file space for TG Users 6

Networking, Operations, and Security – PY5 Objectives New Resources –NICS’ Kraken system upgrade –NCAR’s a Sun Ultra 40 system dedicated to data analysis and visualization Single Sign On –complete PSC deployment of backup MyProxy service –complete integration of Shibboleth support into Internet Framework/TGUP develop full trust model for TeraGrid/Campuses start recruiting campuses and growing usage – bridging authorization with OSG and EGEE to support other activities 7

Networking, Operations, and Security – PY5 Objectives (2) Inca improvements –“views” based on user needs and output of QA and CUE –interface for RP administrators to execute tests on-demand –Knowledgebase for errors, causes and solutions Maintain support for Core Services –TGCDB, POPS, account and allocations management, and the TeraGrid Ticketing System Operational Activities –Monitoring: Inca, grid instrumentation, network instrumentation, … –TeraGrid Operations Center/HelpDesk –TeraGrid Network Operations –HPC Operations: coordination of all RP operational activities –Operation Security and Incident Response 8

Data and Visualization – PY5 Objectives Implement Data Architecture recommendations –User Portal integration; Data Collections infrastructure; archival replication services; continued investigation of new location- independent access mechanisms (Petashare, Reddnet) Improve data movement performance and reliability Complete production deployments of Lustre-WAN –develop plans for next-generation Lustre-WAN and pNFS technologies Architect a comprehensive TeraGrid-wide data analysis plan –establish policies, documentation and training that provides a comprehensive data analysis picture to the user community. Expand resources and capabilities in the TG Visualization Gateway via integration of RP tools into TG Vis Gateway 9

Software Integration – PY5 Objectives Expand the TeraGrid system information available via TeraGrid's integrated information services Deliver an application/service hosting capability on one or more TeraGrid resources Spin-off Software Build & Test capability from Core into its own capability kit Improved coordinated capability user documentation –e.g. CTSS documentation is re-organized around user focused use cases 10

Science Gateways – PY5 Objectives Support Services –ability to routinely count end gateway users –create a gateway software listing capability –step by step integration instructions for gateways –straightforward process for making use of and securing community accounts Targeted Support Projects –GridChem –PolarGrid –OSG cloud on TeraGrid via NIMBUS –SIDgrid –SimpleGrid –Earth Systems Science Gateway –Computational Infrastructure for Geodynamics –SCEC and NEES –Cyberinfrastructure for End- to-End Environmental Exploration 11

Training – PY5 Objectives Offerings to include live, synchronous and asynchronous offerings covering all TeraGrid resources and services – expansion of HPC University offerings – conduct at least 50 training sessions, of which 10 will include new content – at least 30 sessions offered synchronously; at least 8 new async topics Establish mechanism for users to provide TeraGrid with feedback on their evolving training needs Work with other regional, national and international groups (e.g. OSG, NCSI, DOE, NASA, etc.) – avoid duplication of effort 12

Education and Outreach – PY5 Objectives Education - continue K-12 and undergrad professional development and curriculum development efforts –SC09 Education Program - 10 summer workshops and Nov event –Plan to conduct over 100 workshops, institutes and tutorials –Materials reviewed and submitted to CSERD (digital library) Outreach - expand outreach, especially with under-served communities –Host TG’09: June 22-25, 2009 in Arlington,VA –Conduct at least two petascale workshops with Blue Waters –Expand Campus Champions to > 60 campuses, with a focus on adding under-represented campuses across the country (e.g. EPSCoR sites) –Conduct outreach to at least 20 Professional Society Meetings –Continue TeraGrid Pathways program Student Engagement - expand opportunities –Student Competitions - TG09, SC09 –Computational science problem of the week launched –Student Internships, REUs and Workshops –Graduate Research Fellows seminars and allocations –Support at least 300 K-12 students and 500 college students 13

EOT-wide – PY5 Objectives (2) EOT highlights –EOT Highlights produced in time for SC09 –EOT monthly newsletter launched and being distributed Evaluation –pilot instruments through April; move into production in May –create database for longitudinal impact studies –conduct first 6-month analysis by December –continue to review and improve metrics of long-term impact 14

External Relations – PY5 Objectives Provide easily accessible information –revise the public web site to improve the content, navigation and ease of access to information Convey the impact of TeraGrid on research, education and society –develop science impact stories –Science Highlights Work with other regional, national and international groups – e.g. iSGTW and other professional societies –focus on under-represented groups 15

Quality Assurance/Common User Environment – PY5 Objectives Quality Assurance Working Group –expediting problem resolution when service failures detected –improving the use of the Inca monitoring framework –validate common user environment (with CUE WG) –develop/propose a more formal process for CTSS software deployment (with Software WG) Common User Environment Working Group –remove barriers to user movement between TeraGrid resources CUE Documentation (CUED) CUE Management System (CUEMS) CUE Build Environment (CUBE) CUE Testing Platform (CUETP) CUE Variable Collection (CUEVC) 16

Project Management – PY5 Objectives Tracking and reporting –track progress against the IPP –report progress against the IPP quarterly and annually provide updates as otherwise requested Change Management –manage change management process for IPP Planning –anticipate coordinating development of IPP for proposed TeraGrid Extension timing issue 17

PY5 Total TeraGrid Budget Distribution TeraGrid budget most closely resembles the RP budgets –RP expenditures ~4x those of the GIG Bulk of TeraGrid funds go directly to –providing facilities to scientific researchers –assuring they have necessary support to make productive use of them 18

PY5 Total TeraGrid Budget Detail Functional Area Staffing Level (FTE) Staffing Costs ($k) Other Costs ($k) HPC Operations38.2 $ 4,147 $ 4,629 Network Operations and Security18.2 $ 2,588 $ 1,219 Core Services13.2 $ 1,435 $ 106 Data and Visualization35.0 $ 4,738 $ 1,674 Scheduling3.8 $ 615 $ 70 Science Gateways14.0 $ 2,153 $ 30 Maintaining Data Collections1.9 $ 168 $ 155 Helpdesk Support11.0 $ 945 $ 47 User Consulting Support33.5 $ 4,304 $ 465 Advanced Applications Support15.5 $ 2,638 $ - Training8.4 $ 1,140 $ 120 Education and Outreach12.8 $ 1,745 $ 692 Software Development6.1 $ 1,247 $ 5 Project Management8.1 $ 1,555 $ 146 Administrative Costs11.8 $ 2,258 $ 1,767 Material and Supplies0.0 $ - $ 414 Publication Costs0.0 $ - $ 42 Consultant Services0.0 $ - Computer Services0.0 $ - Subawards0.0 $ - $ 906 Facilities Costs0.0 $ - Other Costs11.8 $ 2,258 $ 406 Conference Costs (TG08 and SC08)2.8 $ 294 $ 291 Total234.0 $31,971 $11,416 19

BACKUP SLIDES 20

21 PY5 GIG Budget Distribution Most GIG effort goes toward outreach to and support of the scientific community, to help them understand how the TeraGrid can help with their research efforts, creation of the environment needed to support those research efforts, and subsequent support of researchers making use of TeraGrid facilities

22 PY5 GIG Budget Detail Functional Area Staffing Level (FTE) Staffing Costs ($k) Other Costs ($k) HPC Operations0.0 $ - Network Operations and Security5.1 $ 980 $ 221 Core Services6.1 $ 830 $ - Data and Visualization6.0 $ 1,136 $ - Scheduling1.8 $ 337 $ - Science Gateways8.9 $ 1,518 $ - Maintaining Data Collections0.0 $ - Helpdesk Support2.0 $ 207 $ - User Consulting Support15.4 $ 2,517 $ 110 Advanced Applications Support15.5 $ 2,638 $ - Training2.6 $ 454 $ - Education and Outreach5.7 $ 947 $ 335 Software Development5.5 $ 1,144 $ - Project Management5.6 $ 1,192 $ 20 Administrative Costs5.8 $ 1,317 $ 234 Material and Supplies Publication Costs Consultant Services Computer Services Subawards Facilities Costs Other Costs5.8 $ 1,317 $ 234 Conference Costs (TG08 and SC08)0.0 $ - $ 35 Total85.8 $15,217 $ 955

23 PY5 RP Budget Distribution Most RP effort goes to supplying the computing power, data storage, networking, and staff expertise needed to support the research efforts of TeraGrid users

24 PY5 RP Budget Detail Functional Area Staffing Level (FTE) Staffing Costs ($k) Other Costs ($k) HPC Operations38.2 $ 4,147 $ 4,629 Network Operations and Security13.1 $ 1,608 $ 998 Core Services7.2 $ 605 $ 106 Data and Visualization29.1 $ 3,602 $ 1,674 Scheduling2.0 $ 278 $ 70 Science Gateways5.1 $ 635 $ 30 Maintaining Data Collections1.9 $ 168 $ 155 Helpdesk Support9.0 $ 738 $ 47 User Consulting Support18.1 $ 1,787 $ 355 Advanced Applications Support0.0 $ - Training5.8 $ 686 $ 120 Education and Outreach7.1 $ 798 $ 357 Software Development0.6 $ 103 $ 5 Project Management2.5 $ 363 $ 126 Administrative Costs6.0 $ 941 $ 1,533 Material and Supplies0.0 $ - $ 414 Publication Costs0.0 $ - $ 42 Consultant Services0.0 $ - Computer Services0.0 $ - Subawards0.0 $ - $ 906 Facilities Costs0.0 $ - Other Costs6.0 $ 941 $ 172 Conference Costs (TG08 and SC08)2.8 $ 294 $ 256 Total148.2 $16,754 $10,461