TeraGrid Program Year 5 Overview John Towns Chair, TeraGrid Forum Director, Persistent Infrastructure National Center for Supercomputing Applications University of Illinois TeraGrid Annual Review, April 6-8, 2009

2 Our Vision of TeraGrid Three part mission: –support the most advanced computational science in multiple domains –empower new communities of users –provide resources and services that can be extended to a broader cyberinfrastructure TeraGrid is… –an advanced, nationally distributed, open cyberinfrastructure comprised of supercomputing, storage, and visualization systems, data collections, and science gateways, integrated by software services and high bandwidth networks, coordinated through common policies and operations, and supported by computing and technology experts, that enables and supports leading­edge scientific discovery and promotes science and technology education –a complex collaboration of over a dozen organizations and NSF awards working together to provide collective services that go beyond what can be provided by individual institutions

TeraGrid Annual Review, April 6-8, Strategic Objectives Objectives determined from considering numerous inputs –user input via various mechanisms surveys, user contacts, advisory bodies, review panels, etc. –technical input from TG staff Planning for PY5 started by identifying 5 high level project strategic objectives –Enable science that could not be done without TeraGrid –Broaden the user base –Simplify users lives –Improve Operations –Enable connections to external resources

Project Management Tracking and reporting –track progress against the IPP –report progress against the IPP quarterly and annually provide updates as otherwise requested Change Management –manage change management process for IPP Planning –anticipate coordinating development of IPP for proposed TeraGrid Extension timing issue TeraGrid Annual Review, April 6-8, 2009

Project Management – PY05 Objectives

Advanced User Support Continue to work collaboratively across TG areas to provide advanced support –activities selected based on user input, TRAC recommendations, ad hoc requests, needs indentified by staff, etc. Advanced Support for TeraGrid Applications (ASTA) –maintain ~25 ASTA projects throughout the year –contributions to Science Highlights Advanced Support for Projects (ASP) –“foundation work” to allow use of scientific codes and applications on the HPC resources. –projects with potential to benefit a number (10+) of users in a domain science or TeraGrid users in general Advanced Support for EOT (ASEOT) –content development and presentation; workshop, panel and BOF participation –outreach to new CI programs: DataNet, iPlant, etc.

Advanced User Support – PY05 Objectives

User Support Frontline Support Tasks –prompt and successful resolution of complex user problems share best practices and salient lessons across all RPs tiger teams to diagnose and handle cross-RP issues –user engagement user concerns, experiences and suggestions as source for improvements users advocates and agents in the TeraGrid organization user surveys and other formal instruments Frontline Support Methods –User Services Working Group –User Champions –Campus Champions (with EOT) –Pathways To TeraGrid (with EOT and SGW) –New User Training (with EOT) –Extreme Scalability Working Group (with AUS) –Common User Environment Working Group (with NOS)

User Services – PY5 Objectives further improve: –the responsiveness and quality of problem resolution for our users –the feedback mechanisms by which our users become active partners in enhancing the ability of the TeraGrid (1) By July 2010, the ticket statistics should show that consultants communicate at least once per 7- day period with the user regarding the status of any ticket under investigation (2) in the 2010 user satisfaction survey, promptness and quality of user support are rated at least 85%.

User Facing Projects and Core Services Development and Enhancement –User Portal, POPS, Resource Description Repository TeraGrid-wide operational activities –TeraGrid User Portal, web site, and Wiki –POPS, TGCDB and AMIE accounting infrastructure –central documentation and knowledgebase –allocations and accounting monitoring –catalogs, monitors, and news applications –Resource Description Repository

Plans for 2009 TGUP User Password Reset (Mar 09) Resource Description Repository –Further consolidate, integrate resource information in central environment, IIS –Simplify tasks for RPs –Improve consistency of info for users Migration to integrated backend for Portal, Web, and Wiki –Migrate existing and add new capabilities –More monitoring, measurement for users Work toward user-created portal logins –Integration with Shibboleth authentication IIS-based software catalog –Combining CTSS/3 rd -party software information New access tools –Job submission, metascheduling in TGUP Customization & Personalization –Personalized, dynamic TGUP Home page –Domain views –User Discussion Forums –$PORTAL_HOME file space for TG Users

Plans for 2009 TGUP User Password Reset (Mar 09) Resource Description Repository –Further consolidate, integrate resource information in central environment, IIS –Simplify tasks for RPs –Improve consistency of info for users Migration to integrated backend for Portal, Web, and Wiki –Migrate existing and add new capabilities –More monitoring, measurement for users Work toward user-created portal logins –Integration with Shibboleth authentication IIS-based software catalog –Combining CTSS/3 rd -party software information New access tools –Job submission, metascheduling in TGUP Customization & Personalization –Personalized, dynamic TGUP Home page –Domain views –User Discussion Forums –$PORTAL_HOME file space for TG Users

Networking, Operations, and Security Monitoring –Inca, grid instrumentation, network instrumentation, … TeraGrid Operations Center/HelpDesk –24x7x365 first tier support TeraGrid Network Operations HPC Operations –coordination of all RP operational activities Operation Security and Incident Response –single sign-on and authentication services

Networking, Operations, and Security – PY5 Objectives New Resources –NICS’ Kraken system upgrade –NCAR’s a Sun Ultra 40 system dedicated to data analysis and visualization Inca –integration into Internet Framework/TGUP –interface for RP administrators to execute tests on- demand –integration with ticket systems –Knowledgebase for errors, causes and solutions –“views” based on needs and output of QA and CUE

Networking, Operations, and Security – PY5 Objectives (2) Single Sign On –complete PSC deployment of backup MyProxy service –complete integration of Shibboleth support into Internet Framework develop full trust model for TeraGrid/Campuses start recruiting campuses and growing usage – bridging authorization with OSG and EGEE to support other activities

Data and Visualization Development and enhancement –Argonne’s TeraGrid Visualization Gateway –NCAR’s VAPOR software for petascale visualization and data analysis –TeraGrid Data Architecture –data movement tools –global wide area filesystems –pNFS assessment Resources –Visualization Resources TACC’s Spur Argonne’s TeraGrid Visualization Gateway Purdue’s TeraDre NCAR’s Twister –Data Resources GPFS-WAN Data Capacitor Lustre-WAN

Data and Visualization – PY5 Objectives Implement Data Architecture recommendations –User portal integration –Data Collections infrastructure –Archival replication services –Continued investigation of new location-independent access mechanisms (Petashare, Reddnet) Complete production deployments of Lustre-WAN Develop plans for next-generation Lustre-WAN and pNFS technologies Work with CTSS team on continued improvements to Data kit implementations

Software Integration Development and enhancement –automatic resource selection/metascheduling –workflows –integrated information service –build and test service –applications hosting services –centrally operated data movement agent Operations and maintenance –CTSS Kits, central software services, and software packaging services –information service –Scheduling WG –UCSB’s Batch Queue Prediction service –central Build & Test service

Software Integration – PY5 Objectives

Science Gateways

Science Gateways – PY5 Objectives Support Services –Community Accounts –Gateway Registry Targeted Support Projects –GridChem –PolarGrid –OSG cloud on TeraGrid via NIMBUS –SIDgrid –SimpleGrid –Earth Systems Science Gateway –Computational Infrastructure for Geodynamics –SCEC and NEES –Cyberinfrastructure for End- to-End Environmental Exploration

Education, Outreach and Training, and External Relations

Education, Outreach and Training, and External Relations – PY5 Objectives

EOT Plans for PY5 Education - continue K-12 and undergrad professional development and curriculum development efforts –SC09 Education Program - 10 summer workshops and Nov event –Plan to conduct over 100 workshops, institutes and tutorials –Materials reviewed and submitted to CSERD (digital library) Outreach - expand outreach, especially with under-served communities –Host TG’09: June 22-25, 2009 in Arlington,VA –Conduct at least two petascale workshops with Blue Waters –Expand Campus Champions to > 60 campuses, with a focus on adding under-represented campuses across the country (e.g. EPSCoR sites) –Conduct outreach to at least 20 Professional Society Meetings –Continue TeraGrid Pathways program Training - expand HPC, petascale and on-line offerings –Expansion of HPC University offerings –Conduct at least 50 training sessions, of which 10 will include new content –At least 30 sessions offered synchronously; at least 8 new async topics

EOT – PY5 Objectives (2) Student Engagement - expand opportunities –Student Competitions - TG09, SC09 –Computational science problem of the week launched –Student Internships, REUs and Workshops –Graduate Research Fellows seminars and allocations –Support at least 300 K-12 students and 500 college students EOT highlights –EOT Highlights produced in time for SC09 –EOT monthly newsletter launched and being distributed Evaluation –Pilot instruments through April; move into production in May –Create database for longitudinal impact studies –Conduct first 6-month analysis by December –Continue to review and improve metrics of long-term impact

ER – PY5 Objectives Based on the annual TeraGrid User Survey feedback, the ER team will work on revising the public web site to improve the content, navigation and easy access to information of interest to users and potential users The team will continue to work with NSF to convey the impact of TeraGrid on research, education and society The team will continue to reach out to professional societies to share the benefits of TeraGrid for advancing scientific discovery, especially among under- represented communities The team will continue to work with all TeraGrid working groups to provide communications collateral to assist all teams with conveying the value and benefits of TeraGrid

Quality Assurance/Common User Environment Quality Assurance Working Group –improving availability of grid services –improving reliability of grid services Common User Environment Working Group –remove barriers to user movement between TeraGrid resources providing user-driven recommendations and follow-up procedures

Quality Assurance/Common User Environment – PY5 Objectives Quality Assurance Working Group –expediting problem resolution when service failures detected –improving the use of the Inca monitoring framework –validate common user environment (with CUE WG) –develop/propose a more formal process for CTSS software deployment (with Software WG) Common User Environment Working Group –CUE Documentation (CUED) –CUE Management System (CUEMS) –CUE Build Environment (CUBE) –CUE Testing Platform (CUETP) –CUE Variable Collection (CUEVC)