TeraGrid Annual Review: Science Gateways Nancy Wilkins-Diehr TeraGrid Area Director for Science Gateways TeraGrid Annual Review, April 15-16, 2008

You are each responsible for developing a presentation ( minutes) on successes, plans, failures, and needs in your respective areas. Use this to set context for your area since it will probably open a session of Q&A on that area. TeraGrid Annual Review, April 15-16, 2008

Outline Big picture –Importance of gateways and potential for impact Development since inception –Program description –Who can be a gateway? 2007 progress 2008 plans –Objectives –What’s needed for success TeraGrid Annual Review, April 15-16, 2008

Phenomenal Impact of the Internet on Worldwide Communication and Information Retrieval Implications on the conduct of science are still evolving –1980’s, Early gateways, National Center for Biotechnology Information BLAST server, search results sent by , still a working portal today –1992 Mosaic web browser developed –1995 “International Protein Data Bank Enhanced by Computer Browser” –2004 TeraGrid project director Rick Stevens recognized growth in scientific portal development and proposed the Science Gateway Program Simultaneous explosion of digital information – Analysis needs in a variety of scientific areas –Sensors, telescopes, satellites, digital images and video –#1 machine on Top500 today is 300x more powerful than all combined entries on the first list in 1993 TeraGrid Annual Review, April 15-16, 2008 Only 16 years since the release of Mosaic!

Gateways Greatly Expand Access Almost anyone can investigate scientific questions using high end resources –Not just those in the research groups of those who request allocations Fosters new ideas, cross-disciplinary approaches Encourages students to experiment But used in production too –Significant number of papers resulting from gateways including GridChem, nanoHUB TeraGrid Annual Review, April 15-16, 2008

Highlights: LEAD Inspires Students Advanced capabilities regardless of location A student gets excited about what he was able to do with LEAD “Dr. Sikora:Attached is a display of 2- m T and wind depicting the WRF's interpretation of the coastal front on 14 February It's interesting that I found an example using IDV that parallels our discussion of mesoscale boundaries in class. It illustrates very nicely the transition to a coastal low and the strong baroclinic zone with a location very similar to Markowski's depiction. I created this image in IDV after running a 5-km WRF run (initialized with NAM output) via the LEAD Portal. This simple 1-level plot is just a precursor of the many capabilities IDV will eventually offer to visualize high-res WRF output. Enjoy!” Eric ( , March 2007) SAB Meeting, January 14-15, 2008

National Center for Supercomputing Applications The Big Picture

GridChem Used for Production Science Chemical Reactivity of the Biradicaloid (HO...ONO) Singlet States of Peroxynitrous Acid. The Oxidation of Hydrocarbons, Sulfides, and Selenides. Bach, R. D et al. J. Am. Chem. Soc. 2005, 127, The "Somersault" Mechanism for the P-450 Hydroxylation of Hydrocarbons. The Intervention of Transient Inverted Metastable Hydroperoxides. Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006, 128(5), The Effect of Carbonyl Substitution on the Strain Energy of Small Ring Compounds and their Six-member Ring Reference Compounds Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006,128(14), Azide Reactions for Controlling Clean Silicon Surface Chemistry: Benzylazide on Si(100)-2 x 1 Semyon Bocharov et al.. J. Am. Chem. Soc., 128 (29), , 2006 Chemistry of Diffusion Barrier Film Formation: Adsorption and Dissociation of Tetrakis(dimethylamino)titanium on Si(100)-2 × 1 Rodriguez-Reyes, J. C. F.; Teplyakov, A. V. J. Phys. Chem. C.; 2007; 111(12); Computational Studies of [2+2] and [4+2] Pericyclic Reactions between Phosphinoboranes and Alkenes. Steric and Electronic Effects in Identifying a Reactive Phosphinoborane that Should Avoid Dimerization Thomas M. Gilbert* and Steven M. Bachrach Organometallics, 26 (10), , 2007.* TeraGrid Annual Review, April 15-16, 2008

TeraGrid and caBIG geWorkbench – an open source platform for integrated genomics used to Load data from local or remote data sources. Visualize gene expression and sequence data in a variety of ways. Provide access to client- and server-side computational analysis tools such as t-test analysis, hierarchical clustering, self organizing maps, regulatory neworks reconstruction, BLAST searches, pattern/motif discovery, etc. Clustering methods can allow identification of groups of markers with similar expression. A common application is to search for genes that appear to be co-regulated. Validate computational hypothesis through the integration of gene and pathway annotation information from curated sources as well as through Gene Ontology enrichment analysis. TeraGrid Annual Review, April 15-16, 2008 Source:

geWorkbench Integrages TeraGrid Resources TeraGrid Annual Review, April 15-16, 2008 “Although the new service is TeraGrid-aware, the perspective from geWorkbench does not change. As far as geWorkbench is concerned, it is still connecting to a Hierarchical Clustering caGrid service. The difference is now the caGrid service is a gateway service that submits a TeraGrid job on behalf of geWorkbench. geWorkbench, however, does not notice this difference.” Source:

Evolution of the Gateway Program 2004 “TeraGrid Science Gateway” term originates –We will help them build gateway portals that leverage TeraGrid capabilities and provide web-based interfaces to community tools 2005 Initial Gateway requirements analysis team –Areas of identified commonality include: Web services, auditing, community accounts, flexible allocations, scheduling, outreach 2006 –GT4 with web services –GRAM audit, community account policies –presentations to allocations committees –SPRUCE, metascheduling RAT, –Primer –Outreach (NVO and GISolve schools, LEAD demonstrations, presentations - SURA, HASTAC, GEON, CI-Channel, SC, Grace Hopper, MSI-CI2, Lariat, Science Workflows and On Demand Computing for Geosciences Workshop) TeraGrid Annual Review, April 15-16, 2008

Easy Gateway True and False Test Answers Provided TeraGrid selects all gateways (F) TeraGrid designs all gateways (F) TeraGrid limits the number of gateways (F) All gateways need TeraGrid funding to exist (F) Any PI can request an allocation and use it to develop a gateway (T) Gateway design is community- developed and that is the core strength of the program (T) TeraGrid staff are alerted to gateway work when a proposal is reviewed or when a community account is requested (T) Limited TeraGrid support can be provided for targeted assistance to integrate an existing gateway with TeraGrid (T) TeraGrid Annual Review, April 15-16, 2008

When is a gateway appropriate? Researchers using defined sets of tools in different ways –Same executables, different input GridChem, CHARMM –Creating multi-scale workflows –Datasets Common data formats –National Virtual Observatory –Earth System Grid –Some groups have invested significant efforts here caBIG, extensive discussions to develop common terminology and formats BIRN, extensive data sharing agreements Difficult to access data/advanced workflows –Sensor/radar input LEAD, GEON TeraGrid Annual Review, April 15-16, 2008

2007 – Gateways move into production Web Services –Development of common services Steve Mock begins December, 06 QBETS “where can I run soonest” service Auditing –Provide capability to regularly report number of gateway users GridShib Community Accounts –Finalize community account implementation policy –Provide web interface to account details for TG security staff Allocations –Collaboration with xRAC reviewers to develop instructions for gateway Pis Scheduling –Scheduling working group –Urgent computing workshop Gateway Hosting –Available at IU through peer review Outreach –“Build a Gateway” tutorial at TG07 Downloadable code, documentation –Gateways featured in student competition at TG07 –Cross directorate presentation at NSF, May 2007 –LEAD collegiate forecast competition, April 2007 –GISolve, nanoHUB used in classes –NVO announcement of production TG capabilities at conference in China –Pathways supplement includes evaluation of gateway use by educators Interest from Navajo Tech Primer –TG documentation staff identified to move the Primer into fully functional documentation Addressing issues that prevent current gateways from using TG in production Stu Martin begins January, 2007 TeraGrid Annual Review, April 15-16, 2008

Staffing Transitions TeraGrid Annual Review, April 15-16, 2008 Coordination efforts increased from.5 FTE to 1.5 FTE due to: Increase in management responsibility and tracking requirements Improved support for production gateways

GIG-funded Gateway Transitions TeraGrid Annual Review, April 15-16, 2008

Science Gateways currently receiving targeted support Site (s)CY2007 Effort (FTE- months) National Virtual Observatory Caltech3 High Energy PhysicsCaltech3 LEADIU*, NCSA27 nanoHUBPurdue, PSC8 Open Life Sciences Gateway U Chicago12 TeraGrid BioportalUNC12 SPRUCEANL18 GISolve/SimpleGridNCSA4 CSE-OnlineNCSA0.75 ChemBioGridIU1 CIMA x-ray portalIU; collaborators at Purdue, ANL, and elsewhere 2 MutDBIU*1.2 Indiana Spatial Data Portal IU*0.2 Centralized Life Science Database Service (TeraGrid data collection) IU0.1 SIDGridUC, IU*2 ESGNCAR*0.72 CCSM portalPurdue4.6 Purdue Environmental Data Portal Purdue3.0 Neutron Science TeraGrid Gateway (NSTG) ORNL0.75 Earth Systems GridORNL, NCAR0.15 OSG interop.ORNL, Purdue, TG aw a whole, [other specific TG sites? TACC, SDSC?] 0.15 TeraGrid Annual Review, April 15-16, 2008 *Indicates non-TeraGrid funded contributions

Prominent Gateway Needs from 6/07 Zimmerman workshop Basic services that gateways can use instead of creating their own. Templates and standardized systems to save developers the time of recreating things that others have already built. Standardization that would make TeraGrid a real grid that could support the effective use of allocations and meta- scheduling. Operating more effectively as a community in order to better support the education and development needs of gateway developers. TeraGrid Annual Review, April 15-16, 2008 Source: Ann Zimmerman

What is my vision for the end of the program? Gateway Objectives for PY4 and 5 TeraGrid integration will be straightforward for new and existing gateway developers There will be a set of easy to discover general services provided by and for Gateways The targeted support program will be well-organized We will be able to routinely count end gateway users, who will total 25% of total TeraGrid users There will be a funded cross-directorate gateway program at the NSF TeraGrid Annual Review, April 15-16, 2008

Biggest challenges in the next 2 years TeraGrid processes must work smoothly –Allocations, community account requests, security requirements, community software areas, data collection hosting, accounting –TeraGrid must remain a service organizations Gateway developers invest considerable time in TeraGrid integration, this must be worth their while or they won’t remain interested Reliability at scale Counting end users –Per user accounting for gateways Sustainable gateway funding –TeraGrid support is for integration only Adaptability to changing technologies –Application hosting TeraGrid Annual Review, April 15-16, 2008

TeraGrid is a service organization Must occasionally step back and look at the big picture What are our processes like for the end user? TeraGrid is a tool users use to accomplish other goals –They have to worry about funding the science work, teaching class, writing papers, etc. –They don’t have time to become familiar with all the intricacies of our processes, but do their best to understand what they need to know If TeraGrid is too frustrating to use, users will push harder for their own machines We need to make sure they have a good experience Gateway Security Summit, January 28-30, 2008

What’s the path for a Gateway PI? Write allocation proposal –POPS login –Follow our detailed instructions, can only submit larger requests during very specific windows –Justifications, paper listings, renewals each year Supplements and extensions Use add user form to add each and every developer –Go through this process again if a new platform of interest is added If the PI wants a community account –Submit form providing a contact info (again), short and long description of gateway, gateway URL Gateway is then listed on the public page –Get user portal login, login to portal, go to MyTeraGrid and then community account form –Provide contact info (again), script locations, anticipated run sizes, anticipated data needs, IP address Wait for community account setup at sites Go through this process again if a new platform of interest is added Gateway Security Summit, January 28-30, 2008

If the PI wants to stage software somewhere other than a home directory –Request community software area (CSA) First name, last name, disk required, for how long, directory name, group members, for each group member, requested sites –Need to make sure community account group membership does not intersect with CSA group membership –Go through this process again if a new site of interest is added Now the PI has –An allocation –Developer accounts –A community account –A software area Time to run some jobs Gateway Security Summit, January 28-30, 2008

Let the programming begin Developers add TG calls into their own fully developed gateway –GRAM job submissions –Gridftp Identify striped and non-striped servers –Accounting GRAM audit Report to us quarterly on number of end users using gateways Future attributed-based authentication requirements –Credential management Upload logs that include gateway use of TG Discussion over the next two days about how developers can make use of community accounts We need to make sure the TeraGrid experience is worth this level of effort! Gateway Security Summit, January 28-30, 2008

Community Account Portal Under Development Designed for security-wg Password-protected internal site Type in community account username and bring up associated info Write permission for security staffers to add info about how and when these accounts have been secured at each site Repository for Gateway logs –Automatic mechanism and location for gateways to send logs. –Notification when logs are stale Perhaps recent usage data TeraGrid Annual Review, April 15-16, 2008

Tremendous Potential for Gateways In only 16 years, the Web has fundamentally changed human communication Science Gateways can leverage this amazingly powerful tool to: –Transform the way scientists collaborate –Streamline conduct of science –Influence the public’s perception of science Reliability, trust, continuity are fundamental to truly change the conduct of science through the use of gateways –High end resources can have a profound impact The future is very exciting! TeraGrid Annual Review, April 15-16, 2008