Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science Grid Consortium

Ruth Pordes November 2004TeraGrid GIG Site Review2 Open Science Grid Consortium Building a community driven engineered Grid infrastructure for large-scale science: Building from Grid3, a 25 site >3000 CPU loosely coherent grid infrastructure through partnering of NSF iVDGL, GriPHyN, DOE PPDG, US LHC software and computing. Commitment from US LHC Tier-1 and Tier-2 centers and DOE Laboratories to make resources accessible by and make contributions to the consortium - esp LHC ramp up over the next 4+ years. Peering with other Grids, especially EGEE/LCG in Europe. Partnering with Grid infrastructures in the US especially TeraGrid as well as university campus Grids. Reaching out to scientific communities beyond the original physics drivers: e.g., biology, chemistry, astronomy. Outreach & education to enable remote and small community participation.

Ruth Pordes November 2004TeraGrid GIG Site Review3

Ruth Pordes November 2004TeraGrid GIG Site Review4 Character of Open Science Grid Distributed ownership of resources. Local Facility policies, priorities, and capabilities need to be supported. DOE and NSF funded involvement. Mix of agreed upon performance expectations and opportunistic resource use. Infrastructure based on the Virtual Data Toolkit. Will incrementally scale the infrastructure with milestones to support stable running of mix of increasingly complex jobs and data management. Peer collaboration of computer and application scientists, facility, technology and resource providers - “end to end approach”. Support for many VOs from the large (thousands) to the very small and dynamic (to the single researcher & high school class) Loosely coupled consistent infrastructure - “Grid of Grids”.

Ruth Pordes November 2004TeraGrid GIG Site Review5 Communities span Grids Users need to run their jobs - transparently and seamless - and access their data across many grid infrastructures Facilities need to make their resources accessible to multiple grid environments Grid infrastructures need to interoperate and federate with standard interfaces and secure and accessible services. OSG users need dynamically installed, VO specific environments.

Ruth Pordes November 2004TeraGrid GIG Site Review6 OSG & Teragrid - goals Sustain a production, scalable, performant common infrastructure for applications over the Grid Deploy, integrate and support higher level more complex services for policy, data management, workflow, system performance and optimization Address needs of diverse applications - especially distributed ad-hoc analyses - with many different users - including dynamic and evolving groups of researchers Common goal of virtualization of resources in support of broad science applications

Ruth Pordes November 2004TeraGrid GIG Site Review7 Opportunities for Partnership Teragrid 5 year plan matches the OSG roadmap of 5-7 years - needed to meet the life cycles of initial scientific participants. Increase effectiveness by configuring access to joint set of resources (which need to expand by orders of magnitude in the next few years). Increase efficiency by enabling use of the best architecture for particular problems by scheduling jobs across TeraGrid and OSG resources. Increase efficiency through joint activities such as testing and deployment of new services. Seamless access to storage & data across two infrastructures.

Ruth Pordes November 2004TeraGrid GIG Site Review8 Initial Goals Access to TeraGrid resources for jobs submitted by OSG users - subject to agreements and policies. Access to OSG resources for jobs submitted by Teragrid users - subject to agreements and policies. Understand and address commonality and interoperability in middleware software stacks. Define policies and agreements in support of interfacing and integration.

Ruth Pordes November 2004TeraGrid GIG Site Review9 Proposed Roadmap Gateway Capability/ServiceGateway Users 6 mo  VDT software installed on TeraGrid systems.  OSG-TeraGrid policy and mechanism planning document.  mo  Two OSG applications ported to TeraGrid systems.  First applications operational on OSG and TeraGrid simultaneously.  mo  TeraGrid supports the manual configuration of “virtual clusters” that can be used to expand the size of OSG while maintaining OSG services.  Further OSG applications operational on TeraGrid.  Community allocations, accounting, authorization.  mo  OSG incorporates TeraGrid-based virtual clusters.  Several OSG applications operational on the expanded system.  mo  OSG incorporates virtual storage resources located at TeraGrid sites.  Several OSG applications make use of TeraGrid virtual storage resources.  Simple coordinated operations.  mo  TeraGrid supports on-demand allocation and configuration of virtual clusters with negotiated service level agreements.  TeraGrid used to support other OSG virtual services, such as data distribution.  mo  OSG applications make routine use of dynamically allocated virtual clusters.  Transfer of loosely coupled tasks from TeraGrid to OSG resources implemented and incorporated into TeraGrid metascheduler.  mo  OSG resources used to support data distribution services for TeraGrid users.  Coordinated incident response.  mo  End-to-end monitoring and problem determination.  Fault tolerant services with dynamic fail over from OSG to TeraGrid.  mo  TeraGrid resources a major source of resources for LHC and other major national data-intensive science communities.  3000