Where do we go from here? “Knowledge Environments to Support Distributed Science and Engineering” Symposium on Knowledge Environments for Science and Engineering November 26, 2002 Mary Anne Scott Dept of Energy Office of Science

Distributed Resources; Distributed Expertise Major User Facilities User Institutions Multiprogram Laboratories Program-Dedicated Laboratories Specific-Mission Laboratories Pacific Northwest National Laboratory Ames Laboratory Argonne National Laboratory Brookhaven National Laboratory Oak Ridge National Laboratory Los Alamos National Laboratory Lawrence Livermore National Laboratory Lawrence Berkeley National Laboratory Sandia National Laboratories Fermi National Accelerator Laboratory Princeton Plasma Physics Laboratory Thomas Jefferson National Accelerator Facility Idaho National Environmental and Engineering Laboratory National Renewable Energy Laboratory Stanford Linear Accelerator Center

3 DOE Office of Science Context Research Pre-1995 Foundational technology (Nexus, MPI, Mbone, …) Distributed Collaborative Experiment Environment Projects (testbeds and supporting technology) DOE 2000 Program (pilot collaboratories and technology projects) 2000-present National Collaboratories Program 2001-present Scientific Discovery Through Advanced Computing (SciDAC) Planning In order to inform the development and deployment of technology, a set of high- impact science applications in the areas of high energy physics, climate, chemical sciences, magnetic fusion energy, and molecular biology have been analyzed * to characterize their visions for the future process of science, and the networking and middleware capabilities needed to support those visions * DOE Office of Science, High Performance Network Planning Workshop. August 13-15, 2002: Reston, Virginia, USA.

4 MAGIC for addressing the coordination problem? A team under the Large Scale Network (interagency coordination) Meets Monthly (1st Wed of each month) Federal participants ANL, DOE, LANL, LBL, NASA, NCO, NIH, NIST, NOAA, NSF, PNL, UCAR Other participants Boeing, Cisco, Educause, HP, IBM, Internet2, ISI, Level3, Microsoft, U-Chicago, UIUC, U-Wisconsin Workshop held in Chicago, Aug editors, contributors and participants from Federal Government, agencies and labs; industry, universities, and international organizations ~100 participants “Blueprint for Future Science Middleware and Grid Research and Infrastructure” Middleware And Grid Infrastructure Coordination

5 Driving Factors for Middleware and Grids New classes of scientific problems are enabled from technologies development High energy physicists will harness tens of thousands of CPUs in a worldwide data grid On-line digital sky survey requires mechanisms for data federation and effective navigation Advances in medical imaging and technologies enable collaboration across disciplines and scale Coupling of expertise, collaboration, and disciplines encourage the development of new science and research. Continuing exponential advances in sensor, computer, storage and network capabilities will occur. Sensor networks will create experimental facilities. PetaByte and ExaByte databases will become feasible. Increase in numerical and computer modeling capabilities broaden the base of science disciplines. Increase in network speeds makes it feasible to connect distributed resources as never before. Science Push Technology Pull

6 Future Science (~5yr) DisciplineCharacteristicsVision for the Future Process of Science Anticipated Requirements NetworkingMiddleware Climate Many simulation elements/components added as understanding increases 100 Tby/100 yr generated simulation data, 1-5PBy/yr (per institution) distributed to major users in large chucks for post- simulation analysis Enable the analysis of model data by all of the collaborating community Authenticated data streams for easier site access through firewalls Robust access to large quantities of data Server side data processing (compute/cache embedded in the net) Reliable data/file transfer (accounting for system/network failures) High Energy Physics Instrument based data sources Hierarchial data repositories Hundreds of analysis sites 100s of petabytes of data Global collaboration Compute and storage requirements satisfies by optimal use of all available global resources Productivity aspects of rapid response Worldwide collaboration will cooperative analyze data and contribute to a common knowledge base Discover of publishe (structured) data and its provenace 100 Gbit/se Lambda based point-to-point for single high b/w flows; capacity planning Network monitoring Track world-side resource usage patterns to maximize utilization Direct network access to data management systems Monitoring to enable optimized use of network caching/ compute, and storage resources Publish/subscribe and global discovery Chemical Sciences 3D simulation sets ( TB) Coupling of MPP quantum chemistry and molecular dynamics simulations Validation using large experiment data set Remote steering of simulation time step Remote data sub-setting, mining, and visualization Shared data/metadata w/annotation evolves to knowledge base ~100Gbit for distributed computation chemistry and molecular dynamics simulations Management of metadata Global event services Cross-discipline respoitories International interoperability for collab. infrastructure, respositories, search, and notification Archival publication