International Grid Communities Dr. Carl Kesselman Information Sciences Institute University of Southern California

The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

Enabling International Cooperation l International cooperation valuable, because –Scale of Grid problem is large –Expertise on both sides of Atlantic & Pacific –Important international applications –Cost of noncooperation can be high l Useful cooperation will not just happen but must be explicitly encouraged –Substantial testbed & application projects, jointly sponsored by EU, US, others –Transatlantic Terabit Testbed, etc. –International Virtual Data Grid Laboratory

Grid Forum l IETF like body to codify standard practice l Two meetings held so far, next in April l European Grid forum established to address Europe specific issues

Layered Grid Architecture (By Analogy to Internet Architecture) Application Fabric Controlling things locally: Access to, & control of, resources Connectivity Talking to things: communication (Internet protocols) & security Resource Sharing single resources: negotiating access, controlling use Collective Coordinating multiple resources: ubiquitous infrastructure services, app-specific distributed services Internet Transport Application Link Internet Protocol Architecture

The Grid Physics Network l Petabyte-scale computational environment for data intensive science –CMS and Atlas Projects of the Large Hadron Collider –Laser Interferometer Gravitational- Wave Observatory –Sloan Digital Sky Survey (200 million objects each with ~100 attributes)

Data Grids l Integrate data archives into a distributed data management and analysis Grid l More than storage & network, also e.g. –Caching and mirroring to exploit locality –Intelligent scheduling to determine appropriate replica, site for (re)computation, etc. –Coordinated resource management for performance guarantees –Embedded security, policy, agent technologies for effective distributed analysis

Virtual Data Grids l Only raw data must exist –Dynamic data production l Large extent and scale –national or worldwide, multiple distance scales –large numbers of resources l Sophisticated new services –Coordinated use of remote resources l Transparency in data-handling and processing –Optimize for cost, time, policy constraints, …

Grid Communities & Applications: Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a bunch crossing every 25 nsecs. There are 100 triggers per second Each triggered event is ~1 MByte in size Physicists work on analysis channels. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents Image courtesy Harvey Newman, Caltech

Virtual Data Tools Request Planning and Scheduling Tools Request Execution Management Tools Transforms Distributed resources (code, storage, computers, and network) Resource Management Services Security and Policy Services Other Grid Services Interactive User Tools Production Team Individual InvestigatorOther Users Raw data source GriPhyn Architecture

? Major Archive Facilities Network caches & regional centers Local sites GriPhyn Usage Scenario

iVDGL l International Virtual-Data Grid Laboratory –A place to conduct Data Grid tests at scale –Concrete manifestation of world-wide grid activity –Continuing activity that will drive Grid awareness –A basis for further funding l Scale of effort –For national, intl scale Data Grid tests, operations –Computationally and data intensive computing –Fast networks l Who –Initially US-UK-EU; Japan, Australia –Other world regions later –Discussions w/ Russia, China, Pakistan, India, South America

Structure of the iVDGL

Compute PlatformStorage Platform iVDGL Monitoring Interface iVDGL Mgmt. Interface iVDGL Control Interface Local Management Interface Interface iGLS Experiment Scheduler Health and Status Monitoring iVDGL Configuration Information Access Control and Policy Services iGOC Experiment Management Experiment Data Collection Application Experiments iVDGL Architecture

iVDGL Map Circa Tier0/1 facility Tier2 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link Tier3 facility

iVDGL as a Laboratory l Grid Exercises –Easy, intra-experiment tests first (10-20%, national, transatlantic) –Harder wide-scale tests later (50-100% of all resources) l Local control of resources vitally important –Experiments, politics demand it l Strong interest from other disciplines –HEP + NP experiments –Virtual Observatory (VO) community in Europe/US –Gravity wave community in Europe/US/(Japan?) –Earthquake engineering –Bioinformatics –Computer scientists (wide scale tests)

Conclusions l Application communities for major Grid experiments are international –More communities then those mentioned l International testbeds are coming l Wires are only part of the solution l Common middleware archecture enabling technology