5 th EU DataGrid Conference, Budapest, September 2002 The European CrossGrid Project Marcel Kunze Abteilung Grid-Computing und e-Science Forschungszentrum Karlsruhe GmbH On behalf of the CrossGrid Collaboration Special thanks to Marian Bubak and Jesus Marco Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft

5 th EU DataGrid Conference, Budapest, September 2002 Outline u The CrossGrid Project u CrossGrid Aplications u Testbed Status u Deliverables and Plans

5 th EU DataGrid Conference, Budapest, September 2002 EU Funded Grid Project Space (Kyriakos Baxevanidis) GRIDLAB GRIA EGSO DATATAG CROSSGRID DATAGRID Applications GRIP EUROGRID DAMIEN Middleware & Tools Underlying Infrastructures Science Industry / business - Links with European National efforts - Links with US projects (GriPhyN, PPDG, iVDGL,…)

5 th EU DataGrid Conference, Budapest, September 2002 CrossGrid Collaboration Poland: Cyfronet & INP Cracow PSNC Poznan ICM & IPJ Warsaw Portugal: LIP Lisbon Spain: CSIC Santander Valencia & RedIris UAB Barcelona USC Santiago & CESGA Ireland: TCD Dublin Italy: DATAMAT Netherlands: UvA Amsterdam Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens AuTh Thessaloniki Cyprus: UCY Nikosia Austria: U.Linz 21 institutes 11 countries

5 th EU DataGrid Conference, Budapest, September 2002 Main Objectives u EU V th Framework Programme (IST) Project, started March 2002 u 21 partners from 11 countries u New category of Grid enabled applications n Computing and data intensive n Distributed n Interactive, near real time response (a person in a loop) n Layered u New programming tools u Grid more user friendly, secure and efficient u Interoperability with other Grids u Implementation of standards

5 th EU DataGrid Conference, Budapest, September 2002 Collaboration with other Grid Projects u Exchange of n Information n Software components u Partners n DATAGRID n DATATAG n GRIDLAB n EUROGRID and GRIP u GRIDSTART u Participation in GGF

5 th EU DataGrid Conference, Budapest, September 2002 Workpackages u WP1 – CrossGrid Application Development n Biomedical simulation and visualization n Flooding crisis support n Interactive distributed data analysis in HEP n Weather forecast and air pollution modeling u WP2 - Grid Application Programming Environments n Tools for parallel programming and debugging on the Grid (MPI) u WP3 – New Grid Services and Tools n Portals and roaming access n Resource management n Monitoring n Optimisation of data access u WP4 - International Testbed Organisation n Integration team, certification authority, support and test procedures n Based on EDG 1.2 (see talk of Marcus Hardt in WP6 parallel session) u WP5 - Project Management n Coordination, architecture, dissemination

5 th EU DataGrid Conference, Budapest, September 2002 Biomedical Application CT / MRI scan Medical DB Segmentation Medical DB LB flow simulation VE WD PC PDA Visualization Interaction HDB 10 simulations/day 60 GB/simulation > 20 MB/s

5 th EU DataGrid Conference, Budapest, September 2002 VR-Interaction

5 th EU DataGrid Conference, Budapest, September 2002 Cascade of Flood Simulations Data sources Meteorological simulations Hydraulic simulations Hydrological simulations Users Output visualization

5 th EU DataGrid Conference, Budapest, September 2002 Example of the Flood Simulation - Flow and Water Depth

5 th EU DataGrid Conference, Budapest, September 2002 Distributed Data Analysis in HEP u Objectives n Distributed data access n Distributed data mining techniques with neural networks u Issues n Typical interactive requests will run on o(TB) distributed data n Transfer/replication times for the whole data about one hour n Data transfers once and in advance of the interactive session n Allocation, installation and set-up of corresponding database servers before the interactive session n Integration of user-friendly interactive access (based on PROOF)

5 th EU DataGrid Conference, Budapest, September 2002 Parallel ROOT Facility: PROOF Local Remote Selection Parameters Procedure Proc.C PROOF CPU TagD B RD B DB 1 DB 4 DB 5 DB 6 DB 3 DB 2

5 th EU DataGrid Conference, Budapest, September 2002 u Distributed/parallel codes on the Grid n Coupled Ocean/Atmosphere Mesoscale Prediction System n STEM-II Air Pollution Code u Integration of distributed databases u Data mining applied to downscaling weather forecast Weather Forecast and Air Pollution Modeling

5 th EU DataGrid Conference, Budapest, September 2002 Key Features of CrossGrid Applications u Data n Data sources and data bases geographically distributed n To be selected on demand u Processing n Large processing capacity required; both HPC & HTC n Interactive u Presentation n Complex data requires versatile 3D visualisation n Support for interaction and feedback to other components

5 th EU DataGrid Conference, Budapest, September 2002 Overview of the CrossGrid Architecture Supporting Tools 1.4 Meteo Pollution 1.4 Meteo Pollution 3.1 Portal & Migrating Desktop Applications Development Support 2.4 Performance Analysis 2.4 Performance Analysis 2.2 MPI Verification 2.3 Metrics and Benchmarks 2.3 Metrics and Benchmarks App. Spec Services 1.1 Grid Visualisation Kernel 1.3 Data Mining on Grid (NN) 1.3 Data Mining on Grid (NN) 1.3 Interactive Distributed Data Access 3.1 Roaming Access 3.1 Roaming Access 3.2 Scheduling Agents 3.2 Scheduling Agents 3.3 Grid Monitoring 3.3 Grid Monitoring MPICH-G Fabric 1.1, 1.2 HLA and others 3.4 Optimization of Grid Data Access 3.4 Optimization of Grid Data Access 1.2 Flooding 1.2 Flooding 1.1 BioMed 1.1 BioMed Applications Generic Services GRAM GSI Replica Catalog GIS / MDS GridFTP Globus-IO DataGrid Replica Manager DataGrid Replica Manager DataGrid Job Submission Service Resource Manager (CE) Resource Manager (CE) CPU Resource Manager Resource Manager Resource Manager (SE) Resource Manager (SE) Secondary Storage Resource Manager Resource Manager Instruments ( Satelites, Radars) Instruments ( Satelites, Radars) 3.4 Optimization of Local Data Access 3.4 Optimization of Local Data Access Tertiary Storage Replica Catalog Globus Replica Manager Globus Replica Manager 1.1 User Interaction Services 1.1 User Interaction Services

5 th EU DataGrid Conference, Budapest, September 2002 Status after M6++ u Software Requirements Specifications together with use cases u CrossGrid Architecture defined u Detailed Design documents for tools and the new Grid services (OO approach, UML) u Analysis of security issues and the first proposal of solutions u Detailed description of the test and integration procedures u Testbed first experience n Sites: LIP, FZK, CSIC+USC, PSNC, AuTH+Demo n Basic: EDG release 1.2 n Applications: s EDG HEP simulations (Atlas,CMS) s first distributed prototypes using MPI: n NN distributed training n Evolutionary Algorithms

5 th EU DataGrid Conference, Budapest, September 2002 CrossGrid Testbed Map UCY NikosiaDEMO Athens Auth Thessaloniki CYFRONET Cracow ICM & IPJ Warsaw PSNC Poznan CSIC IFIC Valencia UAB Barcelona CSIC-UC IFCA Santander CSIC RedIris Madrid LIP Lisbon USC Santiago TCD Dublin UvA Amsterdam FZK Karlsruhe II SAS Bratislava Géant

5 th EU DataGrid Conference, Budapest, September 2002 Sample Testbed Sites Valencia GoG farm, Santander (GridWall), FZK

5 th EU DataGrid Conference, Budapest, September 2002 Plans for the Future u Participation in production testbed with DataGrid n All sites will be ready to join by end of September n Common DEMO at IST 2002, Copenhagen, November 4 th -6 th u Collaboration with DataGrid in specific points (e.g. user support and helpdesk software) u Conference together with RI Forum and the “Across Grids” n Santiago de Compostella, Spain, February 9 th -14 th,2003 n With Proceedings (reviewed papers) u CrossGrid workshop, Linz (w/ EuroPVM/MPI 2002), September 28 th -29 th

5 th EU DataGrid Conference, Budapest, September 2002 Linz CrossGrid Workshop Sep.28 th -29 th u Evaluate the current status of all tasks u Contact partners who are developing software we are going to use: discuss interfaces and functionality u Understand what we may expect as first prototypes u Coordinate the operation of testbeds u Agree about common rules for software development: Standard Operational Procedures (SOP) document written u Start to organize the first CrossGrid EU review u Meet with EU DataGrid representatives u Discuss the technology for the future (OGSA) Details at

5 th EU DataGrid Conference, Budapest, September 2002 CrossGrid Schedule and Deliverables u M6 (now) n WPs 1-3 start to develop prototypes of the applications, tool environment, and the new grid services n WP4 are working on launching the first testbed. u M9 (Dec.2002) n WP4 internal status report n WP5 CrossGrid website u M10 (Jan. 2003) n WP4 First testbed prototype release u M11 (Feb. 2003) n WP5 Project leaflet/brochure n WP5 Dissemination and exploitation report n CrossGrid conference in Santiago di Compostella, February 2002 u M12 (March 2003) n WP1 First software release n WP2 First prototypes n WP3 First prototypes n WP5 Report on requirements on integration and interoperability with DataGrid n WP5 Yearly report

5 th EU DataGrid Conference, Budapest, September s: Internet 1990s: Web 2000s: Grid u Where do we need to get to ? n Applications to support an “e-society” (“Cyber-Infrastructure”) n A Grid infrastructure which hides the complexities from the users (“Invisible Computing”) n A powerful and flexible network infrastructure u Where do we need to invest ? n Applications targeted at realistic problems in “e-science” n Prototypes of Grid infrastructures n Maintain and improve the GEANT network u Expression of Interest for EU FP6 program: n “Enabling Grids and e-Science in Europe (EGEE)” Grid-enabled Applications Prototype Grid Infrastructures Gèant: World Class Networking