Applications Using the EGEE Grid Infrastructure C. Loomis (CNRS/LAL) Planning for Grids in SA Meraka Institute, Pretoria, SA 12-13 May 2008

Contents Introduction Scientific Disciplines on the Grid Grid Functionality Highlighted Applications Project Evolution Conclusions Grid Apps. — C. Loomis — 12-13 May 2008

Introduction Grid: A hardware and software infrastructure… Federation Sharing and coherent use of distributed resources. Use of diverse resources (CPU, storage, DBs, infiniband, …). Collaboration Platform to bring different people with different skills together. Mechanism to analyze, publish, and combine previous results. Goal: Adoption and use by large number of users in wide spectrum of different scientific disciplines. Grid Apps. — C. Loomis — 12-13 May 2008

Scientific Disciplines Growth and diversification of applications. Reported apps. only  underestimate! 6/2006 2/2007 1/2008 Astron. & Astrophysics 2 8 9 Computational Chemistry 6 27 21 Earth Science 16 18 Fusion 3 4 High-Energy Physics 11 7 Life Sciences 23 39 37 Others 14 Total 62 118 117 Condensed Matter Physics Comp. Fluid Dynamics Computer Science/Tools Civil Protection Finance Grid Apps. — C. Loomis — 12-13 May 2008

Growing Usage ~22K CPUs in continuous use Usage doubled this last year Grid Apps. — C. Loomis — 12-13 May 2008

Usage by Scientific Discipline Wide (natural) differences in total CPU utilization. Continued heavy use in HEP and LS. Average 20x increase in other areas! Evidence of broad adoption of grid technology. 2006 2007 TOTAL Astron. & Astrophysics 58 1272 1331 Comp. Chem. 347 7389 7736 Earth Science 32 255 287 Fusion 17 1369 1386 High-Energy Physics 7033 5 166110 236444 Life Sciences 9073 9666 18739 Others 2498 10112 12610 8236 1 196173 278533 Grid Apps. — C. Loomis — 12-13 May 2008

Summary of Use Large, growing overall utilization Long-term, habitual use of infrastructure. Broad adoption Growing number of Virtual Organizations ( ≈ experiments). Uptake in broad spectrum of disciplines. Grid Apps. — C. Loomis — 12-13 May 2008

Functionality EGEE core middleware = gLite Job management Gatekeeper, Workload Mgt. System GridWay, GANGA/DIANE Data management: SRM, FTS, LFC SRB, … Metadata management AMGA VO management VOMS (user and group management, authorization) Grid Apps. — C. Loomis — 12-13 May 2008

CERN LHC 9 km © CERN Geneva © CERN Geneva Grid Apps. — C. Loomis — 12-13 May 2008

ATLAS Experiment 7000 tons 20 m 40 m ATLAS Image: ATL-PHO-GEN-2002-002 Grid Apps. — C. Loomis — 12-13 May 2008

High Energy Physics Data Rate: Data Volume: 40 MHz interaction rate 100 Hz of filtered events 1-10 megabytes per filtered event 0.1-1 gigabytes/second Data Volume: LHC runs 24/7. Generates petabytes of data per year! Plus 1-10 times that in simulated data. Data management is the real challenge for LHC. Recording and retrieval. Metadata management for locating interesting data. Chaotic analysis and large productions. kilo- K 103 mega- M 106 giga- G 109 tera- T 1012 peta- P 1015 exa- E 1018 Grid Apps. — C. Loomis — 12-13 May 2008

Demanding Data Flow Real data: Analyzed data: Simulated data: CERN to others Analyzed data: All to all Simulated data: Jürgen Knobloch Grid Apps. — C. Loomis — 12-13 May 2008

GEOSCOPE GEOSCOPE (http://geoscope.ipgp.jussieu.fr/) Localization of earthquakes Determination of rupture modes of the faults Results within a few hours of major earthquakes date = 6 Jan. 2008 time = 5:14:17 depth = 50.9 km magnitude = 6.1 latitude = 37.150° longitude = 22.934° Grid Apps. — C. Loomis — 12-13 May 2008

Noise Determination Process complete data set on EGEE 25 years of data 28 seismological stations and data center Impact on seismological data center design Grid Apps. — C. Loomis — 12-13 May 2008

Evolution Evolve from R&D projects to service infrastructure. Continuous evolution since European DataGrid (2001). EGEE-III starts 1 May 2008 for 2 years. Planned switch to EGI after EGEE-III. Focus in EGEE-III Support, Community Building, Advanced Functionality Grid Apps. — C. Loomis — 12-13 May 2008

Classic Support Direct User Support VO Support Use GGUS system; grid ticketing system. Provide team for handling tickets. Documentation management and generation. VO Support Registration. Resource allocation. Better tools for VO management (user mgt., communication, …). Application Porting Support GILDA (INFN): porting/training using t-Infrastructure GASuC (SZTAKI): porting to production grid infrastructure Grid Apps. — C. Loomis — 12-13 May 2008

Community Building Build strong, self-reliant user communities. Discipline-specific meetings Techniques to aid each discipline (common data, tools, etc.) Dissemination within that discipline Topical Meetings Discuss common problems or needs Highlight tools/techniques to address those needs Grid Apps. — C. Loomis — 12-13 May 2008

Community Building Conferences = Knowledge Transfer Present results from using grid technology. Discuss encountered problems and solutions. Increase interactions between users. EGEE’07 (Budapest) UF1 (CERN) EGEE’06 (Geneva) UF3 (Clermont- Ferrand) UF2-OGF20 (Manchester) Grid Apps. — C. Loomis — 12-13 May 2008

Conclusions Scientists use the EGEE grid: Long-term goals: Routinely and heavily to speed and to enhance their analyses, To federate (share) their resources, and To collaborate effectively. Long-term goals: Provide infrastructure and full range of support services. Have grid technology routinely used in scientific community. Build strong, self-reliant user communities. Grid Apps. — C. Loomis — 12-13 May 2008