1. 09.04.2008 D-Grid and AstroGrid-D Matthias Steinmetz Matthias Steinmetz (AIP)

2. 2 VODCA: D-Grid and AstroGrid-D 09.04.2008 Trends in astrophysics Panchromatic studies  Combine information of high energy, X-ray, optical, IR, Radio etc. Surveys, populations instead of individual objects Moore‘s law for Optics und Detectors Time resolved astrophysics (PanStarrs, LSST)  Killer asteroids  Extra solar planets „Streaming Telescopes“  LOFAR  LSST

3. 3 VODCA: D-Grid and AstroGrid-D 09.04.2008 20102005 Data Growth and Moore‘s Law 199019952000 1 1000 1000000 Moore's Law (integration density on chips): Factor 2 every 18 months Factor 10 every 5 years Factor 100 every 10 years Data volume (storage): Factor 2 every 12 months Factor 32 every 5 years Factor 1000 every 10 years 10000 100000 100 10

4. 4 VODCA: D-Grid and AstroGrid-D 09.04.2008 National Grid Initiatives in Europe Initially: substantial funding of a few national Grid initiatives (USA, Japan) Many national projects in Europe:  Austrian Grid Initiative  DutchGrid  France: e-Toile; ACI Grid  Germany: D-Grid; Unicore  Greece: HellasGrid  Grid Ireland  Italy: INFNGrid; GRID.IT  NorduGrid  UK e-Science: National Grid Service; OMII; GridPP

5. 5 VODCA: D-Grid and AstroGrid-D 09.04.2008 Funding of VO and Grid for Astrophysics in Germany Application Layer  GAVO  BMBF “large facilities, energy, basic research” Middleware  D-Grid initiative → AstroGrid-D  BMBF “information and communication technologies, new services”

6. 6 VODCA: D-Grid and AstroGrid-D 09.04.2008 D-Grid: First Phase of funding ('05) D-Grid Integration Project (DGI) Seven Community Projects: ~100 partner institutions, Funding: 01.09.05 until 31.08.08 Community Grids Integration project Grid-Ressources, Middleware, Services Project specific development HEP-Grid … Community Grid Middleware Project specific development AstroGrid-D Community Grid Middleware Project specific development Project specific development Project specific development Project specific development Community Grid Middleware Community Grid Middleware Community Grid Middleware MediGRIDC3-GridInGridTextGrid Community Grid Middleware Project specific development Community Grid Middleware WISENT

7. 7 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D Astrophysics Form a grid-based infrastructure for data and compute resources Management of distributed data archives Management of astronomy specific resources (robotic telescopes, gravitational wave detectors) Event management (supernovae, gamma-ray bursts) Networking between major research institutes and smaller university partners

8. 8 VODCA: D-Grid and AstroGrid-D 09.04.2008 C3-Grid Collaborative Climate Community Provide storage and compute resources for scientists Form a grid-based infrastructure für data- and compute resources Develop a unified access scheme for distributed data sources Support user in the analysis of data using standardized work flows

9. 9 VODCA: D-Grid and AstroGrid-D 09.04.2008 HEP-Grid High Energy Physics Goals: Form a grid based Infrastructure for data und compute resources Detection of dark matter and dark energy Detection of the Higgs particle HEP = high energy and particle physics Analysis of huge amounts of data delivered by particle accelerators (CERN, Fermilab, Stanford Univ.) Network including many universities and research institutions

10. 10 VODCA: D-Grid and AstroGrid-D 09.04.2008 In-Grid Engineering Methods and models to solve problems in engineering using the grid: Knowledge based support if decision processes in engineering Support of workflows in the area of engineering Distributed, simulations-based product- und process optimization Virtual prototypes and modeling in industry employing grid methods Casting processes Streaming processes Ground water transport processes Magneto-hydrodynamical coupling in fluid structures

11. 11 VODCA: D-Grid and AstroGrid-D 09.04.2008 MediGrid Health Sciences Goal is a grid supported improvement in the interdisciplinary and location independent collaboration of research institutions in the area of biomedical research MediGrid demonstrates medical research as a gird application using biomedical research with multi dimensional data based on the coupling of various genotypes and phenotypes

12. 12 VODCA: D-Grid and AstroGrid-D 09.04.2008 TextGrid Humanities Modular Grid Platform for distributed and cooperative analysis of text data A community grid for the humanities Goals: the joint philological analysis, annotation, edition and publication of text data in philology, linguistics and related fields To form a interdisciplinary, international and interconnected research platform To demonstrate and use synergy in scientific text data analysis Optimizing scientific research owing to an optimized access to primary sources and tools

13. 13 VODCA: D-Grid and AstroGrid-D 09.04.2008 WISENT Energy meteorology To obtain information for characterizing the fluctuating energy production based on solar and wind power To gain insight owing to interdisciplinary cooperation Climate and Weather (meteorology) Transformation of wind and solar energy (physics) To develop powerful and flexible distributed systems (computer science)

14. 14 VODCA: D-Grid and AstroGrid-D 09.04.2008 D-Grid 2: 12 projects AeroGrid: Aviation and Space Research BauVOGrid: Development and Construction BIS-Grid: Information systems Biz2Grid: commercial grids ("SAP & Grid") D-MON: Monitoring of Resources and Services F&L-Grid: Grid-based IT-Services for Research & Education FinGrid: Grid for the Finance Sector (Stock market) GDI-Grid: Geo-Data Infrastructure PartnerGrid:Cooperation in industrial organizations (KMUs) ProGrid: automobile industry SuGI: Grid-Support für small institutions und KMUs

15. 15 VODCA: D-Grid and AstroGrid-D 09.04.2008 D-Grid 3 call (expected soon) New communities Gap projects Service grids Service layer agreements Knowledge management Extension of D-Grid 1 community projects

16. 16 VODCA: D-Grid and AstroGrid-D 09.04.2008 D-Grid-Infrastructure (03/2008) 10.000 CPU-Cores, 3,3 PB Disk, 5,5 PB Tape

17. 17 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D Astrophysics Form a grid-based infrastructure for data- und compute-resources Management of distributed data archives Management of astronomy specific resources (robotic telescopes, gravitational wave detectors) Event management (supernovae, gamma-ray bursts) Networking between major research institutes and smaller university partners

18. 18 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D Simulations Observations Data analysis Integration of Instruments User Interfaces AstroGrid-D: Tasks Virtual Observatory

19. 19 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D: Tasks LISA sanssouci.aip.de LOFAR Galaxy collisions Post processing X-NBODY6++ Visualization

20. 20 VODCA: D-Grid and AstroGrid-D 09.04.2008 D-Grid Storage Cluster Stella I + II GridSphere Dynamo GridSphere AstroGrid-D: Tasks

21. 21 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D Partner institution Heidelberg ZAH Munich/Garching MPA, MPE, TUM Potsdam / Berlin AIP, AEI, ZIB Organization in three major nodes

22. 22 VODCA: D-Grid and AstroGrid-D 09.04.2008 Middleware development Information system StellarIS Telescope Map Job Management and Monitoring Management of Virtual Organizations Grid Application Toolkit (GAT) and Simple API for Grid Applications (SAGA) Gridsphere Portal

23. 23 VODCA: D-Grid and AstroGrid-D 09.04.2008 Application Monitoring & Steering Prototypes of grid-enabled tools for monitoring and steering beta-release for Cactus :  Cactus Simulation Metadata Management based on AstroGrid-D Information Service Stellaris  Integration in a Cactus User Portal and ein Numerical Relativity Portal using GridSphere  Publicly released Demo & Production Service for Cactus user and NumRel-Community  Cooperation between AstroGrid-D and the Center for Computation & Technology (CCT) at Louisiana State University Baton Rouge, USA

24. 24 VODCA: D-Grid and AstroGrid-D 09.04.2008 CACTUS user portal (based on GridSphere )

25. 25 VODCA: D-Grid and AstroGrid-D 09.04.2008 GridSphere-Portal AstroGrid-D User Portal: http://astrogrid-portal.gac-grid.org Integration of Grid Services  AstroGrid-D MDS  Core D-Grid MDS  AstroGrid-D Resource Map  Simple job submission

26. 26 VODCA: D-Grid and AstroGrid-D 09.04.2008 User Interface GAT Grid Application Toolkit  PBS-Adaptor  SGE-Adaptor  GTK4 compatible  GATJobRun and GATFileOps packages  Java-GAT and CoGKit allows Job-Submission without a full Globus 4 Installation

27. 27 VODCA: D-Grid and AstroGrid-D 09.04.2008 StellarIS: resource monitoring (see talk by Ilyia Nickelt) Grid-Ressource- Map Basiert auf GoogleMap

28. 28 VODCA: D-Grid and AstroGrid-D 09.04.2008 Stellaris: Job monitoring Webinterface for simple job monitoring Time table for submitted Jobs Minutes hours days

29. 29 VODCA: D-Grid and AstroGrid-D 09.04.2008 Robotic Telescopes Development of the REMOTE TELESCOPE MARKUP LANGUAGE (RTML) and application to STELLA-I telescope Can include dynamical information like, e.g., weather Development of a Transformation from RTML to RDF to communicate resource information to the AstroGrid-D Information Service User can find suitable telescopes via SPARQL queries Integration of Web-services via SPARQL queries (e.g. Grid- Resource Map) STELLA-I Robotic Telescopes STELLA-I & II on Tenerife

30. 30 VODCA: D-Grid and AstroGrid-D 09.04.2008 Robotic Telescopes Example of a SPARQL Query STELLA-I SPARQL Query: PREFIX RTML: SELECT ?name WHERE { ?telescope RTML:SpectralRegion "optical". ?telescope RTML:Location ?loc. ?loc RTML:Latitude ?lat. FILTER (?lat > 0). ?loc RTML:name ?name. ?loc RTML:Height ?height } ORDER BY DESC(?height) RDF Notation 3 (N3): @prefix RTML:. RTML:RTML [ RTML:mode "ressource"; RTML:uid "rtml://DE.aip.STELLA-I"; RTML:version "3.1a"; RTML:Telescope [ RTML:name "STELLA-I"; RTML:Aperture [ RTML:type "geometric"; RTML:value "1.2"^^RTML:meters]; RTML:SpectralRegion "optical"; RTML:Location [ RTML:name "Izana Observatory, Teneriffa, Spain"; RTML:Latitude "28.300000"^^RTML:degrees; RTML:EastLongitude "- 16.509722"^^RTML:degrees; RTML:Height "2480"^^RTML:meters ; ]]]

31. 31 VODCA: D-Grid and AstroGrid-D 09.04.2008

32. 32 VODCA: D-Grid and AstroGrid-D 09.04.2008

33. 33 VODCA: D-Grid and AstroGrid-D 09.04.2008

34. 34 VODCA: D-Grid and AstroGrid-D 09.04.2008 GUI jsdlproc GT4.0 JSDLRSL/XML Job Submission Description Language (JSDL) Supported by open grid forum (OGF) Job Management

35. 35 VODCA: D-Grid and AstroGrid-D 09.04.2008 JobMonitoring GT4 Stellaris User-Host Job-Submit EPR Ressourcen Wrapper Status- Queries RDF- Upload SPARQL-Query Job-Timeline GridRessourceMap globusrun-ws –status –j job.epr

36. 36 VODCA: D-Grid and AstroGrid-D 09.04.2008

37. 37 VODCA: D-Grid and AstroGrid-D 09.04.2008 Computers Recent Development: GPU – Graphics Cards GeForce 8800 GTX (NVIDIA) Using CUDA Library Special Interfaces and API from GRACE project ported. Berczik et al. 2008 Spurzem et al. 2008

38. 38 VODCA: D-Grid and AstroGrid-D 09.04.2008 original Codes : S.J.Aarseth, S. Mikkola 4th order time integrator, hierarchical time steps Ahmad-Cohen neighbor scheme Regularization of nearby encounters NBODY6++ (Spurzem 1999) parallelized φGRAPE – (Harfst et al. 2006) special hardware GRAPE/MPRACE NBODY6++ UseCase Hardware accelerator, Gain: factor 50-100 new: Reconfigurable Hardware (FGPA), GRACE-Projekt

39. 39 VODCA: D-Grid and AstroGrid-D 09.04.2008 NBODY6++ UseCase ARI-ZAH GRAPE Cluster: ~3.2 Tflop/s sustained Up to 4 million stars (record holder for direct N-Body simulations) Harfst, Gualandris, Merritt, Spurzem, Portegies Zwart, Berczik 2007, New Astron. 12, 357

40. 40 VODCA: D-Grid and AstroGrid-D 09.04.2008 Cluster Finder Combine Information from optical and X-ray catalogues to identify galaxy clusters  Statistics of thousands of clusters is constraining cosmological parameters Grid for  Compute power  Data transfer from distributed archives  Logistics  complex, distributed workflows optical X-ray Cluster Finder

41. 41 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D: GEO600 / LIGO Laser Interferometer Gravitational Wave Observatory

42. 42 VODCA: D-Grid and AstroGrid-D 09.04.2008 GEO600/LIGO Network von 4 Detectors Hanford (2000m side length) USA Livingston (4000m side length )USA GEO600( 600m side length )Germany Virgo(3000m side length )Italy Pathfinder for LISA, Satellite mission with 3 detectors side length: 5*10 9 m!

43. 43 VODCA: D-Grid and AstroGrid-D 09.04.2008 Gravitational waves: Data analysis via the Grid Data analyses via small data packages, “embarrassingly parallel”. Einstein@Home is, like SETI@Home, suitable to exploit idle cycles on work stations. Einstein@Home is an ideal, simple Grid application, supporting many operation system. Checkpointing and Recovery allows a very accurate control of CPU-Requirements and walltime. Automatic software deployment job submission and job management, a good scalability of grid application can be obtained Current workload: 30000 CPU h per week

44. 44 VODCA: D-Grid and AstroGrid-D 09.04.2008 GEO600 – Resource Integration user friendly User-Management via VOMRS Resource information via MDS und StellarIS Grid Service Monitoring automatic job submission on D-Grid resources Job monitoring und job management using a Laptop data management on Astrogrid-D Storage Cluster

45. 45 VODCA: D-Grid and AstroGrid-D 09.04.2008 AstroGrid-D Astrophysics Form a grid-based infrastructure for data- und compute-resources Management of distributed data archives Management of astronomy specific resources (robotic telescopes, gravitational wave detectors) Event management (supernovae, gamma-ray bursts) Networking between major research institutes and smaller university partners