1. EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks The Grid for Astronomy and Astrophysics in Italy and Europe Dr. Giuliano Taffoni INAF - IASF Bologna

2. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Memories of the past A&A community using Grid technology since 2003: Pioneering the access to databases Pioneering instruments control via Grid Data Analysis and simulations A&A participates to the EGEE, EGEE-II, EGEE-III Large projects Small projects Interoperability with other eInfrastructures (VObs) EGEE User Forum - Upsala 14 April 2010 2

3. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 The Grid Landscape for A&A EGEE User Forum - Upsala 14 April 2010 3

4. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Projects and Applications EGEE User Forum - Upsala 14 April 2010 4

5. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 The Early Days Planck Satellite Mission: –Pre-launch simulations activities  Involves both EGEE and DEISA –Monte Carlo simulations (post-launch) MAGIC Telescope: – CORSIKA Monte Carlo code  Hadronic showers + Instrument response = simulated data sample EGEE User Forum - Upsala 14 April 2010 5

6. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 MAGIC project Scientific Goals: Dark Matter, Cosmic Rays, Quantum Gravity, Cosmology … The MAGIC collaboration: –21 institutes (mostly in Europe) –~ 200 members Telescope site in Canary Islands –Observatorio Roque de los Muchachos MAGIC-I: operating since 2004 MAGIC-II: in commissioning (2009) EGEE User Forum - Upsala 14 April 2010 6 MAGIC is a Cherenkov telescope system for g-ray astronomy in the very high energy range (VHE, E > 25 GeV)

7. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 MAGIC on EGEE MAGIC VO started in 2004 Initiative by H. Kornmayer et al. Initial project: 2 main areas Monte Carlo production Data storage and reduction Second phase (2009): Grid Data Center Officially @PIC (Barcelona) Data storage Data reduction Data access for the collaboration SEE poster @ UF2010 EGEE User Forum - Upsala 14 April 2010 7

8. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Planck Satellite PLANCK ESA Space Mission (2009). Measure the Cosmic Microwave Background Radiation. Scientific Goals: Cosmology, CMB anisotropies and polarization, galaxy cluster (SZ effect), gravitational lensing, etc. The Planck collaboration: –Europe and USA –More than 500 scientists Launched 14 May 2009 EGEE User Forum - Upsala 14 April 2010 8

9. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 PLANCK VO started in 2004 –Initiative by F. Pasian, G. Taffoni et al. Initial project: Pre-Launch simulations (systematic, testing, etc.) Second phase (2009) Monte Carlo simulations (cosmological parameters, Systematic, etc…) EGEE User Forum - Upsala 14 April 2010 9

10. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Monte Carlo Schema EGEE User Forum - Upsala 14 April 2010 10 CMB Power Spectrum [cmbfast] CMB maps CMB Map [synfast] Foregrounds and Beam Patterns Instrumental Noise Scanning Strategy Destriping Mission simulation TOD

11. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Monte Carlo on the Grid EGEE User Forum - Upsala 14 April 2010 11 ClientUIAMGAWMSCE S E LevelS (Cosmology Instrument) Cosmology Instrument LFN Config process Job Submission(JDLs) LFC Job Input LFN Output LFN Metadata Check status Sandbox(images+data) Notify LeveS BEgrid Seminar - Bruxelles 22 October 2009

12. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Planck Satellite II Marcos López-Caniego, Andrés Curto, Enrique Martínez-González (Instituto de Física de Cantabria) Study of non-Gaussianity through the study of the non linear coupling parameter. –Requires simulation of 10000 maps –Steps already done:  Evaluate the covariance with 1000 Gaussian simulations C(sims)  Evaluate the covariance matrix with the analytical expression C(theo)  Check that the analytical value for the covariance agrees with its estimator with 1000 Gaussian simulations and 1 scale  The differences are < 6%  Estimate the covariance matrix using 21 scales with sims (~40.000 CPU hours)  Estimate the a coefficients using 21 scales analytically (~50.000 CPU hours) EGEE User Forum - Upsala 14 April 2010 12

13. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Kuiper-belt and Oort-cloud formation EGEE User Forum - Upsala 14 April 2010 13 Initiative by: Jan Astalos - Institute of Informatics Slovak Academy of Sciences Scientific Goals: unified theory of the formation of Kuiper belt and Scattered Disc and Oort cloud High computing demanding The collaboration: Slovakia: Astronomical institute, Slovak Academy of Sciences Italy: INAF-Catania Astrophysical Observatory Poland: Astronomical Observatory of the A.Mickiewicz University Initial project: (EGEE-III) Simulation production (EGEE + TriGrid) (MPI, OpenMPI) DIANE to handle complex workflows

14. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Successful Story France A&A community PDR code (Le Petit et al) : study of the interstellar medium –Grid  explore the input parameter space –Necessary to interpret observations Dynamics of the solar system (Laskar & Gastineau) –N-Body problem – What is the probability of collision of Mercure with the Sun or Venus ? EGEE User Forum - Upsala 14 April 2010 14

15. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Successful Story Very High Energy Astrophysics (Lenain): –Multi-wavelength emission of Blazar –On EGEE  30 000 jobs in 3 months  60 000 CPU hours On site support for training and application porting –Guide users trough Grid complexity –Recycle successful tools and procedures –Support and Training for users and site managers Stops for funding reasons EGEE User Forum - Upsala 14 April 2010 15

16. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 BaSTI: Grid and Archives Stellar population model archive (DB). Data created using FRANEC (FranecG). Web portal Highly Computation demanding: –One model = 100 FRANEC runs –One FRANEC run lasts for 1 -> 20 h EGEE User Forum - Upsala 14 April 2010 16

17. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Interoperability EGEE User Forum - Upsala 14 April 2010 17

18. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 The Virtual Observatory “A virtual observatory is a collection of interoperating data archives and software tools which utilize the internet to form a scientific research environment in which astronomical research programs can be conducted.” Service Oriented Arch EuroVO (www.euro-vo.org)www.euro-vo.org EuroVO-DCA –WP4 interaction with GRID EGEE User Forum - Upsala 14 April 2010 18

19. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 VObs and EGEE interoperability EGEE User Forum - Upsala 14 April 2010 19 Scientific Gateway: bridge two eInfrastructures. –VOSpace vs LFC –UWS vs WMS –Auth and Auth Desktop Grid: VODesktop 19 BEgrid Seminar - Bruxelles 22 October 2009

20. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 ESAC example EGEE User Forum - Upsala 14 April 2010 20 Save Results on VOSpace XMM-Newton RISA – Remote Interface for Science Analysis Vizualize Results with VO tools (VOSpec, Aladin) VO Access data VO Display Request VO Search data XMM-Newton Archive Define SAS Tasks Search data with RISA Web client Run SAS tasks on ESAC GRID VO Download data Send SAS Tasks To the GRID GRIDWAY 20 BEgrid Seminar - Bruxelles 22 October 2009

21. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 This is not the whole story Astro-WISE LOFAR Dynamical models for Galaxies ZEN project AUGER Small regional projects EGEE User Forum - Upsala 14 April 2010 21

22. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Data Avalache Images and data from thousand observation and simulations New instruments –40 M telescope (1TBxImage) –New super computers Need to be reduced Need to be analyzed EGEE User Forum - Upsala 14 April 2010 22

23. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Grid And HPC Numerical Simulations New Simulations need to be analyzed Small scale simulations on the Grid PRACE new paradigm TriGrid experience –HPC WG (Dr. Becciani) –www.trigrid.it EGEE User Forum - Upsala 14 April 2010 23

24. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Towards NGIs: Italian Experience Italian Grid Infrastructure for Astronomy www.grid.inaf.itwww.grid.inaf.it Involves Italian Institute for Astrophysics (INAF) and Universities User and site manager support. Specialized services for A&A Frozen for funding reasons EGEE User Forum - Upsala 14 April 2010 24

25. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Grid For Astronomy Fulfill specific Astronomical Requirements Optimize resources in INAF (HPC + small high throughput clusters) Optimize INAF collaborations Develop scientific gateways for Astronomers, portals, workflows etc. Archive and Catalogue access Focused training and support Coordination of Grid High level services EGEE User Forum - Upsala 14 April 2010 25

26. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Lesson Learned EGEE User Forum - Upsala 14 April 2010 26 Grid is not kind to new users CLI tools are complex and not very user-friendly Optimal data access and workflows need experience For specific VO users most of the complexity can be hidden Wrapper CLI tools with familiar look & feel (Scientific Gateways) CLI still available for power users Mature tools: DIANE, JST, WISEDOM etc. Learning curve can be significantly reduced Better user support

27. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Lesson Learned Grid is still far from users we need mediators Some projects are frozen for this reason We are not able to implement a positive loop –The more we teach and implement grid applications the more new projects will loot at grid computing as a possible resource. EGEE User Forum - Upsala 14 April 2010 27