1. Astronomy toolkits and data structures Andrew Jenkins Durham University

2. Data requirements of cosmological simulations Adrian Jenkins Durham University

3. 3 Talk outline DiRAC and its major users New astronomical instruments and missions Mock catalogues Millennium simulation and database Future directions for simulations

4. DiRAC 2 facility Cambridge HPC Service: data analytic cluster Cambridge COSMOS shared memory service Durham ICC Service: data centric cluster (6720 core - idataPlex) Edinburgh 6144 node Bluegene/Q Leicester IT services: complexity cluster

5. DiRAC2 facility used by Time allocated by RAC. Supports large projects (up to 3 years), and smaller allocations. Large users: UKQCD Virgo Consortium (UK) UKMHD Horizon, Leicester …

6. JWST Launch date: ~2017- 8 Cost >$5 billion EUCLID Launch date:~2019 Cost ~€500 million

8. Future large surveys Photometric e.g. Pan-STARRs, DES, LSST, Euclid-VIS Spectroscopic e.g. BOSS, BigBOSS, Euclid-NIS Multi-wavelength e.g SKA (HI) Wide-field (>10,000 sq deg), wide redshift (z=0-3) z-surveys: 10-50 million galaxies imaging surveys ~billions of galaxies

9. Why build a mock? Test galaxy formation models Test algorithms - validation Test processing pipelines Assess survey performance (FoM) Large surveys need mocks now!

10. Mock catalogues need observables SFR SFH Stellar mass Cold gas mass Black hole mass images Full SED (UV, Optical, FIR, Radio) Galaxies : stars, gas, AGN

11. Euclid OU-LE3 requirements for simulations CSWGOU-SIM Cosmological simulators Instrument simulators

12. Generic needs from Euclid Position, redshift Emission line properties/spectra Line flux, equivalent width Broad photometry to AB~24-24.5 Euclid NIR Euclid VIS Pan-STARRS griz DES grizy CFHTLS ugriyz WFCAM ZYJHK SDSS ugriz VISTA-VHS-VIDEO ZYJHKs Photometric redshifts

13. Specific needs: clustering 1% P(k) accuracy Covariance estimates: P(k) etc Initial conditions for reconstruction Different cosmologies Different galaxy formation models (vary bias)

14. Specific needs – clusters of galaxies DM haloes M>1.e+13Msun, r(Δ  Δ  2500, 500,200; velocity dispersion along axes from DM particles For each galaxy host halo ID, central or sat? Simulated images for cluster detection and mass determination through weak/strong lensimg

15. Specific needs: weak lensing Galaxies and DM to generate kappa map Galaxy shapes with noise (no IA) Galaxy shapes with IA Shear at each galaxy position Image properties: mask, bright stars, chip boundaries, CCD defects, ghosts, variations in depth & background

16. 16 Infrastructure required to make mocks Require large simulations To date these have been simulations of dark matter in large cosmological volumes.

17. 17

18. 18

19. 19

20. 20

21. 21

24. 24 Input simulations Large N-body simulations Approaching a trillion particles

25. MXXL simulation

26. 26

27. 27 Future needs Simulations for Euclid multi-trillion particle simulations Produce multi-petabyte datasets Data growing faster than network capabilities Need to scale databases up Ideally would like to serve the raw simulation data - two or more orders of magnitude larger.

28. 28 Current and future simulations

29. 29 Summary Cosmological simulations are required to make the best use of observatories and space missions The size of the required simulations makes this a Big data problem Databases have proved very successful way of presenting processed data Making the raw simulation data public desirable - but very challenging given financial constraints.