1. Millennium Data Dissemination databases and other services 2012-12-17Millennium Workshop1

2. Raw data: Particles FOF groups and Subhalos Density fields Subhalo merger trees Synthetic galaxies (SAM) Mock catalogues Mock images 2012-12-17Millennium Workshop2

3. SUBFIND Tree Builder Halo Merger Trees SAM SSP FOF Group Finder (Sub-)HalosFOF Groups Simulation Particles Gadget MoMaF’ SAM Galaxies Light-cone Galaxies SkyMaker’SExtractor Synthetic Images SExtractor Catalogues PSF, Noise, Background Scaling IGM MRObs 2012-12-17Millennium Workshop3

4. Virgo-Millennium Database Online access to Millennium simulations etal –portal: http://www/mpa- garching.mpg.de/millenniumhttp://www/mpa- garching.mpg.de/millennium Databases –public: http://gavo.mpa- garching.mpg.de/Millenniumhttp://gavo.mpa- garching.mpg.de/Millennium –private + MyDB: http://gavo.mpa-garching.mpg.de/MyMillennium http://gavo.mpa-garching.mpg.de/MyMillennium Improved database access –MillenniumTAP batch queries –http://galformod.mpa- garching.mpg.de/millenniumtaphttp://galformod.mpa- garching.mpg.de/millenniumtap 2012-12-17Millennium Workshop4

5. Millennium data model (old) 5 5

6. last progenitor main leaf and root Trees in Database 2012-12-17Millennium Workshop6

7. Merger trees particleshalos 2012-12-17Millennium Workshop7

8. Joining Garching and Durham 8 8

9. 2012-12-17Millennium Workshop9

10. Most recent data sets (@MPA) New simulations: –MR-XXL: Angulo etal 2012 –Millennium WMAP7 (Thomas etal, MilliGas?) Updated semi-analytics, with (scaling to) WMAP7, MCMC –Guo etal 2011, 2012 –Henriques etal 2012/3 New synthetic light cone catalogues –Henriques etal 2012 –also some all-sky (SDSS depth) Millennium Run Observatory –Overzier etal 2012, images+Sextractor catalogues Most loaded in new databases –soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) New DB functions: –spatial queries –possibly: scaling, … 2012-12-17Millennium Workshop10

11. AllSky (Paritosh Verma) 2012-12-17Millennium Workshop11 “Red” “Blue” “All” smoothed

12. Most recent data sets (@MPA) New simulations: –MR-XXL: Angulo etal 2012 –Millennium WMAP7 (Thomas etal, MilliGas?) Updated semi-analytics, with (scaling to) WMAP7, MCMC –Guo etal 2011, 2012 –Henriques etal 2012/3 New synthetic light cone catalogues –Henriques etal 2012 –also some all-sky (SDSS depth) Millennium Run Observatory –Overzier etal 2012, images+Sextractor catalogues Most loaded in new databases –soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) New DB functions: –spatial queries –possibly: scaling, … 2012-12-17Millennium Workshop12

13. Spatial queries in the database Spatial3D library ( with Tamas Budavari) Searches for objects in spatial subsets of simulation box is hard C# library for optimising such spatial queries using PH space filling curve Soon in Millennium DBs (needs documentation) 2012-12-17Millennium Workshop13

14. with cover as ( -- some details left out select * from dbo.fSimulationCoverSphere( Simulation::Millennium, Sphere3D::newInstance(0,0,0,20),6) ) select f.* from cover c, millennium..fof f where f.snapnum=67 and f.phkey between c.keymin and keymax and c.shape.ContainsPoint(f.x, f.y, f.z)=1 14

15. Query shape syntax Primitives : –box[ 0,0,0,10,10,10 ], sphere[10,10,10,20] –cone[0,0,0,1,1,1,.1,600] vertex,direction,angle,min-depth,max- depth, –conesegment[0,0,0,1,1,1,.1,500,600] vertex,direction,angle,min-depth,max-depth Combinations : –union/intersection/difference [ shape (, shape)* ] 15

16. courtesy RaulAngulo used in MXXL browserMXXL browser

17. In the future: light cones in SQL (requires galaxy catalogues with dM/dz) 2012-12-17Millennium Workshop17

18. Usage statistics 589 registered users > 17.3 million queries (7532/day) > 287 billion rows (137 million/day) queries,rows,users 2012-12-17Millennium Workshop18

19. Distributions over users 2012-12-17Millennium Workshop19

20. Millennium TAP New DB access service –All MyMillennium users have access Batch queue –longer timeout –capability to kill queries –history Implements IVOA’s Table Access Protocol (TAP) –AJAX frontend: TAPFace –TAP aware tools can query it, TOPCAT Some differences –schema vs database (Non SQLServer DBs can be served) –no mydb (yet) Demo this afternoonDemo 2012-12-17Millennium Workshop20

21. It’s not always as simple as it seems. One should try to understand the data when using the database thanks to Kevin Bundy, Tommaso Treu, Richard Ellis http://astro.berkeley.edu/~kbundy/millennium/ 2012-12-17Millennium Workshop21

22. Other Millennium data dissemination services portal: http://galformod.mpa-garching.mpg.dehttp://galformod.mpa-garching.mpg.de Simulation service(s) –SeSam online (Neistein, Weinmann, Bourges)SeSam –L-Galaxies online (see talk by Matthias Egger)L-Galaxies Image Browsers: –may help define where to point your virtual telescope. –MR-XXL browser (public | interactive) (Angulo),publicinteractive –MRObs image browser (public | private) (Overzier etal)publicprivate 2012-12-17Millennium Workshop22

23. Service vs other data sets Databases –HydroClusters database (Dolag)HydroClusters Virtual telescopes –Smac (Dolag, Reynecke, Bourges)Smac –PHOX (Biffi etal)PHOX Image Browsers: –Phoxy browser + PHOXPhoxy browser 2012-12-17Millennium Workshop23

24. Thank you. 2012-12-17Millennium Workshop242012-12-17Millennium Workshop24