Millennium Data Dissemination databases and other services Millennium Workshop1

Raw data: Particles FOF groups and Subhalos Density fields Subhalo merger trees Synthetic galaxies (SAM) Mock catalogues Mock images Millennium Workshop2

SUBFIND Tree Builder Halo Merger Trees SAM SSP FOF Group Finder (Sub-)HalosFOF Groups Simulation Particles Gadget MoMaF’ SAM Galaxies Light-cone Galaxies SkyMaker’SExtractor Synthetic Images SExtractor Catalogues PSF, Noise, Background Scaling IGM MRObs Millennium Workshop3

Virgo-Millennium Database Online access to Millennium simulations etal –portal: garching.mpg.de/millenniumhttp://www/mpa- garching.mpg.de/millennium Databases –public: garching.mpg.de/Millenniumhttp://gavo.mpa- garching.mpg.de/Millennium –private + MyDB: Improved database access –MillenniumTAP batch queries – garching.mpg.de/millenniumtaphttp://galformod.mpa- garching.mpg.de/millenniumtap Millennium Workshop4

Millennium data model (old) 5 5

last progenitor main leaf and root Trees in Database Millennium Workshop6

Merger trees particleshalos Millennium Workshop7

Joining Garching and Durham 8 8

Millennium Workshop9

Most recent data sets New simulations: –MR-XXL: Angulo etal 2012 –Millennium WMAP7 (Thomas etal, MilliGas?) Updated semi-analytics, with (scaling to) WMAP7, MCMC –Guo etal 2011, 2012 –Henriques etal 2012/3 New synthetic light cone catalogues –Henriques etal 2012 –also some all-sky (SDSS depth) Millennium Run Observatory –Overzier etal 2012, images+Sextractor catalogues Most loaded in new databases –soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) New DB functions: –spatial queries –possibly: scaling, … Millennium Workshop10

AllSky (Paritosh Verma) Millennium Workshop11 “Red” “Blue” “All” smoothed

Most recent data sets New simulations: –MR-XXL: Angulo etal 2012 –Millennium WMAP7 (Thomas etal, MilliGas?) Updated semi-analytics, with (scaling to) WMAP7, MCMC –Guo etal 2011, 2012 –Henriques etal 2012/3 New synthetic light cone catalogues –Henriques etal 2012 –also some all-sky (SDSS depth) Millennium Run Observatory –Overzier etal 2012, images+Sextractor catalogues Most loaded in new databases –soon: MR-XXL trees, Millennium WMAP7 (sim +scaled, halos + galaxies) New DB functions: –spatial queries –possibly: scaling, … Millennium Workshop12

Spatial queries in the database Spatial3D library ( with Tamas Budavari) Searches for objects in spatial subsets of simulation box is hard C# library for optimising such spatial queries using PH space filling curve Soon in Millennium DBs (needs documentation) Millennium Workshop13

with cover as ( -- some details left out select * from dbo.fSimulationCoverSphere( Simulation::Millennium, Sphere3D::newInstance(0,0,0,20),6) ) select f.* from cover c, millennium..fof f where f.snapnum=67 and f.phkey between c.keymin and keymax and c.shape.ContainsPoint(f.x, f.y, f.z)=1 14

Query shape syntax Primitives : –box[ 0,0,0,10,10,10 ], sphere[10,10,10,20] –cone[0,0,0,1,1,1,.1,600] vertex,direction,angle,min-depth,max- depth, –conesegment[0,0,0,1,1,1,.1,500,600] vertex,direction,angle,min-depth,max-depth Combinations : –union/intersection/difference [ shape (, shape)* ] 15

courtesy RaulAngulo used in MXXL browserMXXL browser

In the future: light cones in SQL (requires galaxy catalogues with dM/dz) Millennium Workshop17

Usage statistics 589 registered users > 17.3 million queries (7532/day) > 287 billion rows (137 million/day) queries,rows,users Millennium Workshop18

Distributions over users Millennium Workshop19

Millennium TAP New DB access service –All MyMillennium users have access Batch queue –longer timeout –capability to kill queries –history Implements IVOA’s Table Access Protocol (TAP) –AJAX frontend: TAPFace –TAP aware tools can query it, TOPCAT Some differences –schema vs database (Non SQLServer DBs can be served) –no mydb (yet) Demo this afternoonDemo Millennium Workshop20

It’s not always as simple as it seems. One should try to understand the data when using the database thanks to Kevin Bundy, Tommaso Treu, Richard Ellis Millennium Workshop21

Other Millennium data dissemination services portal: Simulation service(s) –SeSam online (Neistein, Weinmann, Bourges)SeSam –L-Galaxies online (see talk by Matthias Egger)L-Galaxies Image Browsers: –may help define where to point your virtual telescope. –MR-XXL browser (public | interactive) (Angulo),publicinteractive –MRObs image browser (public | private) (Overzier etal)publicprivate Millennium Workshop22

Service vs other data sets Databases –HydroClusters database (Dolag)HydroClusters Virtual telescopes –Smac (Dolag, Reynecke, Bourges)Smac –PHOX (Biffi etal)PHOX Image Browsers: –Phoxy browser + PHOXPhoxy browser Millennium Workshop23

Thank you Millennium Workshop Millennium Workshop24