Gaia, next frontier in Astronomy Jose Hernandez Gaia Data and Calibration Engineer European Space Astronomy Centre (ESAC) Madrid, Spain

Big Data in The Scientific World IT has evolved at an impressive rate over the last decades In parallel and partly driven by the above the IT resources needed have grown at a similar pace When Gaia was being designed it was not clear if the IT could cope with the Data Processing needs…

Thinking in Big Terms Astronomy Data growing at an incredible rate also: – More and better instruments – In Space missions more bandwith, on-board storage – Better IT to do the data Processing and Analysis Lots of Data waiting out there: There are more Stars in the Universe than grains of sand in all the Beaches on Earth together: – Our Galaxy: Stars – Universe: Galaxies – Stars: Would need 1 yottabyte to store the positions – Yet there are more molecules in a glass of water…

Gaia is the next Astrometry mission of the European Space Agency Will be launched towards the end of 2013 Main Objectives: – Build a 3D map of the Galaxy – History and evolution of the Milky Way – Stellar Astrophysics – Multiple systems, exoplanets – Solar System Asteroids – General Relativity

Vault of Heaven

We have stereoscopic vision, each eye perceives a flat image and our brain builds a 3D image This doesn't work for the stars because they are very far away But we can compare the images taken from two opposite points of Earth's orbit

We need to measure extremely small angles It took a long time (1838) to know the distance to the nearest stars ESA's Hiparccos satellite cataloged stars Gaia will be a giant leap in the field: 1000 million y 100 times more precise

1 GigaPixel camera Will observe the stars from different directions and at different times

Blue Photometer CCDs Red Photometer CCDs One Giga Pixel Camera Star Images Motion 2 Radial Velocity Spectrometer CCDs 2 Radial Velocity Astrometric Field CCDs 106 CCDs, 938 million pixels, 2800 cm cm 42.35cm Sky Mapper CCDs

Two telescopes with a 35m focal length 10 Mirrors Common focal plane Prisms in front of the CCDs determining the colors An spectrometer to measure the radial velocity of the brightest stars

Not all the CCD image is read and sent to Earth The onboard Video Processing Units read the areas containing the star images Data is stored compressed in the memory onboard Data is sent to Ground at night (50GB per day)

Gaia Data Processing

DPAC is an European Consortia in charge of the Data Processing 440 Members 75 European Institutions Six Data Processing Centres: – Barcelona – Cambridge – Geneve – Madrid (ESAC) – Toulouse – Torino

Multidisciplinary Teams: – Astrometry experts – IT Engineers – Calibration Engineers – Mathematicians, Staticians – Coordinators – Computers, , Skype, meetings… Lennart Lindegren, Lund (Sweden) Java Workshop, Toulouse Mare Nostrum, Barcelona

Data Processing Very complex due to the large amounts of data and the precision needed As daily data arrives preliminary processing to verify the performance When we have data of the whole sky we start the global Data Processing Then the process will be iterated including more data and more centres. By mid-mission there will be an intermmediate catalogue Final catalogue towards 2021

After Gaia Europe will continue to lead Astrometry Biggest camera ever flown into Space Reference catalogue in Astronomy for the decades to come Astronomy will be different after Gaia But surely the best discovery will be something that we can not imagine today.

Data Management System On the ESA part of the processing we have two main demands: Processing of the daily Data received from the Satellite Global Processing of the Data accumulated (over 6 months, 1 year, 2 years,…) in an iterative manner Need to select data using a few configurable patterns Daily processing more stringent robustness requirements

Data Management System For the last 10 years test using an increasing amount of data Systems all in Java, using Beowulf clusters and NetApp storage One of the critical issues compromising scalability is the IO Daily Processing needs to be finished on time Global processing needs to read many times large amounts of data (50 TB)

Data Management System Since 2008 we have a fruitful partnership with InterSystems A very good symbiosis Developed prototypes and bring in together the InterSystems/NetApp/ESA experts Work very closely with InterSystems engineers, fast turnaround in new features implementation, problem solving,… Now all our systems in production have migrated to Cache