CERN, 25 June 2014CERN-Euclid meeting1 Euclid Science Ground Segment Organisation Fabio Pasian Euclid Consortium SGS Manager (INAF – OATrieste) The presented document is Proprietary information of the Euclid Consortium. This document shall be used and disclosed by the receiving Party and its related entities (e.g. contractors and subcontractors) only for the purposes of fulfilling the receiving Party's responsibilities under the Euclid Project and that identified and marked technical data shall not be disclosed or retransferred to any other entity without prior written permission of the document preparer.

CERN, 25 June 2014CERN-Euclid meeting2 LE1 SOC MOC Ground Station The Ground Segment as seen in the Euclid Science Management Plan Euclid Consortium (ECSGS)

CERN, 25 June 2014CERN-Euclid meeting3 Level 1 Level 2 Level 3 Level E Level S VIS NIRSIREXT LE1 MER SIM SPESHEPHZ LE3 VIS/NIR/SIR/EXT cross-check VIS/NIR/SIR/EXT cross-check SIR cross-check SIR cross-check MER cross-check MER cross-check OPS MOC Ground Station The Ground Segment as seen from the data processing point of view The coloured boxes correspond to the Processing Functions SOC

CERN, 25 June 2014CERN-Euclid meeting4 SGS Product Tree

CERN, 25 June 2014CERN-Euclid meeting5 Processing Functions –are a product of the Euclid SGS (to be eventually delivered to ESA at the end of the mission) –correspond to the processing steps which are performed within an «Euclid pipeline» –are algorithmically devised by the relevant OU and engineered by software development teams (SDC-DEV) –can in principle be run yielding the same results on any SDC site of the SGS (SDC-PROD, different HW environments) In most cases, Processing Functions are developed jointly by OU members and their local SDC-DEV teams –formal OU-SDC interfaces not needed in most cases –easier to develop directly pipeline-quality code –SGS System Team provides tools/standards/support (SDC Leads are members of the System Team)

CERN, 25 June 2014CERN-Euclid meeting6 Development–Verification&Validation SWGsOU SDC-DEV SDC-PROD … requirements validation (on results) code validation algorithms, test data pipeline code, test data pipelines verification 2. only for validation against requirements for every Processing Function 1. in most cases, no interfaces but joint development

CERN, 25 June 2014CERN-Euclid meeting7 «Euclid pipeline» Planned architecture is based on distributed data and processing –Each SDC is both a processing and a storage «node» The most efficient system is the one minimising data transfer (“move the code, not the data”) –Run the pipeline where the main input data is stored No specialised SDC: any pipeline is expected to run on any SDC –Each SDC runs the same code through virtualisation –Possibility of running a full lower level processing pipeline on sky «tiles», up to the preparation of catalogues of objects Better system –Robust (predictable, able to recover from errors and unexpected behaviour) –Reliable (produce the same results given the same input) –Scalable (cope with changing demand, scale, no upper limit) –Maintainable (same software everywhere) The System Team provides the needed tools/mechanisms

CERN, 25 June 2014CERN-Euclid meeting8 SGS Product Tree

CERN, 25 June 2014CERN-Euclid meeting9 Organisation Group Project Office ECSGS Management Config. Lead O. Mansutti PA/QA Lead C. Vuerli IOT Coordination A. Gregorio Proj.Ctr. Support D. Fierro OUs SystemTeam Lead C. Dabin OU-NIR A. Grazian R. Bouwens OU-VIS H. Mc Cracken N. Shane OU-SIR M. Scodeggio C. Surace OU-EXT J. Mohr G. Verdoes-Kleijn OU-MER A. Fontana M.Kuemmel, M.Douspis OU-SIM S. Serrano A. Ealet OU-SHE A. Taylor F.Courbin,T.Schrabback OU-SPE O. Le Fèvre M. Mignoli OU-LE3 J-L. Starck F.Abdalla, E.Branchini OU-PHZ S. Paltani F. Castander SDCs SDC-DE J. Koppenhoefer F. Raison SDC-CH P. Dubath SDC-FI E. Keihanen H. Kurki-Suonio SDC-ES C. Neissner N. Tonello SDC-IT A. Zacchei M. Frailis SDC-FR M. Poncet J-J. Metge SDC-UK K. Noddle M. Holliman SDC-NL O. R. Williams A. Belikov SDC-US J. Rector H. Teplitz Abstraction Layer (IAL) M. Melchior Architecture Performance K. Noddle Data Modeling C. Dabin Monitoring & Control L. Vibert Archive Data A. Belikov Archive Metadata P. Osuna Common Tools K. Noddle Orchestration M. Poncet Data Quality M. Brescia ECSGS Manager F. Pasian ECSGS Scientist M. Sauvage ECSGS Deputy C. Dabin LE1 common infrastructure M.Frailis

CERN, 25 June 2014CERN-Euclid meeting10 Thank you for your attention