1. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 1 General Introduction Yves Jung

2. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 2 Pipeline Usage Automatically in Paranal on a dedicated platform (Instrument Scientist) Automatic data classification Automatic association with the calibration data Requires robustness Interactively in Garching to deliver properly calibrated data packages (DFO group contact person) Better calibration data association improves data products quality Most extensive usage brings most of the feedback External user once the public release has been achieved Only possible when the pipeline has been validated Usually from Gasgano Recipes need to be clean (no memory leak allowed)

3. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 3 Pipeline Requirements A recipe should never crash ! Proper errors handling Meaningful error message required No memory leaks allowed ! Easy to check (use --mem-check esorex option) Clean code makes it easy to avoid them A recipe should run faster than the input data exposure time

4. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 4 Pipeline Packages Dictionaries in iiinstrument-dic-x_y_z.tar.gz Defines QC keywords: ESO-DFS-DIC.IIINSTRUMENT_QC Defines raw files header keywords: ESO-VLT-DIC.IIINSTRUMENT_ICS, ESO-VLT-DIC.IIINSTRUMENT_OS, etc… Configuration and Calibration files in iiinstrument-calibdb-x_y_z.tar.gz Calibration data for the automatic data reduction in Paranal Association rules between data and reduction recipes Configuration files for Gasgano Data Reduction Package in iiinstrument-x_y_z.tar.gz Instrument specific data reduction library Recipes Pipeline user’s manual (for the public release)

5. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 5 Pipeline Dependencies FITS IO Currently QFITS 6.2.0 (http://www.eso.org/projects/aot/qfits) CFITSIO replaces QFITS from CPL 4.0 on Common Pipeline Library Current version 3.1 (http://www.eso.org/cpl) Version 4.0 planned for October 2007 Recipes execution Esorex (currently 3.6.1 from http://www.eso.org/observing/cpl/esorex.html) Gasgano (currently 2.2.4 from http://www.eso.org/observing/gasgano) Additional dependencies (See C. Izzo presentation)

6. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 6 Pipeline Template iiinstrumentp.tar.gz is a pipeline template Instrument name: IIINSTRUMENT Recipe name: rrrecipe Version number 0.0.1 Implements the standard way of reading keywords, loading and saving data (See S. Castro presentation), … Can be installed and executed without modification Should be the starting point for a pipeline project The Technical Developers manual describes the different parts of the template The template contains two main subdirectories iiinstrument/ contains the instrument specific data reduction library recipes/ contains the recipes (that usually call functions implemented in the library)

7. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 7 First steps Read the Deliverables Specification for VLT/VLTI Instruments document (VLT-SPE-ESO-19000-1618) Read the Technical Developers manual (VLT-MAN-ESO-19500-3349) Install QFITS, CPL, ESOREX, GASGANO, autotools (See R. Palsa pres.) Unzip/untar iiinstrumentp.tar.gz Rename iiinstrument, Iiinstrument, IIINSTRUMENT, rrrecipe everywhere (Use a 5 letters name for iiinstrument !!!) Put the new package under CVS Follow the development plan with your contact at ESO/SDD/DFS

8. SDD/DFS Y. Jung VLT 2 nd Generation Instrumentation Pipelines, 19 Apr 2007 - 8 During the development Pay attention to the memory handling (See J. M. Larsen presentation) Follow unit tests procedures (See L. K. Lundin presentation) Run acceptance tests (See A. Modigliani presentation)