1. raw data recipes SGDP Science Grade Data Products Wolfram Freudling DPD / SDP Group

2. Science grade data products (SGDPs) are data products which can be used as-is to extract scientific conclusions, or to carry out quantitative measurements. Instrument signature removed physical units signal-to-noise ratio close to the optimum error estimates all aspect of data taking and processing fully documented

3. Data Product Survey Nov 2008 About 50% overlap with DFS tickets

4. Proposed Projects

5. Science Ready Pipelines as reported by Survey CRIRES LS Spectroscopy GIRAFFE/MEDUSA Hawk-I UVES VISIR

6. address issues which apply to several instruments Multi-OB image combination Telluric lines tool illumination correction distortions background subtraction detector “monitoring”  Austrian contribution

8. Looking for science-ready instrument to develop Reflex workflow Hawk-I: – Major problem with background subtraction => fixed – Distortions: not enough calibration data needed recipe to analyze distortions distortions not stable => Austrian in-kind

9. Looking for science-ready instrument to develop Reflex workflow Hawk-I: – Major problem with background subtraction => fixed – Distortions: - not enough data - needed recipe to analyze distortions - distortions not stable => Austrian in-kind UVES: – error-bars – Combination of calibrations – CR-like artefacts => most issues fixed draft of report by Daniel Bramich

10. Extrapolation: Every instrument needs substantial work before data products can be considered science-grade Many easy to spot problems Problems often involve both calibration data and pipeline Bottle-neck is the resources available for validation and verification Only 2.5 astronomers work in SDP group => need IOT involvement Conclusions so far

11. New process to set priorities

12. http://eso.org/observing/dfo/sdp/

13. Why? Tickets driven development not good way to set priorities Rush to submit something to pipeline priority meeting PSD resources are spread too thin: – do fewer things with more resources Projects are too software-centric – full support of projects by IOT/SDP Goals of projects are ill-defined – start with good specifications, test plan

14. “Proposals should be articulated, comprehensive, and set some specific measurable goals to improve data quality. “ DP B Proposals will be assessed by Data Products Board. Instrument upgrades should use the same route. SDP Proposals

15. The Board The Data Product Board is chaired by the head of SDP meets twice per year, monthly progress meetings Members: IOT Coordinator, head of LPO/PSO, the head of DMO/DPD, head of SDD/PSD, representative from INS/IPD First meeting: Feb 2011

16. but... IOT/IS does not have the time, resources, motivation... to write a SDP proposals It is de-motivating if IOT proposal needs DPB approval We will not be able to process some data if operational critical tickets are not addressed right away I prefer to talk directly to software developer We should work only on projects which we can complete. Approved projects get lots of resources. We need to use our resources efficiently. This is understood and accepted.

17. What should be in a proposal / data products requirements Specific requirements for data quality Proposal is not compilation of tickets Separate requirements from implementation suggestions Prototypes for algorithms are great, but... Detail resources

18. Effect on Pipeline Development All pipeline development frozen be default Tickets should be submitted but will not be attended immediately. Exceptions have to be approved on a case- by-case basis “new” pipelines under development not affected.

19. Pipelines not frozen VIMOS: support for upgrade XSHOOTER: still in development for several modes VISIR: basic support for the new detector only; additional development must go through an enhancement proposal OMEGACAM: support for commissioning VIRCAM: pipeline still in development for several modes PRIMA: still in development

20. Questions? Comments?