1. The Astro-Wise Pipeline Conceptual Design Review Roeland Rengelink rengelink@strw.leidenuniv.nl Requirements Development Framework Prototype

2. Baseline Requirements Data Flow System User Requirements (URD) overview of the scientific requirements, includes calibration and processing requirements Date Flow System Calibration Plan (CP) detailed description of all calibration procedures Date Reduction Specifications (DRS) specifies the data reduction tasks for the calibration and science data

3. Additional Requirements Multi-instrument support Multi-site operations Database Driven operations Extendable On the fly reprocessing

4. Database Requirements Persistence Object Oriented DB technology Hide schema (code is DB schema) Hide ingest/update Queries Relational DB technology Hide SQL Dynamic (new) User defined objects / tables Multi-Site Integrated bulk data storage

5. User Interface Requirements Developers Use pipeline API to write scripts Command Line users Recipes GUI Pipeline operations Quality Control Processing status

6. Pipeline components Data reduction recipes Data reduction tools Infrastructure Administration User Interface Multiple developers Multiple users Multiple sites

7. Development Framework Deliver usable code Working Understandable Correct

8. Python Object Oriented Programming Modularity Reusability Rapid Application Development Incremental development Inline documentation Understandable code Unit testing framework Working code

9. Unit testing Write a test for each unit of functionality To know that it works To reduce the cost of modifying code We find bugs (also in eclipse and LDAC) But, Requires discipline 25 % overhead

10. CVS Essential for collaborative projects Initial investment A different mind set Requires discipline Commit/Update early and often Only commit working code

11. Astro-Wise Pipeline - Scope In: Calibration operations defined in CP Co-added, calibrated, pointings Out: Mosaics Masking, other than CCD defects Homogenization PSF These could be covered in WP2

12. Astro-Wise Pipeline Prototype - Scope Reduce WFI data Baseline implementation calibration procedures Low-level data reduction routines Technology demo OO database interface Oracle9i back end Technology demo parallel processing New approach under development Establish development framework

13. Astro-Wise Pipeline Prototype - Status See CVS No photometric calibration yet Limited set of recipes Database back ends Parallel processing Documentation