MIRI software; implementation plan Peter Roelfsema

MIRI science support meeting - Amersfoort 11/01/20072 Experience from (HIFI) DP Do take the time to think before you start running Do take the time to think before you start running For Herschel we actually (largely) made what we envisioned when the Herschel operations scenario was established in 1999/2000 For Herschel we actually (largely) made what we envisioned when the Herschel operations scenario was established in 1999/2000 Work out scenario’s; ‘ a day in the life of…’ Work out scenario’s; ‘ a day in the life of…’ Identify all assumptions Identify all assumptions i.e. explicit and implicit assumptions i.e. explicit and implicit assumptions Look carefully at other missions Look carefully at other missions ISO; pipeline initially different between IDT and SOC, later merged into common source code (all IDL) ISO; pipeline initially different between IDT and SOC, later merged into common source code (all IDL) Spitzer; pipeline in SSC is an entirely closed box Spitzer; pipeline in SSC is an entirely closed box Herschel; integration fundamental in system design Herschel; integration fundamental in system design ‘Smooth transition’ pays off ‘Smooth transition’ pays off HIFI is already using the operational system since HIFI DM  early (2 years before launch) DP testing and fast ILT data reduction HIFI is already using the operational system since HIFI DM  early (2 years before launch) DP testing and fast ILT data reduction Make the system user friendly as early as possible Make the system user friendly as early as possible Get early ‘helpdesk’ for Calibration scientists and developers Get early ‘helpdesk’ for Calibration scientists and developers

MIRI science support meeting - Amersfoort 11/01/20073 Steps to get MIRI SW implemented – ‘the plan’ 1. Get the top level view – scenario document What are the aim/needs for the operations phase What are the aim/needs for the operations phase What are the aims/needs for ground calibration What are the aims/needs for ground calibration How are these related; smooth transition between phases? How are these related; smooth transition between phases? What is the relationship between pipeline and interactive analysis What is the relationship between pipeline and interactive analysis What is the timeline, major milestones What is the timeline, major milestones 2. What is needed to support this view Are there systems already supporting this Are there systems already supporting this  e.g. Hershel… or are we fixed/limited by JWST/STScI context? What conversion/new implementation is needed What conversion/new implementation is needed What/how much manpower is needed What/how much manpower is needed 3. How do we implement this

MIRI science support meeting - Amersfoort 11/01/20074 Status 1. Get the top level view  Significant work on establishing data analysis requirements Initial identification of relevant Initial identification of relevant work packages Some initial guestimates of required Some initial guestimates of required effort needed …but not from scratch Framework effort depends on Framework effort depends on whether we can re-use existing systems  …but no progress on top level scenario (since July 2006) Largely due to unavailability of personnel Largely due to unavailability of personnel SubsystemEffort FPATBD Imager 3 my Coronograph LRS 5 my MRS 10 my CalibrationTBD FrameworkTBD

MIRI science support meeting - Amersfoort 11/01/20075 Next steps Priority of activities: 1. Establish top level scenario  april? (…march??) Requires people; are we confident they are available? Requires people; are we confident they are available?  PRR, RSH, BV, JB? 2. Investigate framework approach Can go on in parallel Can go on in parallel HCSS is a good candidate given available expertise HCSS is a good candidate given available expertise Fundamental question; are there ‘new’ processing problems? Fundamental question; are there ‘new’ processing problems? New in the sense of requiring a novel data analysis framework design New in the sense of requiring a novel data analysis framework design Note: a framework could be anywhere between Note: a framework could be anywhere between very expensive and essentially for free Needs coordination within EC but also STScI Needs coordination within EC but also STScI

MIRI science support meeting - Amersfoort 11/01/20076 Next steps – cont. 3. Detailed implementation plan  may? Further detailing of work packages Further detailing of work packages Timeline of development Timeline of development Agreement on who does what Agreement on who does what Requires partner commitments Requires partner commitments Major questions: Major questions: What are important milestones What are important milestones March EC meeting? March EC meeting? Who takes the lead for scenario document Who takes the lead for scenario document Who does the coordination with EC/STScI Who does the coordination with EC/STScI

MIRI science support meeting - Amersfoort 11/01/20077 ‘social’ aspects Fully establish and document responsibilities of participants Fully establish and document responsibilities of participants Who does what, when? Who does what, when? Make that visible to all Make that visible to all We are all in the same boat: establish a trust-based development We are all in the same boat: establish a trust-based development Rely on ‘the others’ to do what they said they would Rely on ‘the others’ to do what they said they would Rely on ‘the others’ to (also) not be totally dumb Rely on ‘the others’ to (also) not be totally dumb Proper communication is essential Proper communication is essential Regular meetings Regular meetings Travel Travel Feedback from users, between developers… Feedback from users, between developers… Documentation and documentation access Documentation and documentation access

MIRI science support meeting - Amersfoort 11/01/20078 Essential design elements Separate system in different components Separate system in different components HIFI-DP as SPG, CA, TA etc as clear (overlapping) subunits of DP HIFI-DP as SPG, CA, TA etc as clear (overlapping) subunits of DP Identify system interfaces as soon as possible Identify system interfaces as soon as possible Internal and external Internal and external …note that FITS is not a priori most suitable as an interface …note that FITS is not a priori most suitable as an interface Identify which operational processes need to be supported Identify which operational processes need to be supported E.g. improve calibration, system distribution and installation, update pipeline module, data search and retrieval, change instrument operating mode (in on board software!) E.g. improve calibration, system distribution and installation, update pipeline module, data search and retrieval, change instrument operating mode (in on board software!) System must be easily adaptable and extensible System must be easily adaptable and extensible Modular and easily changeable Modular and easily changeable

MIRI science support meeting - Amersfoort 11/01/20079 Long term issues Need mechanisms to incorporate ‘new’ instrument knowledge Need mechanisms to incorporate ‘new’ instrument knowledge From calibration team, from instrument team and from astronomers From calibration team, from instrument team and from astronomers Works only if all are using similar (identical?) processing philosophies Works only if all are using similar (identical?) processing philosophies Maintainability Maintainability It still should function many many years from now It still should function many many years from now CPU hardware and OS’s will change many times CPU hardware and OS’s will change many times Documentation should be continuously updated Documentation should be continuously updated Distribution mechanism Distribution mechanism Distributed development has some extra costs… Distributed development has some extra costs… Overhead due to extra communication… and misunderstandings Overhead due to extra communication… and misunderstandings Travel! Travel!