CAA-OR (End of Phase 1) CAA DWP Operations Review 2007-08 K. Yearby, I. Bates, H. St. C. K. Alleyne Space Systems Group, University of Sheffield, U.K.

Presentation Overview Introduction Summary of the status of the archive Recommendations of the OR-2 review board Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Archive Status (as of April 2008) ICD ID 2001 2002 2003 2004 2005 CP_DWP_TCOR Ingested Ingested Ingested Ingested Ingested CP_DWP_COR Validating Validating Ingested Ingested Ingested CP_DWP_LOG Ingested Ingested Ingested Ingested Ingested CP_DWP_UT_PIOR Delivered Delivered Delivered Delivered Delivered ICD ID Initial delivery Update CP_DWP_TED Ingested - CP_DWP_WEC_UM Ingested -

OR-2 Review Board Recommendations The board was pleased that responsibility for TED and the correlator has been satisfactorily transferred to Sheffield. R30: The DWP team have provided a clear data delivery schedule with milestones – the board urges that this schedule should be adhered to. DWP response: The DWP_TCOR dataset has been delivered in accordance with the schedule given in the 2007 annual report. Regrettably, this has not been the case for the other data products, although with the exception of the caveats, and the DWP_COR datasets for 2001 and 2002, all products have been delivered prior to this final review.

Presentation Overview Introduction Status of the instrument data products delivery DWP_TCOR DWP_COR_FX, ST DWP_LOG DWP_UT_PIOR Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Instrument Data Products Delivery Status DWP_TCOR The WEC time correction files allow the standard 2ms accuracy of the RDM timestamps to be improved to about 10μs. The correction consist of two terms: The offset specifies the difference in microseconds between the reference time (VC0 reset pulse) for the packet and the onboard time stamp for that packet The diff specifies the difference between the onboard time and UTC and is derived from time difference (basically onboard time minus ground receive time) measurements provided by ESOC, similar measurements extracted from the Wideband DSN data, and analysis of the time stamps of the housekeeping telemetry All DWP_TCOR data for the first phase (2001-2005) has been delivered and is online in the archive. In addition data for 2006 is delivered already. Further production is waiting for the availability of the WBD raw data DVDs for 2007 onwards.

Instrument Data Products Delivery Status DWP_COR_FX, ST The correlator data was originally the responsibility of the University of Sussex. Following their loss of funding this was transferred to Sheffield. The Sussex production software was not suitable for large scale use and omitted some important support data, so a new suite of production software was developed based loosely on that used for the CSDS (prime and summary parameter) dataset. This is used to process up to six months data at a time, including validation plots which are visually inspected before delivery. Data for 2001 and 2002 have been processed and are currently subject to manual validation before delivery to the CAA. Data for 2003 to 2005 inclusive has been delivered to the CAA and ingested.

Instrument Data Products Delivery Status DWP_LOG These are produced by automatic analysis of WEC telemetry. Validation is performed by translating CEF files to CSV (comma separated value) format, importing to a spreadsheet, and visually inspecting the result. Data for 2001 to 2005 inclusive has been delivered to the CAA and ingested. DWP_UT_PIOR These files already exist in the DWP planning system as plain ASCII files. For delivery to the CAA a procedure to process each planning period file has been developed to convert the plain ASCII files to CEF by inserting appropriate field and record delimiters, removing predicted scientific events that fall outside the particular planning period and extracting the names of the files used in the generation of the ASCII PIOR file to store in the CEF caveat metadata of the newly generated PIOR CEF file. Data for 2001 through to 2005 inclusive have been delivered to the CAA.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Calibration and Cross-calibration activities DWP does not have its own sensors, but processes data from other WEC instruments, and PEACE. DWP does not currently participate in any of the cross calibration activities, but we are happy to assist if needed.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA DWP_TED Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Software Delivery Status No DWP software is planned to run in the archive, but… DWP_TED The WEC telemetry extraction and decommutation (TED) software is managed by Sheffield. The latest version is 2.5.0.108 (based on version 2.4.3) and appears satisfactory for those using it. It is archived as documentation for reference purposes only. The TED software is the first stage in the processing of WEC science data by all WEC teams. It can be used in two ways: As a stand alone application (levelone) reading the RDM, extracting the data for one or more WEC instruments, and writing it to decommutated science data (DSD) files. As a library, allowing application software to read the WEC RDM directly.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the documentation delivery DWP_WEC_UM DWP_ICD Overview of the archiving activity for the extension Open issues, problems etc. Summary

Documentation Delivery Status DWP_WEC_UM This has been delivered and ‘ingested’ in the sense it is available from the CAA under the DWP documentation section. DWP_ICD Changes from the DWP_ICD_V4 include: An update to DWP_COR procedure and CEF metadata description (due to move from Sussex to Sheffield) DWP_PIOR is now a CEF product, not plain ASCII Changes to the presentation of the CEF metadata Personnel updates This new version is still to be delivered to the CAA.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the data products provision Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Archiving Activity for the Extension 2009 2008 2007 2006 ICD ID End of mission + 3 months 2009-03-31 2008-10-31 2008-07-31 CP_DWP_UT_PIOR CP_DWP_LOG CP_DWP_COR Delivered CP_DWP_TCOR DWP_COR, DWP_LOG and DWP_UT_PIOR: For late 2008, and 2009 it is expected data can be delivered monthly, with two to three months delay. DWP_COR: Some minor changes to the production procedure will be needed for DWP_COR data 2006 onwards. This is because the PEACE RDM is not held online at Sheffield, but instead the much smaller intermediate files containing only the support data needed for the correlator.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the data products provision Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Management DWP_UT_PIOR Summary

Open Issues, Problems etc. Management At the start of the CAA contract, responsibility for the DWP_COR dataset and the DWP_TED software was with the University of Sussex. Following the loss of funding for this group at the end of March 2006, responsibility for these products was transferred to Sheffield. The University of Sheffield team also suffered a 50 percent reduction in national funding, from 3 people to 1.5, although using internal reserves we were able to continue to employ 2 people (Keith Yearby and Simon Walker) for Cluster operations and data processing, in addition to our ESA funded CAA developer (Ian Bates). The contract for Ian Bates expired at the end of April 2008. It is intended that Keith Yearby and Simon Walker will now share responsibility for CAA work, with the ESA CAA funding being used to make good the shortfall in national (STFC) funding for their positions. This assumes national funding will continue at current levels, which is uncertain after September 2008.

Open Issues, Problems etc. DWP_UT_PIOR The PIOR dataset was unique amongst the DWP products in that the PIOR applied to all four Cluster spacecraft simultaneously, and the ’standard’ C1..4_CH_ Observatory and Experiment header files were not able to carry this information. This problem was resolved by a recent addition to the original metadata specification allowing a MULTIPLE enumerated value to specify subsequently all four cluster spacecraft. CEFpass 1.3.2 does not recognise the MULTIPLE value but the dataset does parse with ‘dummy’ CEH files. We await feedback from the CAA that this dataset can be ingested.

Presentation Overview Introduction Status of the instrument data products delivery Status of the instrument calibration and cross-calibration activity Software development: status and delivery to the CAA Status of the data products provision Status of the documentation delivery Overview of the archiving activity for the extension Open issues, problems etc. Summary

Summary Current Status Planned Extension Activity DWP_COR, DWP_TCOR, DWP_LOG, DWP_UT_PIOR, DWP_TED and DWP_WEC_UM for Measurement Years 2001, 2002, 2003, 2004 and 2005 are all processed and in most cases delivered to the CAA and ingested. Planned Extension Activity DWP_COR, DWP_TCOR, DWP_LOG and DWP_UT_PIOR 2006, 2007, 2008 and 2009 are to be processed and delivered to the CAA as described in the Extension Data Delivery Schedule.

The End!