CIS Data Archival Status Alain Barthe, Iannis Dandouras and the CIS Team CLUSTER ACTIVE ARCHIVE: Final Review of the first phase ESTEC, 20 May 2008

Summary Introduction Status of data products delivery Instrument calibration & cross-calibration Software development Documentation delivery Archiving activity for the extension Open issues

Ion Composition and Distribution Function Analyser 1. Introduction : The CIS Experiment CODIF (CIS-1) Ion Composition and Distribution Function Analyser 3D ion distributions with mass-per-charge composition determination ~0 – 40 keV/e Energy Range HIA (CIS-2) Hot Ion Analyser 3D ion distributions with high angular resolution 5 eV/e – 32 keV/e Energy Range

1. Introduction: Management Structure The CIS CAA team is mainly composed of : Iannis Dandouras : CIS PI and CAA scientific correspondent Henri Rème : CIS Co-PI Alain Barthe : Software engineer (CAA contract) CAA software and data processing Emmanuel Penou : Software engineer Graphics visualization tool development (cl) Other contributions from CIS Co-Is and CESR members : Claude Aoustin : CIS Instrument Technical Manager Sylvain Brunato : Operations and webmaster of http://cluster.cesr.fr:8000 Lynn Kistler : CIS-1 (CODIF) calibration updates (UNH)

Introduction: Recommendations of the CAA OR-1 Review Board R27 : “The board expects that the CIS team should provide a concrete, realistic delivery plan, with delivery milestones, that can be used as an indication of progress...” The correction of a bug in the telemetry raw-data has shifted for some months the delivery schedule, but it has since been fixed and all high-resolution CIS data up to end of 2005 have been delivered (October 2007). CIS-CODIF ground calculated moments have also been delivered to CAA, in February/March 2008, for H+, He+ and O+, up to the end of 2005. R28 : “We recommend that the calibration of CIS is demonstrated to be as good as possible. For instance, it should be determined how often an under which circumstances there appears to be an inconsistency between Whisper, PEACE and HIA densities.” This has been discussed in the cross-calibration working group, and a section on the accuracy of the CIS calculated densities will be included in the CIS archival paper, that will be in the Tenerife Workshop Proceedings book.

1. Introduction: Recommendations of the CAA OR-1 Review Board R29 : “The CIS team should discuss with the CAA the best method to provide and run the partial moments software – will be run on the CAA or will users run it independently ?” The CIS-3D-MOM software for calculating partial (or total) moments has been delivered to the CAA, and is available online for CAA users. Users have to install locally the software and use CIS-3D CEF files (in Corrected_Particle_Count_Rate units), downloaded from CAA, as input for the software. However, as spacecraft attitude parameters are not part of the CIS-3D files, it is not possible to produce velocity moments in GSE frame, but only in the ISR2 reference frame (pseudo-GSE).

2. Data products delivery Processed Level 1 data : decommutated, decompressed and time-tagged telemetry data, basis from higher-level processing, which can be used interactively with the CIS graphical visualisation tool (cl) Calibration files : self-documented ASCII files, necessary to handle level-1 data Graphical products : PNG files (6 hours energy-time ion spectrograms) in two graphic resolutions Caveats : ASCII files including both general caveats and caveats for specific data intervals Onboard calculated moments (Level 2 data) : Public access CDF files, delivered monthly to the CSDS (CIS Primary Parameters) All these products, for year 2001 to 2005 included, have been delivered or are available for download at http://cluster.cesr.fr:8000 or national CSDS servers.

Example of CIS Ion Spectrogram and Data Caveats available at http://cluster.cesr.fr:8000

2. Data product delivery Calibrated high-resolution data (level 3 data) : These are processed 3D ion distribution functions, specifically generated for CAA, in various physical units, and in CEF format : CIS-CODIF_HS : High-sensitivity side, 4 ions species CIS-CODIF_LS : Low-sensitivity side, 4 ions species CIS-CODIF_RPA: Retarding Potential Analyser mode, 4 ions species CIS-HIA_HS_MAG : Magnetospheric mode, high-sensitivity side CIS-HIA_HS_SW : Solar-wind mode, high-sensitivity side CIS-HIA_LS_SW: Solar-wind mode, low-sensitivity side All these datasets (except for CODIF RPA) have been delivered to the CAA for years 2001 to 2005 included (October 2007 the latest delivery). The RPA data are being reprocessed, and will be delivered in June.

2. Data product delivery Ground calculated moments : Ground calculated moments CEF files, generated with our CIS-3D-MOM software in batch mode and using CIS-3D CEF files delivered to CAA as input. They include density, bulk velocity (GSE frame), pressure tensor, temperatures (parallel and perpendicular to the magnetic field and total temperature). CIS-CODIF_HS ground moments : H+, He+ and O+ moments, up to the end of 2005, have been delivered to CAA (February and March 2008). He++ moments are not supplied because these data are contaminated in the instrument by the "tail" of the H+ time-of-flight distribution (mass spillover), resulting in strongly overestimated He++ densities. CIS-CODIF_LS ground moments : This mode is selected only for SC4 H+, He+ and O+ moments have been generated and will be delivered by end of May. CIS-HIA ground moments are not supplied : same calibrations used as for onboard calculated moments, but with degraded time, energy and angular resolution (no added value) if necessary users can calculate them interactively with the software delivered.

Example of CIS-CODIF Moments on-ground calculated from CEF 3D ion distribution functions

2. Data product delivery Raw CODIF products (by nature uncalibrated): Three datasets remain to be delivered : 3D 64 m/q ranges with low angular resolution (4 x 3 solid angles ion distribution in high-mass resolution)) Monitor Rates (16 azimuths x 16 energies x 18 counters) Selected events (detailed time-of-flight, energy and position information for a sample of the detected ions) Software development to generate these products is well advanced. We still have to finalise the metadata part and we are planning to start delivering them by the end of July.

Instrument calibration & cross-calibration All calibration files up to end of 2005 delivered to the project. Work in progress, at UNH (USA), for updating CODIF calibrations for data beyond 2005: well advanced for data until April 2007, expected to be ready in June. Cooperation with the RAPID team, for CIS-RAPID cross-calibrations. CIS-WHISPER cross-calibrations. Participation in the activities of the cross-calibrations working group.

4. Software : Correction of HIA time-tagging A bug was discovered in the on-board time tagging of the CIS science data telemetry packets, which can occasionally produce a shift of the time tag by one spin period. After some spins the data time tagging is automatically re-synchronised. A software was thus elaborated, to identify the signature of this bug in the CIS telemetry packets, check time-tagging consistency, and “cleverly” reconstruct the correct time sequence and eliminate any potential data loss or degradation. This bug affects only HIA data, starting from May 2004, and required a massive reprocessing of the CIS data, starting from the L0 raw-data cd-roms, for years 2004 and beyond.

4. Software : Ground moments computation Partial ground moments computation : CIS-3D-MOM software : A library was first written in C to read/write CEF files efficiently. A software was developed and delivered, accepting all CODIF or HIA CEF datasets (in Corrected-Particle-Count-Rate), that computes the density, velocity, pressure and temperature, and produces CEF files (in the ISR2 reference frame: pseudo-GSE). We added the possibility to select energy range or solid angles to compute partial moments (command line interface). A supplementary software module then transforms the ISR2 results in GSE, reading an ISR2-to-GSE matrix file (operational for in-house batch processing). For interactive use, we have to iterate with CAA to find the best way to provide the auxiliary data needed to transform the results from ISR2 to GSE (interfacing issue for using the software on any computer).

Documentation delivery Most recent version V2.1 of the CAA-CIS ICD was submitted to the project on March 2007. Article on the CIS data archival under preparation for the Tenerife Workshop special book.

6. Activities for the extension Inclusion of a new CIS CODIF calibration update (expected for June), covering year 2006 and beyond. Inclusion of a new CIS HIA calibration update (expected also for June), covering year 2006 and beyond. Processing and data delivery of all datasets (CIS-3D and ground moments) for years 2006 - 2009, as soon as the corresponding calibrations become available. Reprocessing as necessary. Listening to users feedback for other demands.

Activities for the extension : delivery schedule Products Period Delivery CODIF_HS 2006 ~ 2007 October / November 2008 CODIF_LS CODIF_RPA 2001 ~ 2005 June 2008   CODIF_HS_MOMENTS 2006 ~2007 November / December 2008 CODIF_LS_MOMENTS (SC4 only) May 2008 CODIF_RATES 2001 ~ 2007 July / August 2008 CODIF_TOF_EVENTS CODIF_64M/Q HIA_HS_MAG August / September 2008 HIA_HS_HS_SW HIA_LS_SW

7. Problems, action items The resolution of the telemetry time-tagging bug has shifted the delivery of CIS-3D datasets, but all data up to end of 2005 have been already reprocessed and delivered between September and October 2007. A solution has still to be found for coordinate transformations of velocity components for CAA users of our CIS-3D-MOM software, from ISR2 to GSE.