1. CIS Data Archival Status
Alain Barthe, Iannis Dandouras and the CIS Team
CLUSTER ACTIVE ARCHIVE:
Final Review of the first phase
ESTEC, 20 May 2008

2. Summary Introduction Status of data products delivery
Instrument calibration & cross-calibration
Software development
Documentation delivery
Archiving activity for the extension
Open issues

3. Ion Composition and Distribution Function Analyser
1. Introduction : The CIS Experiment
CODIF (CIS-1)
Ion Composition and Distribution Function Analyser
3D ion distributions with mass-per-charge composition determination
~0 – 40 keV/e Energy Range
HIA (CIS-2)
Hot Ion Analyser
3D ion distributions with high angular resolution
5 eV/e – 32 keV/e Energy Range

4. 1. Introduction: Management Structure
The CIS CAA team is mainly composed of :
Iannis Dandouras : CIS PI and CAA scientific correspondent
Henri Rème : CIS Co-PI
Alain Barthe : Software engineer (CAA contract) CAA software and data processing
Emmanuel Penou : Software engineer Graphics visualization tool development (cl)
Other contributions from CIS Co-Is and CESR members :
Claude Aoustin : CIS Instrument Technical Manager
Sylvain Brunato : Operations and webmaster of
Lynn Kistler : CIS-1 (CODIF) calibration updates (UNH)

5. Introduction: Recommendations of the CAA OR-1 Review Board
R27 : “The board expects that the CIS team should provide a concrete, realistic delivery plan, with delivery milestones, that can be used as an indication of progress...”
The correction of a bug in the telemetry raw-data has shifted for some months the delivery schedule, but it has since been fixed and all high-resolution CIS data up to end of 2005 have been delivered (October 2007).
CIS-CODIF ground calculated moments have also been delivered to CAA, in February/March 2008, for H+, He+ and O+, up to the end of 2005.
R28 : “We recommend that the calibration of CIS is demonstrated to be as good as possible. For instance, it should be determined how often an under which circumstances there appears to be an inconsistency between Whisper, PEACE and HIA densities.”
This has been discussed in the cross-calibration working group, and a section on the accuracy of the CIS calculated densities will be included in the CIS archival paper, that will be in the Tenerife Workshop Proceedings book.

6. 1. Introduction: Recommendations of the CAA OR-1 Review Board
R29 : “The CIS team should discuss with the CAA the best method to provide and run the partial moments software – will be run on the CAA or will users run it independently ?”
The CIS-3D-MOM software for calculating partial (or total) moments has been delivered to the CAA, and is available online for CAA users.
Users have to install locally the software and use CIS-3D CEF files (in Corrected_Particle_Count_Rate units), downloaded from CAA, as input for the software.
However, as spacecraft attitude parameters are not part of the CIS-3D files, it is not possible to produce velocity moments in GSE frame, but only in the ISR2 reference frame (pseudo-GSE).

7. 2. Data products delivery
Processed Level 1 data : decommutated, decompressed and time-tagged telemetry data, basis from higher-level processing, which can be used interactively with the CIS graphical visualisation tool (cl)
Calibration files : self-documented ASCII files, necessary to handle level-1 data
Graphical products : PNG files (6 hours energy-time ion spectrograms) in two graphic resolutions
Caveats : ASCII files including both general caveats and caveats for specific data intervals
Onboard calculated moments (Level 2 data) : Public access CDF files, delivered monthly to the CSDS (CIS Primary Parameters)
All these products, for year 2001 to 2005 included, have been delivered or are available for download at or national CSDS servers.

8. Example of CIS Ion Spectrogram and Data Caveats available at http://cluster.cesr.fr:8000

9. 2. Data product delivery
Calibrated high-resolution data (level 3 data) : These are processed 3D ion distribution functions, specifically generated for CAA, in various physical units, and in CEF format :
CIS-CODIF_HS : High-sensitivity side, 4 ions species
CIS-CODIF_LS : Low-sensitivity side, 4 ions species
CIS-CODIF_RPA: Retarding Potential Analyser mode, 4 ions species
CIS-HIA_HS_MAG : Magnetospheric mode, high-sensitivity side
CIS-HIA_HS_SW : Solar-wind mode, high-sensitivity side
CIS-HIA_LS_SW: Solar-wind mode, low-sensitivity side
All these datasets (except for CODIF RPA) have been delivered to the CAA for years 2001 to 2005 included (October 2007 the latest delivery).
The RPA data are being reprocessed, and will be delivered in June.

10. 2. Data product delivery
Ground calculated moments : Ground calculated moments CEF files, generated with our CIS-3D-MOM software in batch mode and using CIS-3D CEF files delivered to CAA as input.
They include density, bulk velocity (GSE frame), pressure tensor, temperatures (parallel and perpendicular to the magnetic field and total temperature).
CIS-CODIF_HS ground moments :
H+, He+ and O+ moments, up to the end of 2005, have been delivered to CAA (February and March 2008).
He++ moments are not supplied because these data are contaminated in the instrument by the "tail" of the H+ time-of-flight distribution (mass spillover), resulting in strongly overestimated He++ densities.
CIS-CODIF_LS ground moments :
This mode is selected only for SC4
H+, He+ and O+ moments have been generated and will be delivered by end of May.
CIS-HIA ground moments are not supplied :
same calibrations used as for onboard calculated moments, but with degraded time, energy and angular resolution (no added value)
if necessary users can calculate them interactively with the software delivered.

11. Example of CIS-CODIF Moments on-ground calculated from CEF 3D ion distribution functions

12. 2. Data product delivery
Raw CODIF products (by nature uncalibrated): Three datasets remain to be delivered :
3D 64 m/q ranges with low angular resolution (4 x 3 solid angles ion distribution in high-mass resolution))
Monitor Rates (16 azimuths x 16 energies x 18 counters)
Selected events (detailed time-of-flight, energy and position information for a sample of the detected ions)
Software development to generate these products is well advanced.
We still have to finalise the metadata part and we are planning to start delivering them by the end of July.

13. Instrument calibration & cross-calibration
All calibration files up to end of 2005 delivered to the project.
Work in progress, at UNH (USA), for updating CODIF calibrations for data beyond 2005: well advanced for data until April 2007, expected to be ready in June.
Cooperation with the RAPID team, for CIS-RAPID cross-calibrations.
CIS-WHISPER cross-calibrations.
Participation in the activities of the cross-calibrations working group.

14. 4. Software : Correction of HIA time-tagging
A bug was discovered in the on-board time tagging of the CIS science data telemetry packets, which can occasionally produce a shift of the time tag by one spin period. After some spins the data time tagging is automatically re-synchronised.
A software was thus elaborated, to identify the signature of this bug in the CIS telemetry packets, check time-tagging consistency, and “cleverly” reconstruct the correct time sequence and eliminate any potential data loss or degradation.
This bug affects only HIA data, starting from May 2004, and required a massive reprocessing of the CIS data, starting from the L0 raw-data cd-roms, for years 2004 and beyond.

15. 4. Software : Ground moments computation
Partial ground moments computation : CIS-3D-MOM software :
A library was first written in C to read/write CEF files efficiently.
A software was developed and delivered, accepting all CODIF or HIA CEF datasets (in Corrected-Particle-Count-Rate), that computes the density, velocity, pressure and temperature, and produces CEF files (in the ISR2 reference frame: pseudo-GSE).
We added the possibility to select energy range or solid angles to compute partial moments (command line interface).
A supplementary software module then transforms the ISR2 results in GSE, reading an ISR2-to-GSE matrix file (operational for in-house batch processing).
For interactive use, we have to iterate with CAA to find the best way to provide the auxiliary data needed to transform the results from ISR2 to GSE (interfacing issue for using the software on any computer).

16. Documentation delivery
Most recent version V2.1 of the CAA-CIS ICD was submitted to the project on March 2007.
Article on the CIS data archival under preparation for the Tenerife Workshop special book.

17. 6. Activities for the extension
Inclusion of a new CIS CODIF calibration update (expected for June), covering year 2006 and beyond.
Inclusion of a new CIS HIA calibration update (expected also for June), covering year 2006 and beyond.
Processing and data delivery of all datasets (CIS-3D and ground moments) for years , as soon as the corresponding calibrations become available.
Reprocessing as necessary.
Listening to users feedback for other demands.

18. Activities for the extension : delivery schedule
Products
Period
Delivery
CODIF_HS
2006 ~ 2007
October / November 2008
CODIF_LS
CODIF_RPA
2001 ~ 2005
June 2008
CODIF_HS_MOMENTS
2006 ~2007
November / December 2008
CODIF_LS_MOMENTS
(SC4 only)
May 2008
CODIF_RATES
2001 ~ 2007
July / August 2008
CODIF_TOF_EVENTS
CODIF_64M/Q
HIA_HS_MAG
August / September 2008
HIA_HS_HS_SW
HIA_LS_SW

19. 7. Problems, action items
The resolution of the telemetry time-tagging bug has shifted the delivery of CIS-3D datasets, but all data up to end of 2005 have been already reprocessed and delivered between September and October 2007.
A solution has still to be found for coordinate transformations of velocity components for CAA users of our CIS-3D-MOM software, from ISR2 to GSE.