1 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Production of ESOC datasets for the Cluster Active Archive Harri Laakso ESA, Science and Robotic Exploration Directorate, Code SRE-OOC Noordwijk, The Netherlands 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 20131

2 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archives  ESOC datasets contain files in the following format  Documents in pdf  Digital data in xls or txt  ESOC has grouped datasets in 35 categories  ESOC has provided files up to the end of 2006 but will soon process the other years  6 categories where each contains different kinds of documents o These will be placed in the normal CAA document area  29 categories where each contains a number of similar files (based on a template converted into pdf, xls or txt)  each group can be handled as a single dataset, so they end up to 29 datasets (per spacecraft)  Some datasets contain only a single file that is updated annually  few of the datasets are obsolete now  Each category requires a separate conversion pipeline

3 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Handling of ESOC Datasets  Creation of metadata  Define file timespan: validity interval of the file, i.e. start and end time of the file o Most datasets are easy o anomalies are among the most difficult (i.e., when it started and ended) ESOC will take care of this  Read (part of) the file content: to create the file data (description etc) that could be used later in searching/browsing the file content in the GUI  Each template area/keyword could be a variable, so one file could end up to one record, i.e. one ISO time range and a number of variables that are mostly text strings  Conversion of digital data into CEF  txt: should normally be quite straightforward o Some of the lines may not be complete, or the content of lines may vary  xls: handling of these files may be more challenging if they contain several sheets o Some datasets may contain both digital data and plots whose handling is more complex: o convert digital data into CEF o Graphics: conversion of graphics is NOT needed unless they contain some important/useful annotation CAA graphics can be updated to visualize some of the ESOC parameters Some plots can be generated by the CAA in order to view them with QL tool

4 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Reports AR_Spacecraft Anomaly Reports CLU_AR-011pdf CLU_AR-012pdf … ARTS_Ground Segment Anomalies and Observation Reports CLU-1019pdf CLU-1020pdf … Monthly Reports CLUSTER_Monthly_MOR_April_2001pdf CLUSTER_Monthly_MOR_August_2001pdf … Weekly Mission Operations Reports Cluster_MOR_no022_(week )pdf Cluster_MOR_no023_(week )pdf … ESOC Archive: L1 (Reports and Technical Notes)

5 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Spacecraft System Technical Notes and Reports AGC Fluctuation Reports AGC 2001 and 2002pdf AGC 2003 and 2004pdf Cluster AGC WG final report_2003pdf Cluster radio link disturbancepdf NAVITEC_2004_Noordwijkpdf ESOC Archive: L1 (Reports and Technical Notes) AGC was not observed in (some occurred in 2005). It has reoccurred in recent years. At least a table with the occurrences shall be kept up to date. Related publications may be kept in the folder or moved to public download area (TBD).

6 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Spacecraft System Technical Notes and Reports Eclipse and Battery Reports Batteries in orbit performance_paperpdf Battery Conditioning 2nd mission extension_July 2004pdf Long Eclipse Season 2001pdf Long Eclipse 2002pdf Long Eclipse 2003pdf Moon and Short Eclipses 2001pdf Short Eclipse 2002pdf Short Eclipse 2003pdf Short Eclipse 2004pdf Moon eclipse June 2004pdf Power Budget 2002pdf ESOC Archive: L1 (Reports and Technical Notes) Payload constraints documents [per eclipse season] in pdf, together with excel sheet with the same version number [new products] Eclipse reports available for most years [except for years when eclipses lasted for most part of the year]

7 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Spacecraft System Technical Notes and Reports Manoeuvre Reports 10N Reaction Control Thruster Leakage_2003pdf Constellation_change_Season2003_ManReppdf Constellation_change_Season2004_ManReppdf TrinksSensorsResults_ 01 June 2004pdf ESOC Archive: L1 (Reports and Technical Notes) There are also digital data (excel) available that contain detailed data of the maneuvers

8 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Spacecraft System Technical Notes and Reports Miscellaneus Technical Notes with Relevance to Science Data 2nd mission extension performance evaluation_2004pdf Anite_Time Stamping of Science Data_2003pdf Anomaly Report_Summary 2001pdf Aspoc SC1 failure report_2000pdf CTU Switch-Over Status on Spacecraft 3_Rev 3pdf DWP Voltage monitoring_2003pdf SSR Bit Error count and Proton Flux from launch till Nov 2003pdf Time Correlation Chains Test Report_2004pdf Time Correlation_note 1992pdf Science Telemetry Reallocation Reports Re-Allocation of CIS Nominal Mode Science TM to RAPID on SC2pdf Re-allocation of science TLM on Cluster 2 and 4pdf Solar Array Performance Evaluation and Updates Degradation of solar array_Apr 2002pdf Degradation of the solar arrays_Nov 2000pdf Solar Array Performance Evaluation_2003pdf ESOC Archive: L1 (Reports and Technical Notes)

9 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Single file with several sheets per spacecraft some information is repeated in different sheets; the last working sheets will be taken out There exist a single file for the whole mission It is not clear if these data need to be converted or even be made available They are produced only for certain time intervals Battery conditioning plan shall be kept and shall be updated: but only the battery capacity is needed in digital format for each S/C with the plots (as they contain notes) Battery Conditioning Plans BTR_Conditioning_Plan_ xls … ESOC Archive: L2 (Spacecraft System Data) Battery Conditioning History Data Conditioning history from Oct 2000 to Apr 2005xls

10 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Should we convert the energetic particle fluxes from other spacecraft? They could be made as reference datasets there is a single file covering all years and all s/c: take the summary file for entire mission. It shall be explained (in the ICD) how the data are sampled and which correction was done: e.g. different sampling time (from AOS to AOS) and normalization to one word group. there is a single file covering all years and all s/c [only digital data] BIT ERROR COUNTS table and plot ssr_biterror_pflux_2001xls … ESOC Archive: L2 (Spacecraft System Data) Evaluation of Solar Array Performance Data Solar Array evolution and prediction from aug 2000 to mar 2005xls

11 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 IPD_EPD_MEP Temperature Data 2001_epd_ipd_mepxls … ESOC Archive: L2 (Spacecraft System Data) Spacecraft Event Logs _Raindance_CTU reboot_CTURTU switch over_SSR anomalies Spacecraft_Event_Log_2001xls … one file/year these are annual files (but there may possibly exist a single file for all years?)

12 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 SPO_Special Payload and Platform Operations 0098_010104_CLU_SPECIAL_OPS_REQUESTdoc/pdf … ESOC Archive: L2 (Spacecraft System Data) this contain poor information (e.g. no execution time) during first years; this was changed in later years. There is an index file that refers to the request “Special Operations Request”: this is a dataset of its own. It shall be used by the CAA to extract the start/stop interval of the SPOs and shall point to the SPOs, which contain the actual information what manual operation was performed.

13 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archive: L2 (Spacecraft System Data) TDA Mode Changes TDAMODESFROM2001FORSC1txt … This is useful input for the mode dataset which is currently created from the AUX dataset (1-min resolution) the ESOC file has a better time resolution than the one generated by the Data Centres. o Explain in the ICD how to read the table e.g. which CAB will be active in TDA 5,6. Make one file per year. o Spacecraft user manual contains a table that explains the TDA mode changes; one could include s/c user manual on the CAA website (if it is not confidential)

14 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Thermal Eclipse data_Permanent and Switchable Bus Current Data 2001_long eclipse_thermalxls … ESOC Archive: L2 (Spacecraft System Data) Thrusters Summary_Manoeuvre history table Manoeuvre_Summary_from 2000 to Mar 2005xls the files contain a lot of s/c parameters during the eclipse this document was released only until 2005; data are given per eclipse season there is a web-based document the results are given in the reports (see L1) this is a single file updated annually there is another maneuver summary file

15 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 Mission Planning Data O_0096_0098_01037Atxt O_0099_0101_01038Atxt … ESOC Archive: L3 (Mission Planning Data) These include command sequences and are available until year 2011 after which a new format was adopted these are planned operations and basically include: OBRQ + antenna selection. o JSOC OBRQ contain the same information (should CAA archive them?) o It is not needed if the more important TC history (full mode) could be provided. This would reflect the actual operations performed (Planned by JSOC and including the manual operations by SPOs).  it is not clear what effort it would be to get the TC history extracted from SPEVAL in execution time order for the entire mission Command history files contain the actually executed commands with the used parameters. This is more useful than these files. See also note: it does not include parametershttp://cluster.esoc.esa.int/cluster_tools/tc/tc.php?sc_id=3

16 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archive: L4 (Flight Dynamics Data) Covariance Matrix File COVM_060117OAS1_2001_0001txt COVM_060117OAS2_2001_0001txt … Eclipse File ECLI_060117OAS1_2001_0001txt ECLI_060117OAS2_2001_0001txt … LTEF_060117OAS1_2001_0001txt LTEF_060117OAS2_2001_0001txt … PEACE has requested the profile of eclipse; this is provided by FD and format is good (time resolution is 1-min) One can have a single file per spacecraft for all eclipses This may not be needed because STEF is already on CAA

17 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 RUN ON 2008/10/14 AT 10:04:38 Z INPUT: EXPECTED SATELLITE ID = 1 INPUT: START TIME OF ECLIPSE SEARCH = 2008/12/28 00:00:00 INPUT: NUMBER OF HOURS TO SEARCH = 24 DETAILED PRINT OF ECLIPSE OF SUN BY MOON & EARTH ================================================ DISTANCE FROM (KM, NEGATIVE = INSIDE) VISIBLE TIME MOON PENUMBRAUMBRA EARTH PENUMBRA UMBRA FRACT. ================================================================================= 2008/12/28 0: 0: 0 START OF ECLIPSE SEARCH NO ECLIPSES 2008/12/28 15:11: /12/28 15:12: /12/28 15:13: /12/28 15:14: /12/28 15:15: /12/28 15:16: /12/28 15:45: /12/28 15:46: /12/28 15:47: /12/28 15:48: New dataset: Eclipse Profiles

18 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archive: L4 (Flight Dynamics Data) Extract of Flight Dynamics Reports Orbit File STOF_060117OAS1_2001_0001txt STOF_060117OAS2_2001_0001txt … SATT_Spacecraft Attitude and Spin Rates file SATT_060331OAS1_2001_0001txt SATT_060331OAS2_2001_0001txt … They are not needed because STOF and SATT already exist on CAA

19 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 data gaps/Recalls are not needed; This is s/c database and will be given only after the end of the mission ESOC Archive: L5 (Ground Segment Data) Data recall log and Data gap analysis GAPS 2001xls … DataBase Changes

20 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 These are the actual contact times, not the planned ones. So they are more important than the OMS 11 (planned passes), see below For ESOC, these are weekly files. Could the GSU log files be converted into less files, e.g. one file for each spacecraft with all passes, or all passes over one ground station during the entire mission? These are the planned schedules, and do not need to be archived ESOC Archive: L5 (Ground Segment Data) Ground Station usage and event logs AOS_LOS logs GSU_2002_001_001_006xls Official ESTRACK schedule_OMS11 Scheduling_Week_VL1_2001_08xls …

21 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archive: L5 (Ground Segment Data) Science Data Return SCIENCE_RETURN_0852txt … Science return ratio for frames and formats per PP (txt ) - not needed (The excel file (below) with the science data summary would be enough) If in the Science data return graph the annotation about the cause of the missing data (e.g. AR-xyz) would be moved into a comment column of the digital data sheet that would be enough. Single file for the whole mission Sheets “DATA_SUMMARY_year” contain average of four spacecraft. “All history data details” gives summary for each spacecraft plus spacecraft average “orbit_pp gives details for each planning period – is this needed Science data return summary Science Return from Oct 2001 to Apr 2005xls

22 18 th CAA Cross-Calibration Workshop Cosener’s House, Abingdon, UK Oct 2013 ESOC Archive: L5 (Ground Segment Data) Time Correlation data Coefficient used TIME_CORR_from 2000 to 2004xls Drift of the on board clock TIME CORR from to xls TIME CORR from to xls … Internal delays used Delays Apr 2005xls These are source files to DWP to produce TCOR (plus DSN from Iowa) DWP: which of these datasets should be saved on the CAA?