Double Star Active Archive - STAFF-DWP Data errors and reprocessing Keith Yearby and Hugo Alleyne University of Sheffield Nicole Cornilleau-Wehrlin LPP.

Slides:



Advertisements
Similar presentations
During spacecraft-level environmental testing, after all instruments were integrated, the government assembled a data clerk team to support test data collection.
Advertisements

STATISTIC & INFORMATION THEORY (CSNB134) MODULE 12 ERROR DETECTION & CORRECTION.
 RAID stands for Redundant Array of Independent Disks  A system of arranging multiple disks for redundancy (or performance)  Term first coined in 1987.
EECE499 Computers and Nuclear Energy Electrical and Computer Eng Howard University Dr. Charles Kim Fall 2013 Webpage:
Rosetta Magnetic Field PDS Review B. J. Anderson.
REDUNDANT ARRAY OF INEXPENSIVE DISCS RAID. What is RAID ? RAID is an acronym for Redundant Array of Independent Drives (or Disks), also known as Redundant.
CAA 10th Cross Calibration Workshop, Paris, France, 2-4th Nov CLUSTER / STAFF Action items C. Burlaud, P. Robert, M. Maksimovic, N. Cornilleau-Wehrlin,
Two-Gyro Performance: Scheduling and Acquisitions Merle Reinhart.
IMPACT Report SSMO Receiving Review– January 23, 2007.
Instrument TrainingIDPU - 1 UCB, Dec 6, 2006 THEMIS INSTRUMENT TRAINING IDPU.
Error Detection and Correction.  Corrupted files  Attachments that won’t open  Files that won’t download  Videos that won’t play Errors occur when.
THEMIS-SCM THM – SCM – CDR – 08-April-2004 in Velizy
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
Dr. José M. Reyes Álamo 1.  Review: ◦ Statement Labels ◦ Unconditional Jumps ◦ Conditional Jumps.
TYPES OF STATISTICAL METHODS USED IN PSYCHOLOGY Statistics.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 20 Slide 1 Critical systems development 3.
GLAST Large Area Telescope Instrument Flight Software Flight Unit Design Review 16 September 2004 Software Watchdog Steve Mazzoni Stanford Linear Accelerator.
Objectives 2.1Scatterplots  Scatterplots  Explanatory and response variables  Interpreting scatterplots  Outliers Adapted from authors’ slides © 2012.
Data Integrity © Prof. Aiman Hanna Department of Computer Science Concordia University Montreal, Canada.
The Analysis of Binary Inspiral Signals in LIGO Data Jun-Qi Guo Sept.25, 2007 Department of Physics and Astronomy The University of Mississippi LIGO Scientific.
Andrei Gheata, Mihaela Gheata, Andreas Morsch ALICE offline week, 5-9 July 2010.
All rights reserved © Altec ExoMars 2018 Rover Operations Control Centre Science instruments data pipeline G. Martucci.
Improving Loss Resilience with Multi- Radio Diversity in Wireless Networks by Allen Miu, Hari Balakrishnan and C.E. Koksal Appeared in ACM MOBICOM 2005,
Searching for Gravitational Waves from Binary Inspirals with LIGO Duncan Brown University of Wisconsin-Milwaukee for the LIGO Scientific Collaboration.
ERROR DETECTING AND CORRECTING CODES -BY R.W. HAMMING PRESENTED BY- BALAKRISHNA DHARMANA.
ISUAL Data Formats & Science Data Processing S. Geller.
STAFF Report Patrick Robert, Rodrigue Piberne & STAFF team.
A4 1 Barto "Sequential Circuit Design for Space-borne and Critical Electronics" Dr. Rod L. Barto Spacecraft Digital Electronics Richard B. Katz NASA Goddard.
STAFF Report Prepared by R. PIBERNE. 2 1.Status of software development 2.Status of Experiment Calibration / Cross Calibration 3.Status of dataset delivery.
1 CAA 2009 Peer Review, Jesus College, Cambridge, UK, March CAA Peer Review: Selected Recommendations.
DAA PEACE Status A. Fazakerley, B. Mihaljčić, I. Rozum, A. Lahiff, G. Watson, D. Kataria UCL Department of Space and Climate Physics Mullard Space Science.
Data Editing Strategies Common edits Invalidation vs. contamination? What is a considered a spike? What not to edit! Automatic edits Reasons for edits.
Error Correction On Trunk Links. Why bother with error correction? Forward Error Correction (FEC) allows the receiver to detect and correct errors. In.
15th CAA Cross-calibration workshop CIS archiving activities report University College of London 2012, April
Cluster Active Archive Status of DWP Data Activities Simon Walker, Keith Yearby, Michael Balikhin Automatic Control and Systems Engineering, University.
Cluster Active Archive Status of DWP Activities Simon Walker, Keith Yearby, Hugo Alleyne ACSE, University of Sheffield.
CIS : Data Quality Indices Iannis Dandouras, Alain Barthe, Sylvain Brunato CLUSTER ACTIVE ARCHIVE April 2012.
Cluster Active Archive Status of DWP Data Activities Simon Walker, Keith Yearby, Michael Balikhin Automatic Control and Systems Engineering, University.
SENG521 (Fall SENG 521 Software Reliability & Testing Preparing for Test (Part 6a) Department of Electrical & Computer Engineering,
Double Star Active Archive - DWP/STAFF 1 Double Star Active Archive STAFF/DWP Keith Yearby and Hugo Alleyne University of Sheffield Nicole Cornilleau-Wehrlin.
Laboratoire de Physique des Plasmas 20 TH CAA CROSS-CAL M EETING CLUSTER-STAFF REPORT O CTOBER 2014 G ÖTTINGEN The STAFF Team Laboratoire de Physique.
DAA Status/Progress B. Mihaljčić, A. Fazakerley, G. Watson, C. Anekallu UCL Department of Space and Climate Physics Mullard Space Science Laboratory 21.
WEC meeting TED status and WEC timing.
C osmic R Ay T elescope for the E ffects of R adiation 27 June 2005 Digital Engineering Digital Sub-System Bob Goeke.
Cluster Active Archive Status of DWP Data Activities Simon Walker, Keith Yearby, Michael Balikhin Automatic Control and Systems Engineering, University.
DAA Status/Progress B. Mihaljčić, A. Fazakerley, N. Doss, G. Watson UCL Department of Space and Climate Physics Mullard Space Science Laboratory 18 th.
PLASTIC Post Launch Data Flow Lynn Kistler & Lorna Ellis May 2006.
21st CAA Cross Calibration Workshop Leiden, March 2015 WHISPER report WHISPER team LPC2E/CNRS.
14 th CAA Cross-calibration Workshop CIS archiving activities York October 5-7, 2011.
Double Star Active Archive - STAFF- DWP 1 K.H. Yearby, S.N. Walker, M. Balikhin University of Sheffield.
Operational Flight Software
SOC Physical Resource Allocations
Development of KAGRA Burst Pipeline
Status Report of EDI on the CAA
Cluster Active Archive – Wideband data BM2 mode
NAC flat fielding and intensity calibration
An Automated calibration method from shower data
CAA-OR (End of Phase 1) CAA DWP Operations Review
H. Rème, I.Dandouras and A. Barthe IRAP, Toulouse, France
DATA COMMUNICATION AND NETWORKINGS
Solar Wind Core Electrons
Annual Report of the DWP Experiment 9th CAA Operations Review
Error Correcting Code.
Status Report of EDI on the CAA
Operational Description
Digital Sub-System Bob Goeke.
Telecommunications Engineering Topic 2: Modulation and FDMA
Sheffield team interests
9.4 Enhancing the SNR of Digitized Signals
Types of Errors Data transmission suffers unpredictable changes because of interference The interference can change the shape of the signal Single-bit.
Presentation transcript:

Double Star Active Archive - STAFF-DWP Data errors and reprocessing Keith Yearby and Hugo Alleyne University of Sheffield Nicole Cornilleau-Wehrlin LPP

Boom deployment failure + Resets The STAFF antenna boom failed to deploy which means interference is very high, particularly below 100Hz. The worst affected datasets (10Hz waveform data and spectral matrix data) will not be archived in calibrated form, but could be accessed from the raw data. Power spectral density data is processed to synthesise a single axis measurement aligned for minimum interference. This is then integrated over 4 seconds (about one spin) to provide a reasonably isotropic measurement. Particle correlator data is not affected by the interference. Single event upsets (SEU) occur at a rate between one every ~500 hours above L=2.5, increasing to about one per hour at L=1.5. These are corrected by the watchdog, and normal operation restored in less than one second. Very little impact on data quality.

Spacecraft telemetry errors The Double Star raw telemetry data is released to instrument teams even when it contains uncorrected transmission errors. There are no flags or check bits to indicate that uncorrected errors may be present. –It would be extremely useful if an ‘uncorrected errors’ flag could be provided in the raw data header during reprocessing. This should be available as an output from the Reed Solomon decoder. The best we can do at present is to check that the data is correctly formatted and within the bounds of what the instrument can physically measure. Data containing transmission errors can occasionally fall through these checks. Errors will typically appear in the PSD or ACF data as individual points with much higher values than those around them. These errors are mostly rare and less of a problem than the high level of interference due to the failure to deploy the STAFF antenna boom.

Housekeeping data filter A simple software tool has been developed to remove HK frames containing transmission errors from the data stream. It is controlled by a parameter database so could be used by other instruments. The database defines which parameters have values that are static or change in a predictable way during normal operation. Provided such parameters in the current frame match either the frame before or the frame after, then the current frame is considered valid. Limitations: –More than one unexpected, but real, change in a parameter could cause valid frames to be rejected. –Cannot detect errors in parameters that are not static or predictable. However, as errors tend to occur in bursts, it is quite effective in practice. Example - counter parameter - should increment by 1 each frame: –123, 124, 125, 126, 127, 255, 129, 130, 131, 0, 1, 2, 3, 4, 5, 6 –Frame containing 255 is rejected because it matches neither value before or after. –Transition from 131 to 0 is OK because each matches a value on one side.

Science data errors It is difficult to check the science data itself for transmission errors as almost all binary values are valid. However, each science packet has a 48 bit header which is checked for self consistency (eg. packet size corresponds to commanded mode). This will catch any error burst which includes the header. It would also be possible to reject a packet if the header of the immediately following packet is bad (error burst likely to include end of current packet), but this is not currently implemented. Raw data has one second time resolution - identified bad data is not included in 4 second averaged data for archive.

(Re)processing With the exception of the combined plots which already exist, none of the data has yet been processed to create DAA products. All other STAFF-DWP datasets will be processed for the first time when re-processed raw data is available. Software tools to do this are in place, with the exception of the caveat datasets. It is not planned to systematically reprocess the combined plots, though individual days might be reprocessed if substantially improved raw data became available. Problem - no space left on RAID storage at Sheffield used for Cluster / Double Star raw data. Cannot currently import any reprocessed raw data. In short term can move older Cluster data to alternative storage.