1. ROCC Operations and The PSA RSP Archive Concept Review 28 September 2017
Tanya Lim
Archive Scientist

2. Data Flow

3. Data Selection in the ROCC Archive
“The Data Archive and Retrieval Capability shall provide cataloguing of all ROCS data in such a way to guarantee an easy data browsing based on different parameters, like:
Time / Time Range (UTC/LTST/LMST/RET)
Sols
Product generation time
Product status (incomplete, preliminary, valid, final)
Experiment Cycle / Vertical Survey / subsystem
Geographical location on Mars surface
Activity
Data product
Other metadata as reported in the ROCC to PPL ICDs.“

4. Delivery ROCC to ESAC From ROCC-ESAC ICD..
Delivery process description on ROCC side. It will be mainly composed by 5 steps:
PPL archive preparers will review their instrument data products
SDMT members define the local targets
A-SDP post-processing pipeline is executed
Geometry information insertion
Target(s) references insertion
SDMT members create and validate the delivery package
Delivery package submission

5. PDS Data Levels Raw – Re-packed telemetry
Partially Processed – Partly calibrated
Calibrated – All instrument artifacts removed
Derived – Scientific Results
Minimum acceptable level for final archive is Calibrated

6. What will be Generated at ROCC?
Raw:
Micro-Omega (Raw Image Cube)
WISDOM (Raw Sounding)
Partially Processed:
Adron (Data frames)
ISEM (Partially Processed Spectrum)
Calibrated:
Clupi (Calibrated image, but also as secondary products)
Ma_Miss (Acquired science spectra represented as physical quantities)
MOMA (Contents of bins/sec)
RLS (Calibrated Spectrum with Raman Shift vector and corrected in intensity)
Derived:
PanCam (not all Products)
Colours reflect extent to which PI definitions agree with PDS definitions

7. So….
Only products selected by PI’s in ROCC archive will be selected for archival at ESAC
Most products are generated outside of ROCC and not on tactical timescales
These have to be delivered to ROCC, ingested into the ROCC system then manually selected for archiving at ESAC

8. Challenges for ESAC of this Approach
Lack of Data Consistency
Potentially inconsistent data processing
Potential gaps
Potential lack of continuity between data processing levels
Inconsistent versioning between products in archive
Duplicate products
Not all data delivered… a lot of manual chasing…
Keeping track of what we have and don’t have
Version Control Overall
Knowing when to conduct a data review
Keeping track of proprietary status with an inconsistent data set
Explaining these inconsistencies to the community

9. Mitigation
Allow ESAC to view the ROCC DAR to see what products are present
Ideally the DAR could indicate which have/have not been delivered
Also allowing access to metadata would help ensure continuity ahead of delivery
Track pipeline changes at ROCC
Persuade PIs to allow deliveries to be as automatic as possible
Ensure that the DAR can support this
Keep up to date with secondary product generation, PI pipelines and plans
Ensure delivery procedures from PIs to ROCC are well understood and tested

10. Rover Data Arrival At ESAC
Bundle increments will arrive to a staging area
These will be inspected by the AS and approved for ingestion
These will then be ingested into the archive and re-validated on ingestion
All data is ingested as proprietary
Technically, within PSA, data release can be on a six month rolling basis or on request from PI via AS or PS
An operational understanding for data release is not yet in place for EMRSP

11. These are normal work Open Issues
Delivery schedule – current working assumption is monthly
ESAC handling of deliveries
Instrument Data Dictionaries
Proprietary Handling – from an organisational standpoint
What happens post-mission?
These are normal work

12. Questions?