ROCC Operations and The PSA RSP Archive Concept Review 28 September 2017 Tanya Lim Archive Scientist

Data Flow

Data Selection in the ROCC Archive “The Data Archive and Retrieval Capability shall provide cataloguing of all ROCS data in such a way to guarantee an easy data browsing based on different parameters, like: Time / Time Range (UTC/LTST/LMST/RET) Sols Product generation time Product status (incomplete, preliminary, valid, final) Experiment Cycle / Vertical Survey / subsystem Geographical location on Mars surface Activity Data product Other metadata as reported in the ROCC to PPL ICDs.“

Delivery ROCC to ESAC From ROCC-ESAC ICD.. Delivery process description on ROCC side. It will be mainly composed by 5 steps: PPL archive preparers will review their instrument data products SDMT members define the local targets A-SDP post-processing pipeline is executed Geometry information insertion Target(s) references insertion SDMT members create and validate the delivery package Delivery package submission

PDS Data Levels Raw – Re-packed telemetry Partially Processed – Partly calibrated Calibrated – All instrument artifacts removed Derived – Scientific Results Minimum acceptable level for final archive is Calibrated

What will be Generated at ROCC? Raw: Micro-Omega (Raw Image Cube) WISDOM (Raw Sounding) Partially Processed: Adron (Data frames) ISEM (Partially Processed Spectrum) Calibrated: Clupi (Calibrated image, but also as secondary products) Ma_Miss (Acquired science spectra represented as physical quantities) MOMA (Contents of bins/sec) RLS (Calibrated Spectrum with Raman Shift vector and corrected in intensity) Derived: PanCam (not all Products) Colours reflect extent to which PI definitions agree with PDS definitions

So…. Only products selected by PI’s in ROCC archive will be selected for archival at ESAC Most products are generated outside of ROCC and not on tactical timescales These have to be delivered to ROCC, ingested into the ROCC system then manually selected for archiving at ESAC

Challenges for ESAC of this Approach Lack of Data Consistency Potentially inconsistent data processing Potential gaps Potential lack of continuity between data processing levels Inconsistent versioning between products in archive Duplicate products Not all data delivered… a lot of manual chasing… Keeping track of what we have and don’t have Version Control Overall Knowing when to conduct a data review Keeping track of proprietary status with an inconsistent data set Explaining these inconsistencies to the community

Mitigation Allow ESAC to view the ROCC DAR to see what products are present Ideally the DAR could indicate which have/have not been delivered Also allowing access to metadata would help ensure continuity ahead of delivery Track pipeline changes at ROCC Persuade PIs to allow deliveries to be as automatic as possible Ensure that the DAR can support this Keep up to date with secondary product generation, PI pipelines and plans Ensure delivery procedures from PIs to ROCC are well understood and tested

Rover Data Arrival At ESAC Bundle increments will arrive to a staging area These will be inspected by the AS and approved for ingestion These will then be ingested into the archive and re-validated on ingestion All data is ingested as proprietary Technically, within PSA, data release can be on a six month rolling basis or on request from PI via AS or PS An operational understanding for data release is not yet in place for EMRSP

These are normal work Open Issues Delivery schedule – current working assumption is monthly ESAC handling of deliveries Instrument Data Dictionaries Proprietary Handling – from an organisational standpoint What happens post-mission? These are normal work

Questions?