CCSDS Meeting data Archive Ingest - June 2006 CNES 1 CCSDS - MOIMS Area Data Archive Ingest WG CNES Report Rome meeting – June 2006 Claude Huc

CCSDS Meeting data Archive Ingest - June 2006 CNES 2 Contents  CNES standards, evolution of data engineering domain (RNC)  Data exchange standard for archiving, defined in France  Current work on the ‘Producer Archive Interface Specification’

CCSDS Meeting data Archive Ingest - June 2006 CNES 3 CNES standards (RNC) current content of the data engineering domain At the highest level v An introductory document justifying its existence v The OAIS reference model On a technical level v A document defining Criteria for evaluating data formats v The DEDSL abstract standard v The DEDSL XML/DTD syntax v The EAST specification v The PAIMAS New document recently introduced: v Long term archiving of data: Applicable rules and recommendations for projects producing data

CCSDS Meeting data Archive Ingest - June 2006 CNES 4 CNES standards (RNC) applicable rules and recommendations for projects Contents  Description of participants and their roles  3 Basic rules The basic information to be archived should be precisely defined with close cooperation between the Project team and the Archiving department. The objective is not to simply keep files but to ensure that the information will always be available in a useable form. The user communities for this data should be clearly identified Archiving of information from Space experiments involves at least : v Archiving the core data for the experiment: u main data, u auxiliary data (orbit, attitude, etc.), v Archiving metadata associated with the data, v Archiving the documentation.

CCSDS Meeting data Archive Ingest - June 2006 CNES 5 CNES standards (RNC) applicable rules and recommendations for projects contents  Rules related with the syntax Data structure v Files with records v XML files v Data bases Coding v Interger and real numbers v Character strings in data files Image files v Raster or vector format Documentary data v Structured files (XML, HTML, XHTML, Open Document Format/ISO 26300…) v PDF/A files

CCSDS Meeting data Archive Ingest - June 2006 CNES 6 CNES standards (RNC) applicable rules and recommendations for projects  Rules for content Synthesis of main content v Main data v Complementary data for experimental data v Metadata v Documents and publications v Graphic data Main data v Identification v Calibration v Quality v Standardisation: time, date, latitude, longitude, spatio-temporal reference v Domain standards

CCSDS Meeting data Archive Ingest - June 2006 CNES 7 CNES standards (RNC) applicable rules and recommendations for projects  Metadata Metadata for identifying a set of data v Overall description of a set of data v Description of granules Metadata for using data  Auxiliary data  Data to help researchers  Documents and publications  Relation between project process and work phases defined in the PAIMAS

Project phases Project-Archive interactions Documents delivered by the Project Comments Phase APreliminary definition phase Archiving plan, draft versionThe preliminary agreement between the Project team and the Archive department should be described in detail during the Preliminary Requirements Review (REP). Phase BFormal definition phase Archiving plan, complete version The formal agreement between the Project team and the Archive Department should be described in detail during the Preliminary Design Review (RDP). Phase C and D Formal definition phase Final version of archiving planDefinition, production and qualification of ground segment. Phase E – kick-off Transfer and validation phase Formal acceptance prior to beginning transfer to ensure: - compliance of data with what is expected - correct operation of transfer process Possible revision of archiving plan during phase E. Acceptance report The archiving plan may be revised depending on events which occur during phase E: total or partial failure of an instrument or on the contrary extending of the mission beyond its nominal duration. Phase E - routine Transfer and validation phase In-depth validation. The data and documents to be archived are not all delivered by the Project team at the same time. The coherence and completeness of all documents should thus be validated during phase E.

CCSDS Meeting data Archive Ingest - June 2006 CNES 9 Exchange standard for data to be archived  Published in March 2006 by the Ministry of Culture and Communication and the Ministry of the Economy, Finances and Industry  Purpose: to exchange information between the Archive departments and their partners Producers Archive management Supervisory bodies Archive users

CCSDS Meeting data Archive Ingest - June 2006 CNES 10 Exchange standard for data to be archived  This covers: transfer between producer and archive (SIP) transfer between archive and user community (DIP) changes to the archive at the producer’s request deletion of documents

CCSDS Meeting data Archive Ingest - June 2006 CNES 11 Exchange standard for data to be archived Example of sequence diagram Producer Archive SIP transfer Transfer receipt Notification of SIP validation Anomaly warning Receipt of anomaly warning

CCSDS Meeting data Archive Ingest - June 2006 CNES 12 Exchange standard for data to be archived  UML modelling  Taking into account of signatures XMLDsig PKCS7  Implementation as an XML diagram  Objective to automate transfer and validation while taking national specificities and regulations into account

CCSDS Meeting data Archive Ingest - June 2006 CNES 13 Ongoing work for Producer Archive Interface Specification  The document is currently being thoroughly revised Some concepts have been changed The terminology has also evolved Much thought has gone into the work while taking into account: v the initial work achieved v analysis done by NASA v descriptions of actual cases v the new version of XFDU This will be covered in a special presentation

CCSDS Meeting data Archive Ingest - June 2006 CNES 14 The end…