1. DAI
Working Group Report
David Giaretta (WG Chair) 1 1 1

2. Start and/or Target Publication Date
DAI Executive Summary
Goals for this meeting cycle: (1) Initial review of OAIS and ISO RIDS (2) Decide on DEDSL/XML Schema disposition (3) Progress IPELTU (4) Forward planning of DAI work in the context of overall architecture
Working Group Status: OK
46 RIDS analysed for OAIS and ISO – no major issues identified. Actions assigned for feedback on RIDS
Decided that DEDSL/XML Schema will be proposed as OB (hope to make new project for BB when 2nd proto is available
Progress made on IPELTU and actions assigned to aim for completion January 2017 (2 month delay)
Progress and agreements on overall DAI architecture – consistent with SM&C architecture
Discussion with SANA registry expert about using SANA Registry for Representation Information
Feedback provided to MOSSG and Roger Thompson on MOIMS Function and Service Views and RID Requirement
50% of attendees were from Industry and 10% from Academia
Weekly Webex meetings, and development of documents, have continued through the year
Problems and Issues:
Need further discussions with SM&C, Roger and SANA to ensure consistency between these and DAI forward plans
See additional slides below
Planning:
Interaction with other WGs
Held joint meeting with SM&C to discuss DAI Architecture. Plans agreed to look at using SM&C COM and MAL prototype software to implement Archival Information Packages as defined by OAIS
Coordinated with SEA on overall MOIMS architecture diagrams
Coordinated with SANA reps on registry needs
Resolutions
DEDSL-XML/XSD should be published as OB CCSDS O-1 and remove CNES from resource list for prototyping
Appoint John Garrett as DAI Deputy Chair
Proposed CMC resolution of support for DAI plans and interactions with larger ISO community (see last page)
Area and WG name
CCSDS Ref Nr
Activity
Document Title
Status
Start and/or Target Publication Date
Comments
MOIMS DAI
CCSDS M-2
Update
Reference Model for an Open Archival Information System (OAIS).
Reviewed 40 suggested changes received so far
Draft update should be produced 6/30/2017
Publication 7/7/2019
Collecting suggested changes as broadly as possible before producing draft for Agency review
CCSDS M-1
Audit and Certification of Trustworthy Digital Repositories
Reviewed 5 suggested changes received so far
Draft update should be produced 7/18/2017
Publication 10/30/2019
As above
CCSDS M-1
Producing RB
Information Preparation to Enable Long Term Use (IPELTU)
Development of document progressing
MB to be published by 3/31/2018
JAXA is interested, resources being identified. Possible issues getting resources allocated / schedule.

3. Major achievements:
Started major reviews of OAIS and ISO with major international interest
Development of draft overall architecture, consistent with architectures of other WGs

4. DAI Upcoming New Work Items
Area and WG name
CCSDS Ref No.
Activity
Document Title
Target Start/ Publication date
Resources Needed (total, Editor, Proto1, Proto 2)
Comments, Rationale.
What if not started?
MOIMS DAI
TBD GB
Digital Archive Architecture Description Document
Start 2017
Publication 2020
2017
2018
2019
4 WM for GB
2 WM for GB
0 WM Proto 1
0 WM Proto 2 4 4 4

5. Special DAI WG topics for the CMC
Rationale for this discussion:
The real work of the DAI WG is not well understood by the space agencies, hence the space agencies are faltering in their support.
DAI WG needs an expression of commitment from the CMC because the external (ISO) community has expressed concern that the CCSDS space agencies will not fully support their needs (because of 1).
DAI WG has a new long-range notional plan, and we seek CMC notional approval (understanding that project approvals will come much later).

6. Important starting points for this discussion
The DAI working group has been focused on long-term preservation of data, such as space mission science results.
“Long-term” implies access even after the mission has been terminated for many years.
DAI has not addressed “near-term” telemetry archive access (for example) by MOC systems while the mission is still underway.
HOWEVER: it is important to note that long-term preservation requires functions to be performed during the mission, capturing data descriptions, etc.
The DAI WG is developing an architectural concept that should harmonize both near-term and long-term archive capabilities.
The SM&C working group has had “near-term” archive work on it’s future work plan for some time, but it has not started.
DAI and SM&C have jointly discussed the working group roles and the work plan for both near-term and long-term archive standards during the Rome meeting.

7. Special CMC discussion – OAIS Review
OAIS 5-year review is due. DAI WG has already started gathering/evaluating comments.
ISO community has expressed concerns about CCSDS being “inclusive and transparent” with non-space-agency participants.
Simply from organization standpoint, not from history
Concern is that externals will not have a “voice”
As in the past, DAI plans to include ISO participants early in the CCSDS review schedule (The ISO DIS review will be done earlier than other WG documents).
This is normal DAI WG practice, but it deviates from normal CCSDS practice.
If DAI did not include ISO during early review, then there would be much rework during the normal ISO DIS/FDIS phase.
The ISO community has asked for CMC commitment (to DAI and ISO) to support this approach for the OAIS review.
CMC has a responsibility to respond to the ISO community with a resolution (see last page)

8. Special CMC discussion – DAI Direction
DAI WG has been inaccurately accused of inefficiencies
See historic record of standards production (see next page)
This is a large output using resources outside space agencies.
DAI has a plan for future work
Discuss tentative schedule for future work (see next page)
DAI created the most popular CCSDS standard (OAIS) that has “gone viral” and been adopted globally, outside of space programs.
In 2015, OAIS was 10% of the total downloads of all 300+ standards in CCSDS (1% of documents is 10% of downloads)
It is true that DAI has participation from many other organizations that are not space-related; National Archives, Libraries, Industry, and Academia etc.
We think this is positive for resources, CCSDS stature, etc.
If the CMC thinks this is not positive, the DAI WG would like to understand that rationale.
This is a parallel to DTN which started in CCSDS and is now evolving to an IETF standard.
We think these standards that become applicable outside of the space community are a great thing for CCSDS, but we would like the CMC to confirm their agreement.

9. DAI WG History & *Notional* Future Schedule
Current
Future
Historic

10. DAI WG Technical Direction
DAI WG (starting with Panel 2 work) has been extraordinarily successful with archive process standards (OAIS) and some interface standards (PAIS).
Interoperable processes are just as important as interoperable protocols
After DAI WG completes the OAIS review, we plan to return to a focus on interoperable protocol work.
As a minimum, we need CMC “blessing” to go down this path
Understanding that each project will require separate CMC approval.
Even better, we need CMC agency resource support to start more protocol work with leading technologies.
Any added support level would be greatly appreciated
Importance of long-term digital preservation is not fully appreciated in CCSDS
Google the topic “digital dark ages”
CCSDS agencies should be addressing *interoperable* access to long-term archives of science results after the mission is over.
Establish a “framework” for the community, then address archive protocols *only* for CCSDS space agencies.

11. Concept– Standardized Archive System Architecture
Producer Archive
Application
Consumer Archive
Application
Legacy USER Interfaces (i.e. data warehouse interfaces
Producer Archive I/F
Consumer Archive I/F
Multiple Future
Bindings/plugins
For many various
Archive types.
Including vendor-
unique archives
or datastores.
Mission Telemetry
Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive Abstraction Layer
Legacy DATA Interfaces (i.e. data warehouse interfaces, proprietary interfaces, etc.)
Science Data
Archive I/F
Enterprise Data
Archive I/F
Web/HTML
Archive I/F
Doc/PDF
Archive I/F
ARCHIVE -- Long-term Data Preservation Archive (w/ Archive Information Packages, etc.)

12. Conclusions
DAI has a draft plan for future work on interoperability protocols.
DAI has a firm plan for the OAIS review, incorporating ISO participants (external to CCSDS).
DAI work has been recognized as important to not only the space community, but to the world at large.
DAI looks for CMC agreement to proceed.
DAI encourages more support from CCSDS space agencies to the DAI WG to insure a “balanced” approach (balancing internal/external to CCSDS)
If YOUR space agency expects
YOUR MISSION DATA to be widely accessible
after the mission, then YOU need experts in the DAI WG ! ASAP !

13. Recommended CMC Resolution:
Considering that:
The DAI Working Group has traditionally folded ISO participation into the “early” CCSDS Review processes, and;
This has been shown to be successful and efficient for document reviews that include major ISO community participation.
And recognizing that:
The DAI Working Group has much participation from communities external to the space agencies, such as national archives and libraries, and;
This participation is important not only to fulfil the need to establish archive processes, architectures and protocols for the external community, but to secure the needs of and stature of CCSDS in the global standards arena.
The CCSDS Management Council resolves that:
The DAI Working Group should proceed with its review plans to include broad community and ISO participation, and;
The DAI Working Group should endeavor to establish a framework architecture that meets the needs of the Space Agencies of CCSDS as well as the broader ISO community, and;
The CCSDS Agencies should consider anew the importance of the work of long-term preservation of the digital heritage of the spaceflight missions of the world, and endeavor to improve agency participation in the DAI Working Group.

14. Backup

15. “Digital Dark Ages”
Failure to preserve digital data in the long term may result in our era being the most poorly-recorded in history (or lack of history)
Widely recognized in the internet and archive community
CCSDS already has a key role (OAIS) but there is not currently an industry-wide approach.
By publishing interoperable protocols for archives, CCSDS can insure that an industry-adopted solution covers the needs of the space and science community.
Space Agencies should be working on preservation of space mission products *and* the foundational archive architecture to support that.
No other organization seems to be working on the foundational architecture for ubiquitous archive access.

16. Future Work – rationale for protocol doc tree
PAIP because PAIS is not yet a protocol/API/binding
PAIP would complete the “Producer” suite of specs.
Data Archive ADD would describe the overall architecture for archives all the way down to protocols
May be better covered in an existing DAI book?
However, “ADD” will be familiar terminology to CMC
On the Consumer side, produce a “Consumer” counterpart for each “Producer” spec
Consumer Archive Interface Operations Concept
Consumer Archive Interface Methodology Abstract Standard
Consumer Archive Interface Specification
Consumer Archive Interface Protocol
This is a draft approach – not set in stone.
Still need to add domain-specific “bindings/plug-ins/APIs”

17. More Detailed Draft Concept - Standardized Archive System Architecture
Protocol view
Archive Producer Application
Archive Consumer
Application
Archive Consumer Application
Archive Producer I/F
Archive Abstraction Layer
Archive Consumer I/F
User Systems
Archive Abstraction Layer
Science Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive
Common Calls
Other TBD
Bindings
Science Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive
Common Calls
Other TBD
Bindings
Communications Layers
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Communications Layers
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Bit layer
Structure layer
Semantic layer
Object layer
Provenance Info
Context Info
Reference Info
Fixity Info
Access Rights Info
Provenance Info
Context Info
Reference Info
Fixity Info
Access Rights Info
Representation Information
Data
Representation Information
Data
Preservation
Description Info
Preservation
Description Info
Archive Systems
Bit layer
Structure layer
Semantic layer
Object layer
Archive Information Package Interface Layer
Archive
Preservation activities/processes