1. DAI WG material for joint meeting with SM&C WG
20 Oct 2016

2. Important starting points for this discussion
The DAI working group has been focused on long-term preservation of data, such as space mission science results.
“Long-term” implies access even after the mission has been terminated for many years.
DAI has not addressed “near-term” telemetry archive access (for example) by MOC systems while the mission is still underway.
HOWEVER: it is important to note that long-term preservation requires functions to be performed during the mission, capturing data descriptions, etc.
It’s entirely possible that near-term archive processes/protocols are a subset of long-term archive processes/protocols.
DAI has recently focused on processes (OAIS, IPELTU, etc.) that are foundations for protocols.
Future protocol work needs to respond to those processes, and be compatible with current interface specification (PAIS, etc.).
Now working on incorporating protocol development in the work plan
Partly stimulated by meeting with Vint Cerf at GSFC last Spring.

3. New Architecture Concept
The DAI WG has developed an architecture concept for archives that mirrors (somewhat) the SM&C architecture.
This is notional/draft, and not absolutely locked in as a plan.
DAI intends to coordinate with SM&C and will be happy to accept any support that SM&C can provide.
The architecture will support the needs of a *much larger* community than space agencies.
“Core” portions of the architecture will support all disciplines.
“Protocols/plug-ins/APIs” will be adapted to specific disciplines (“Designated Communities” in OAIS-speak)

4. Standardized Archive System Architecture
Producer Archive
Application
Consumer Archive
Application
Legacy USER Interfaces (i.e. data warehouse interfaces
Producer Archive I/F
Consumer Archive I/F
Mission Telemetry
Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive Abstraction Layer
Legacy DATA Interfaces (i.e. data warehouse interfaces, proprietary interfaces, etc.)
Science Data
Archive I/F
Enterprise Data
Archive I/F
Web/HTML
Archive I/F
Doc/PDF
Archive I/F
ARCHIVE -- Long-term Data Preservation Archive (w/ Archive Information Packages, etc.)

5. Standardized Archive System Architecture
Producer Archive
Application
Consumer Archive
Application
CAIS
CAIP
PAIS
PAIP
Producer Archive I/F
Consumer Archive I/F
AAL
Doesn’t bind to message
/comm systems, rather
Archive application layer
functions
Mission Telemetry
Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive Abstraction Layer
ADD
Multiple Future
Bindings/plugins
For many various
Archive types.
Including vendor-
unique archives
or datastores.
Science Data
Archive I/F
Enterprise Data
Archive I/F
Web/HTML
Archive I/F
Doc/PDF
Archive I/F
ARCHIVE -- Long-term Data Preservation Archive (w/ Archive Information Packages, etc.)

6. Future Work – rationale for protocol doc tree
PAIP because PAIS is not yet a protocol/API/binding
PAIP would complete the “Producer” suite of specs.
Data Archive ADD would describe the overall architecture for archives all the way down to protocols
May be better covered in an existing DAI book?
On the Consumer side, produce a “Consumer” counterpart for each existing “Producer” spec
Consumer Archive Interface Operations Concept
Consumer Archive Interface Methodology Abstract Standard
Consumer Archive Interface Specification
Consumer Archive Interface Protocol
This is a draft approach – not set in stone.
Still need to add domain-specific “bindings/plug-ins/APIs”

7. More Detailed Draft Concept - Standardized Archive System Architecture
Protocol view
Archive Producer Application
Archive Consumer
Application
Archive Consumer Application
Archive Producer I/F
Archive Abstraction Layer
Archive Consumer I/F
User Systems
Archive Abstraction Layer
Science Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive
Common Calls
Other TBD
Bindings
Science Data
Binding
Enterprise Data
Web/HTML
Doc/PDF
Archive
Common Calls
Other TBD
Bindings
Communications Layers
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Communications Layers
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Archive
Common Calls
Doc/PDF
Web/HTML
Enterprise
Science
Other
Bit layer
Structure layer
Semantic layer
Object layer
Provenance Info
Context Info
Reference Info
Fixity Info
Access Rights Info
Provenance Info
Context Info
Reference Info
Fixity Info
Access Rights Info
Representation Information
Data
Representation Information
Data
Preservation
Description Info
Preservation
Description Info
Archive Systems
Bit layer
Structure layer
Semantic layer
Object layer
Archive Information Package Interface Layer
Archive
Preservation activities/processes

8. DAI WG History & *Notional* Future Schedule
Current
Future
Historic

9. More discussion points
This application of SOA originated with SM&C approach.
Would like to hear about “upgrades” to more current technologies.
SM&C has said that RESTful and HTML5 are no problem.
What about Cloud/Containers/Microservices technologies
Can SM&C prototypes be “adapted” to serve as the Archive component prototypes?
You SM&C guys have gotten your agencies to support SM&C so well. Can you be advocates to drum up some agency support to develop this architecture (these protocols) in DAI from the same agencies?
See backup material

10. Backup

11. DAI WG Technical Direction
DAI WG (starting with Panel 2 work) has been extraordinarily successful with archive process standards (OAIS) and some interface standards (PAIS).
Interoperable processes are just as important as interoperable protocols
After DAI WG completes the OAIS review, we plan to return to a focus on interoperable protocol work.
As a minimum, we need CMC “blessing” to go down this path
Understanding that each project will require separate CMC approval.
Even better, we need CMC agency resource support to start more protocol work with leading technologies.
Any added support level would be greatly appreciated
Importance of long-term digital preservation is not fully appreciated in CCSDS
Google the topic “digital dark ages”
CCSDS agencies should be addressing *interoperable* access to long-term archives of science results after the mission is over.
Establish a “framework” for the community, then address archive protocols *only* for CCSDS space agencies.

12. “Digital Dark Ages”
Failure to preserve digital data in the long term may result in our era being the most poorly-recorded in history (or lack of history)
Widely recognized in the internet and archive community
CCSDS already has a key role (OAIS) but there is not currently an industry-wide approach.
By publishing interoperable protocols for archives, CCSDS can insure that an industry-adopted solution covers the needs of the space and science community.
Space Agencies should be working on preservation of space mission products *and* the foundational archive architecture to support that.
No other organization seems to be working on the foundational architecture for ubiquitous archive access.

13. Conclusions
DAI has a draft plan for future work on interoperability protocols.
DAI has a firm plan for the OAIS review, incorporating ISO participants (external to CCSDS).
DAI work has been recognized as important to not only the space community, but to the world at large.
DAI looks for CMC agreement to proceed.
DAI encourages more support from CCSDS space agencies to the DAI WG to insure a “balanced” approach (balancing internal/external to CCSDS)
If YOUR space agency expects
YOUR MISSION DATA to be widely accessible
after the mission, then YOU need experts in the DAI WG ! ASAP !

14. Recommended CMC Resolution:
Considering that:
The DAI Working Group has traditionally folded ISO participation into the “early” CCSDS Review processes, and;
This has been shown to be successful and efficient for document reviews that include major ISO community participation.
And recognizing that:
The DAI Working Group has much participation from communities external to the space agencies, such as national archives and libraries, and;
This participation is important not only to fulfil the need to establish archive processes, architectures and protocols for the external community, but to secure the needs of and stature of CCSDS in the global standards arena.
The CCSDS Management Council resolves that:
The DAI Working Group should proceed with its review plans to include broad community and ISO participation, and;
The DAI Working Group should endeavor to establish a framework architecture that meets the needs of the Space Agencies of CCSDS as well as the broader ISO community, and;
The CCSDS Agencies should consider anew the importance of the work of long-term preservation of the digital heritage of the spaceflight missions of the world, and endeavor to improve agency participation in the DAI Working Group.