DigCCurr 2007: What digital curators do and what they need to know The CASPAR view on: What digital curators do and what they need to know : Research Perspectives David Giaretta CASPAR Project Director

DigCCurr 2007: What digital curators do and what they need to know What digital curators do: Struggle with: Funders –Reluctant to provide long-term commitments Information providers –Unwilling to provide what is needed Users –demanding ever more sophistication Ways to ensure info is understandable Cost control, Cost estimates Ways to capture required info

DigCCurr 2007: What digital curators do and what they need to know The CASPAR Consortium The CASPAR Consortium

DigCCurr 2007: What digital curators do and what they need to know What do digital curators need to know

DigCCurr 2007: What digital curators do and what they need to know Curation: do preservation and publication/access - but do not confuse them Needs of access: –Responsive –Sophisticated search techniques –Users often familiar with the material Needs of Preservation: –Ensure the information trapped in the bits is authentic and understandable To the Designated Community Transient tools and technologies with changing demands and implementations Not transient Curation also implies making fit for purpose – adding to info

DigCCurr 2007: What digital curators do and what they need to know There are disincentives for preservation: COST Money Time Budget available If cost of preserving old information increases… Need to show that costs are contained

DigCCurr 2007: What digital curators do and what they need to know Preservation can be sold as benefiting Publication/Access : Use of Unfamiliar Data Global Cyber-Infrastructures allow users to find and try to use data from many sources –Some sources will be familiar –Most available sources will be unfamiliar How can one be sure that the unfamiliar data is used correctly Need understanding –Garbage in – garbage out Need to be able to deal with unfamiliar data whether it is contemporary or old (preserved)

DigCCurr 2007: What digital curators do and what they need to know Digital Preservation… Easy to do… …as long as you can provide money forever Easy to test claims about tools… …as long as you live a long time

DigCCurr 2007: What digital curators do and what they need to know Know what is being preserved: the great Data / Document divide Need to preserve information & knowledge – not just “the bits” –Documents, videos are rendered – simple? –Data – must be processed – in new ways - harder Publication of data as well as documents What is the cost of publication and preservation?

DigCCurr 2007: What digital curators do and what they need to know Information is the important thing What information? –Documents…… –Data……. Original bits? Look and feel? Behaviour? Performance? Explicit/ Implicit/ Tacit Information : Any type of knowledge that can be exchanged. In an exchange, it is represented by data. Long Term is long enough to be concerned with the impacts of changing technologies, including support for new media and data formats, or with a changing user community. Long Term may extend indefinitely. Ensure that the information to be preserved is Independently Understandable to (and usable by) the Designated Community.

DigCCurr 2007: What digital curators do and what they need to know Things change/disappear Software Hardware Environment –E.g. Network links to related information People –What is “common knowledge” How can we ensure that the information trapped in the “bits” remains understandable despite all these changes? How can a digital curator even be aware of these changes?

DigCCurr 2007: What digital curators do and what they need to know Your time is short… Neither you nor your institution (or preservation project) will last forever The chain of preservation is only as strong its weakest link Need to be prepared to hand over How can whole collections be handed over? How can the information in the archive managers’ heads be handed over?

DigCCurr 2007: What digital curators do and what they need to know No repository is an island You/your organisation/project cannot do everything –Things change –You will not be around forever Must somehow tap into other resources How can we find these resources? How can we share the resources? Where do the resources come from?

DigCCurr 2007: What digital curators do and what they need to know Wisdom of the world “given enough eyeballs, all bugs are shallow” However it may be a statistical process Is there one right description of something? How can we decide between alternatives?

DigCCurr 2007: What digital curators do and what they need to know We cannot foretell the future Need to manage knowledge to keep archives alive through time –Preservation is a process, not a one-time event –Preservation is expensive – costs need to be shared –Open Archival Information Systems Reference Model (ISO 14721) provides a general conceptual framework ( At least monitor the Designated Community How can this be done over time?

DigCCurr 2007: What digital curators do and what they need to know OAIS – not just the Functional Model diagram The Information Model is key Information Object Representation Information 1+ interpreted using 1+ Data Object interpreted using Physical Object Digital Object Bit Sequence 1+ Recursion ends at KNOWLEDGEBASE of the DESIGNATED COMMUNITY (this knowledge will change over time and region)

DigCCurr 2007: What digital curators do and what they need to know Representation Information The Data Object is “interpreted using” the Representation Information (RepInfo) The Reference Model is designed to ensure that an OAIS is not set the impossible task of having to provide all possible RepInfo immediately Hence: –Take account of the Designated Community and its associated Knowledge Base The amount of RepInfo is not fixed –Additional RepInfo will be needed over time How do we define a Designated Community? How? By whom?

DigCCurr 2007: What digital curators do and what they need to know CASPAR information flow architecture Rep Info Virtualisation How do we capture the Representation Information?

DigCCurr 2007: What digital curators do and what they need to know Authenticity Evidence

DigCCurr 2007: What digital curators do and what they need to know Support infrastructure Registries of Representation Information Representation Information Gap Manager Orchestration Manager Toolkits –Representation Information –Preservation Description Information

DigCCurr 2007: What digital curators do and what they need to know Some shared infrastructure

DigCCurr 2007: What digital curators do and what they need to know CASPAR aims Produce tools and techniques to support digital preservation and make it easier to share the cost –must be relatively easy to use –must have a low “buy-in” in terms of effort required for adoption –must avoid requiring wholesale change of everyone else’s systems –must be decentralised and reproducible so that it can live on after the formal end of the CASPAR project –must be “preservable” –must be open: open source, open standards Cannot do everything –Working closely with the UK Digital Curation Centre

DigCCurr 2007: What digital curators do and what they need to know Can you tell who is selling preservation snake oil? Write out everything as XML? Write things onto holographic storage? Etch text onto titanium sheets? Just migrate to the newest format? CASPAR? …. How to decide?

DigCCurr 2007: What digital curators do and what they need to know Validation Demonstrate theoretical basis “Accelerated lifetime” tests –Changes in hardware –Changes in environment –Changes in Designated Community Demonstrate increased trustworthiness –Measured using Certification process (as/when available)

DigCCurr 2007: What digital curators do and what they need to know Links CASPAR: DCC: OAIS (ISO 14721) Audit and Certification ISO standard development: