1. @ulccwww.ulcc.ac.uk IRMS Cymru October 2015 From EDRMS to digital archive: a wish-list for ways to preserve digital records

2. @ulccwww.ulcc.ac.uk About me Ed Pinsent, Digital Archivist at ULCC since 2004 Teaches digital preservation on the DPTP Background as archivist / records manager Experience in web-archiving, repository management, metadata projects, migration, digitisation, project management, etc. See more at digital archives blog http://dablog.ulcc.ac.uk/ http://dablog.ulcc.ac.uk/

3. @ulccwww.ulcc.ac.uk Scope of this presentation Do we need to preserve born-digital records? If so, how can we do it? Will refer to EDRMS (Electronic Document Records Management Systems)… And to a preservation environment, e.g. a dedicated digital archive / repository

4. @ulccwww.ulcc.ac.uk Some issues 1.Need for keeping records may exceed life of the EDRMS 2.Need for keeping records may exceed life of file formats they are stored in 3.Can we get content / metadata out of the EDRMS? 4.If we need preservation, where do we do it?

5. @ulccwww.ulcc.ac.uk #1: The EDRMS problem You may have a long-term business need for retention May only affect a small percentage of records (e.g. Pension records, contracts) Will your EDRMS last that long?

6. @ulccwww.ulcc.ac.uk #2: The file format problem Digital file formats may become obsolete / unsupported / unreadable Common response in digital preservation is to migrate But it’s extra work: picking a suitable target format, finding a conversion tool Doing this creates even more metadata We’d want to automate it, ideally

7. @ulccwww.ulcc.ac.uk #3: Can we export from EDRMS? Can we get the digital objects / files out? Can we get the metadata out? Do we want to? –Do we want MOREQ conformance? –Reports on the system itself (audit trail) –Reports on use of the records –Both add authenticity / legal admissibility to records

8. @ulccwww.ulcc.ac.uk #4. Where can we do preservation? In the EDRMS itself? Or in a dedicated archival repository system?

9. @ulccwww.ulcc.ac.uk EDRMS might not work for preservation According to DIRKS (2012)… Q: Will the EDRMS automatically manage the long term needs of records? A: No. There are a number of factors that can still threaten the long term accessibility and use of records. Vendor support Software upgrades Move to new EDRMS must be handled carefully Data loss, corruption, alteration risks

10. @ulccwww.ulcc.ac.uk EDRMS might not work for preservation Archives New Zealand: Archives New Zealand is now in a position to accept digital records exported into its custody where the records have been appraised as having ongoing archival value. Requirements for digital transfer remain flexible to incorporate existing digital records not accompanied by recordkeeping metadata to the standard.

11. @ulccwww.ulcc.ac.uk Possible Gaps (Technical) 1.No “bridge” from EDRMS to archive repository 2.Not always clear if we can export out of EDRMS, or in what format 3.Do we want to capture the EDRMS audit trail? 4.Can’t easily change file formats in the EDRMS 5.Additional metadata needed for preservation

12. @ulccwww.ulcc.ac.uk Possible Gaps (Human) Records ManagerArchivistIT Manager All I care about is current records and the next five years Our selection policy means I have to preserve a history of this organisation forever I don’t care about either of you

13. @ulccwww.ulcc.ac.uk Life Cycle of Record Creation Maintenance and Use Preservation Destruction Disposition EDRMS Functions Archive Repository Functions

14. @ulccwww.ulcc.ac.uk What is an archival repository? A dedicated space for storing and managing archival copies securely Allows us to protect important content Can carry out migration in a controlled and professional way Uses tools which EDRMS doesn’t have Can create preservation metadata

15. @ulccwww.ulcc.ac.uk Some proposed steps towards preservation 1.Survey 2.Define use cases 3.Do preservation planning 4.Define ingest requirements 5.Define data management requirements 6.Define access requirements 7.Think about migration

16. @ulccwww.ulcc.ac.uk Terms… Migration = transforming digital objects from one file format to another Moving = taking electronic records out of the EDRMS Export = will use this term to apply only to metadata

17. @ulccwww.ulcc.ac.uk 1. Survey Survey record holdings Create an inventory Identify record types with long retention needs (e.g. 30-50 years) Note file format types in use May be a small percentage

18. @ulccwww.ulcc.ac.uk 2. Use cases Identify types of users Find out what these users expect to be able to do with the records Make decisions about how much you can support technically

19. @ulccwww.ulcc.ac.uk Example A Internal users might want… –To find documents –To read documents –To amend or edit them (unlikely, but possible) –To delete them –To operate a disposal rule (if they are the records manager)

20. @ulccwww.ulcc.ac.uk Example B External users might want access, but… –Documents might be closed to the public –When can they be opened? –Can they be released in redacted form? –What about an auditor or other legal official? –What are your legal obligations?

21. @ulccwww.ulcc.ac.uk And what about… Individual digital objects –Will you need to retain formatting? Colours? Fonts? –Editability and processability? –Formulae in spreadsheets? –Email attachments? Provenance and original order –Will you retain original file names? Folder names? References? History of use?

22. @ulccwww.ulcc.ac.uk The value of use cases Have implications for what kind of preservation system you are going to build –Functionality –Search facilities –Access mechanisms (security marking, redaction)

23. @ulccwww.ulcc.ac.uk The value of use cases Have implications for managing digital objects –When and how will you migrate objects? –What can you afford to lose in migration? –How much do you need to support?

24. @ulccwww.ulcc.ac.uk 3. Preservation planning When will you move? What will you be moving? What about keeping metadata from the EDRMS, e.g.: –File plan scheme? –Retention scheduling rules? –Use history? –Access / security markings? Will you migrate file formats? When? How?

25. @ulccwww.ulcc.ac.uk Can we move / export? TNA have noted: Most systems “have not been designed for easy import and export of information.” “It is rare that an EDRMS export captures everything held in the system”. (Migrating Information between EDRMS, Digital Continuity Project, 2010)

26. @ulccwww.ulcc.ac.uk Bonus export slide Microsoft ain’t all bad… Outlook has export functions, e.g. to.pst SharePoint can export to.cmp file Both methods may allow replay of content and context in a preservation environment

27. @ulccwww.ulcc.ac.uk 4. Define ingest rules Move objects from EDRMS Export EDRMS metadata Extract technical metadata from objects Identify access / closure requirements Build preservable packages

28. @ulccwww.ulcc.ac.uk 5. Define data management How will you store and manage metadata? –Keep it as a chunk of XML –Incorporate metadata with your database –Map it to METS What about preservation metadata?

29. @ulccwww.ulcc.ac.uk 6. Define Access Devise access rules for records, e.g.: –Security markings –Authentication procedures –Redaction processes Apply them to your preserved objects

30. @ulccwww.ulcc.ac.uk 7. Migration Convert old file formats into supported file formats Devise acceptance criteria (part of preservation planning); what essential properties must not be lost through migration? Learn from your use cases; you cannot support “everything” Choose conversion tools Identify formats that may cause problems Decide when you will migrate Keep records when you do it (preservation metadata)

31. @ulccwww.ulcc.ac.uk Possible result Digital archive Access function Internal user External user Auditor Search Read Edit Audit Dispose - Preserves - Holds metadata - Keeps some object functionality - Replicates some functions of EDRMS - Creates and serves accessible copies - Redacts - Migrates

32. @ulccwww.ulcc.ac.uk Wishlist: Technical More bridge-building between systems Keeping as much original metadata as possible Can we map MoREQ to METS? Move towards agreement on the ideal sharing format (e.g. XML) Automation (where possible)

33. @ulccwww.ulcc.ac.uk Wishlist: Planning An integrated digital records-to-digital archives lifecycle Do migration early, where appropriate Authenticity and integrity of records retained Someone in organisation taking ownership of preservation

34. @ulccwww.ulcc.ac.uk Wishlist: Human Records ManagerArchivistIT Manager We need long- term retention of these records, but I can’t do it in my EDRMS I’ll learn more about practical digital preservation and build a digital archive You both have credible business plans, so we’ll invest in the IT you need and support it