KEEPS – a system for UELMA preservation and security Jason Duffing Office of the Revisor of Statutes, Minnesota

Office of revisor of statutes Available on our website Statues from 1851 to 2016 Laws from 1849 to present Rules to 1982

KEEPS Custom software solution for the preservation of our UELMA documents. KEEPS stands for Keep Electronic Edits Preserved & Secure

UELMA UNIFORM ELECTRONIC LEGAL MATERIAL ACT Minnesota enacted UELMA IN 2013 (Minnesota Statute chapter 3E) UELMA establishes an outcomes based technology neutral framework for providing online digital legal material with the same trustworthiness as traditional pater publications.

UELMA Minnesota Implementation Server-side authentication Complete versioning system Tracking and auditing tools developed Support for multiple formats (only PDF is authentic) Long term preservation strategy (KEEPS System)

Authentication Every published PDF is fingerprinted (secure hash) Upload PDFs to web site to verify authenticity What gets authenticated Documents containing legal material: Chapters, Sections Publication front matter, tables, indexes (book aids) change one character in the PDF and it will return a new hash

Authentication Upload Page A screenshot of the MN Revisor of statutes authentication page

Authentication Success A screenshot of the MN Revisor of statutes authentication result page --does not matter what the PDF file is named. The Name from the revisor system will be presented to the user

Authentication Failure A screenshot of the MN Revisor of statutes authentication result page with a failure message

Versioning Authenticating older version of rule Example: Minnesota Rule 6264.0400 A screenshot of the MN Revisor of statutes authentication result page that authenticates a historical version and notifies the user that a updated version of this document is available.

UELMA Authentication Pros: Easy Cheap No third party dependencies Cons: Requires manual action Requires internet connection Things Considered Adobe PDF Authentication

Formats Store and hash (fingerprint) PDF and XML PDF is authenticatable XML is not made available and is not authenticatable at this time HTML is transformed on-the-fly from XML Allows documents to be displayed with modern techniques Allows future features to be integrated into HTML easily

Preservation Legal Requirements Minnesota Statutes 3E.07 Subdivision 2.Requirements if preservation in electronic form. If legal material is preserved under subdivision 1 in an electronic record, the official publisher shall: (1) ensure the integrity of the record; (2) provide for backup and disaster recovery of the record; and (3) ensure the continuing usability of the material. A record has “integrity” if it is preserved without any alteration that would impair its use as an authentic record.

What are we protecting from Hackers Catastrophic System Failures Coworkers This slide has a picture of a hacker, system failure, and a coworker

Why isn’t a tape backup good enough Not easy to restore from tape Hard to detect when documents have changed Problem could go unnoticed for years If restores aren’t practiced. You may not discover issues with your backup system until it is too late. Things Considered Dark archive at the Minnesota Historical Society Documents placed in the vault, never touched again

System Requirements Speed Independence Accuracy Monitoring Sleuthing Validate all UELMA documents daily. Independence Must run independently from our other software with out user intervention Accuracy Must detect any change to a document down to bit level manipulation. Monitoring Email notifications of validation errors. Sleuthing A web interface combining all available information.

Write Once Ready Many (WORM Disk) Essential to ensuring our data integrity Once inserts are made no updates or deletes are possible

Publish Legal documents to web Production Database Legal Documents KeepsServer Worm Disk Diagram of a high level view of the keeps system

Keeps server technologies Oracle 12c UELMA PDF’s are stored in Oracle database Native SHA2 hashing Minimize data IO KEEPS server GreenTec WORM disks Hardware enforced write once. Windows Server 2012 R2 Java se8

Server Architecture KEEPS Server Schedule Manager Archiver Scheduled tasks: Must not run concurrently. Queue-able if another task is running. Schedule Manager Archiver User initiated tasks KEEPS Document Repository (WORM Disk) Table Representation Populator BulkArchiver Validator

System review Worm Table Production Database Document Worm Disk This process can be made to work with any kind of file

What gets compared Document name Date & time of document creation Hash of Production PDF to Hash on worm

Hash stored in Production Hash Compare Hash stored in Production Database Worm Disk Hash Diagram of how hashes get compared Stored Production Hash get compared to worm drive Stored production PDF’s get rehashed and compared to the worm disk

Testing Individual Corruption tests Load tests Unauthorized document inserted into database A document removed from the database Changes to an existing document in the database Load tests Archival of 50k Statute pdf’s (6.3GB) occurs in < 37 minutes. Validation of 500k pdf’s (65GB) occurs in < 30 minutes.

Screen shoot of Keeps validation error report screen

Screen shot of KEEEPS Error detail screen

Archive Projections 510k UELMA Documents 54k Additions in 2016 increasing at 2% annually We project the size of our repository will double in the next 8 years Graph of document size estimates

Preservation Legal Requirements Requirements if preservation in electronic form. If legal material is preserved under subdivision 1 in an electronic record, the official publisher shall: (1) ensure the integrity of the record; Night validation for updates (2) provide for backup and disaster recovery of the record; and Tape backup Production Copy of PDF stored on WORM disk Tape backup of Worm (3) ensure the continuing usability of the material. Due to our backups and nightly validation we can ensure that our documents will be available online forever A record has “integrity” if it is preserved without any alteration that would impair its use as an authentic record.

Questions?