Advisory Committee on the Electronic Records Archives April 29-30, 2009 Program Director’s Update.

Slides:



Advertisements
Similar presentations
Chapter 1: The Database Environment
Advertisements

1 of 15 Information Access Internal Information © FAO 2005 IMARK Investing in Information for Development Information Access Internal Information.
Presented to: By: Date: Federal Aviation Administration Registry/Repository in a SOA Environment SOA Brown Bag #5 SWIM Team March 9, 2011.
Digital Preservation A Matter of Trust. Context * As of March 5, 2011.
The Legal Foundation TRICARE Management Activity HEALTH AFFAIRS 2009 Data Protection Seminar TMA Privacy Office.
Configuration management
U.S. Government Printing Office Packaging and Metadata PREMIS Implementers Panel Library of Congress June 13, 2007.
2009 Data Protection Seminar
Digital Futures International Forum - Tuesday 18th September 1 Digital Futures International Forum The Digitisation Standard: Back & Forth Stephen Clarke.
NetDMR Electronic DMR Reporting
Copyright Hub Software Engineering Ltd 2010All rights reserved Hub Document Manager Product Overview.
Iowa Code and Rules Easy Navigation and Search Scope Analysis &Planning Phases Completed Request for Execution Funding.
DRS 2 one in a series of periodic updates Harvard University Library Andrea Goethals October 21, 2009 DRS = Digital Repository Service.
Washington State Digital Archives Presented by: Adam Jansen Digital Archivist Washington State Archives
Authentication of the Federal Register Charley Barth Director, Office of the Federal Register United States Government.
George W. Bush Presidential Library Electronic Records Alan Lowe April 24, 2012.
The National Declassification Center Releasing All We Can, Protecting What We Must Public Interest Declassification Board NDC Project Update April 22,
Transformations at GPO: An Update on the Government Printing Office's Future Digital System George Barnum Coalition for Networked Information December.
Briefing for NOAA Managers
KEEP Project Overview House Government Efficiency Committee March 14, 2012 Matt Veatch State Archivist & KEEP Project Manager
ERA UPDATE ACERA November 6 th and 7 th Items 2  Status  Operations and Maintenance  Corrective and Adaptive  Data at Rest  Q&A November 2012.
Technology Steering Group January 31, 2007 Academic Affairs Technology Steering Group February 13, 2008.
Technology Steering Group January 31, 2007 Academic Affairs Technology Steering Group February 13, 2008.
Electronic Records Archives: Building the Archives of the Future: An Update Fynnette Eaton Change Management Officer BRIDG Meeting June 17, 2004.
1 From Filing Cabinet to Desktop and Network: Records Management in N.C. State Government Ed Southern Government Records Branch N.C. Office of Archives.
Persistent Digital Archives and Library System (PeDALS) A Guide for Wisconsin State Agencies.
Digitization at the National Archives and Records Administration Doris Hamburg Director, Preservation Programs James Hastings Director, Access Programs.
UIS EDEN Workflow Engine Overview of workflow engine for IU’s OneStart portal.
AIIM Presentation Selecting and Implementing A Records Management System June 5, 2008.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
TRANSACTION PROCESSING SYSTEM Liew Woei Song Muhammad Hofiz Achoson.
U.S. Department of the Interior U.S. Geological Survey USGS Records Management Program: Initiatives, Opportunities, and Assistance Administration and Enterprise.
A Dynamic Solution for Electronic Records: The National Archives & Records Administration’s Electronic Records Archives Kenneth Thibodeau, Director Electronic.
Records Management: It’s Not Just Paper
City of Seattle Office of the City Clerk Open Government = Access Challenges and Opportunities with Digital Records.
Persistent Digital Archives and Library System (PeDALS) SC Department of Archives and History.
May 16, 2007 Enterprise Content Management- Paperless Government: Association of Governmental Accountants Enterprise Content Management- Paperless Government:
Module 9 Configuring Messaging Policy and Compliance.
Relationships July 9, Producers and Consumers SERI - Relationships Session 1.
OEI’s Services Portfolio December 13, 2007 Draft / Working Concepts.
I.Information Building & Retrieval Learning Objectives: the process of Information building the responsibilities and interaction of each data managing.
0 GPO’s Federal Digital System October 21, 2008 Selene Dalecky – Lisa LaPlant – Blake Edwards U.S. Government Printing Office.
Meet and Confer Rule 26(f) of the Federal Rules of Civil Procedure states that “parties must confer as soon as practicable - and in any event at least.
1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015.
Archiving and Record Retention Service Cammy Webster Assistant Director - CSD DIS Jan 23, 2007.
Safeguarding the Freedom of Information: Digital Archive Initiatives in the United States Federal Government Michael Paul Huff Information Resource Officer.
U.S. Department of Agriculture eGovernment Program eGovernment Working Group Meeting February 11, 2004.
Developing Policy and Procedure Management System إعداد برنامج سياسات وإجراءات العمل 8 Safar February 2007 HERA GENERAL HOSPITAL.
1 Strategic Plan for Digital Archives Programme DAP PROJECT SCOPE OVERVIEW STATUS.
Lesson Overview 3.1 Components of the DBMS 3.1 Components of the DBMS 3.2 Components of The Database Application 3.2 Components of The Database Application.
Records Administration Conference RACO May 12, 2005 Ronald Reagan Building and International Trade Center Washington, DC ERA Program Update Fynnette.
ECM and Shared Services Overview AITR Meeting April 23, 2009.
GPO’s Federal Digital System December 10, 2009 U.S. Government Printing Office.
ARMA 2005: CHICAGO 50th Annual Conference and Exposition September 20, 2005 ERA Program Update Fynnette Eaton and Mark Conrad ERA Program Staff The Electronic.
Enterprise Solutions Chapter 10 – Enterprise Content Management.
National Archives and Records Administration Status of the ERA Project RACO Chicago Meg Phillips August 24, 2010.
Fedora and the Preservation of University Electronic Records Project NHPRC Electronic Records Research Grant Kevin L. Glick Manuscripts and Archives, Yale.
Preserving Electronic Mailing Lists as Scholarly Resources: The H-Net Archives Lisa M. Schmidt
Introduction to Databases Dr. Osama AL Rababah. Objectives In this capture you will learn: Some common uses of database systems. The characteristics of.
ERM E-Gov Initiative Transfer of Permanent Geospatial Data Records to NARA Briefing for FGDC Coordination Group Bruce Ambacher, NARA September 2, 2003.
Building Preservation Environments with Data Grid Technology Reagan W. Moore Presenter: Praveen Namburi.
Chang, Wen-Hsi Division Director National Archives Administration, 2011/3/18/16:15-17: TELDAP International Conference.
Digitalcommons.unl.edu Archiving Department Records.
Delivering Results and Satisfaction: Strategic Planning for Information Professional Leaders and Managers SLA 2016 Annual Conference Donald Malcolm Smith,
Committee on Information Technology Planning and Budget Sub-Committee
An Overview of Data-PASS Shared Catalog
An Introduction to Tessella and The Safety Deposit Box Platform
Creating The Oregon State Electronic Documents Repository
Electronic Records Management Program
Digital Preservation at NARA Policy, Records, Technology
Presentation transcript:

Advisory Committee on the Electronic Records Archives April 29-30, 2009 Program Director’s Update

Topics  Development and deployment of the ERA instance for the G.W. Bush presidential records  Plans for further development

Where is ERA? Where is ERA?

Rocket Center, WV

Erma Ora Byrd Conference & Learning Center

The Search & Access ERA Instance for G. W. Bush Electronic Presidential Records

What Does the Base ERA Do?  Focus:  Functions:  Federal Records  Nationwide records management program  National Archives  Creation, review and approval of records schedules  Manage transfer of physical and legal custody of all types of records  Systematically collect, create, and manage lifecycle data about records  Actual transfer, inspection, and archival storage of electronic records

What Does the Search & Access ERA Do?  Focus:  Functions:  Presidential Electronic Records  George W. Bush Presidential Library  Rapid ingest of very large volumes of electronic records  Automatic indexing on ingest  Immediate searchability, based on index  Creation of different versions to support structured search of priority records  Basic case management for review and redaction of sensitive content.

Search and Access Instance Development  Achieved Initial Operating Capability December 8, 2008  LMC proposed and received NARA and EOP agreement on an expedited method for transfer of electronic records.  NARA has enjoyed excellent collaboration from the EOP.  NARA implemented a contingency plan for access to high priority e-records, the finding aid for WH paper records and the database of digital photography, pending completion of processing into ERA.

1/26 EOP Transfer & Ingest Overview ARMS (PRA) = 1.9 TB PDS = TB WARDS =.018 TB SAN B 1 PDS (delta) = TB WARDS (delta) = TB SAN A Exchange 12/5 (IOC) 12/8 ARMS (SAN) 12/15 PDSWARDS 1/15 RMS 1/30 Merlin One = 36 TB Non-Pri Types = 20TB RMS = 1.0 TB 6.0 Storage Arrays ARMS (PRA) PDS WARDS PDS (delta) WARDS (delta) Merlin One 1/20 Data Type SW Drops SASS Operations (Ingest) 7.0 SAN B Returns 12/12 Merlin One January 15, 2015 RMS 2/11 Snap Server RMS (Update) 11 SAN A 2 Merlin One 2 = 36 TB Exchange Non-Pri Types = 0.2 TB SAN B 2 Exchange = 57 TB ARMS (FRA) = 5.1 TB ? 5/16?

G.W. Bush Presidential Electronic Records Records Number of objects Gigabytes of Data Shipped to ERA Data Center Status Priority Records ( )44,815,1841,68812/8/2008>99% available for search in ERA. There are technical problems with the remaining messages. MS Exchange ( ) 150,000,000 estimated 16,500 estim ated Expected mid May In temporary storage. Conversion to standard format, separation from federal records, and identification of responsible EOP component largely complete. Presidential Diary682,193112/8/2008 and 1/26/ % available for search in ERA digital photography11,220,04431,0001/26/2009Problems require shipment of a second set, expected in mid May Index to White House paper records 313, /26/ % available for search in ERA, but about 6% of the records appear to be missing some pieces of data. Visitor and worker access to EOP buildings 28,922, /8/2008 and 1/26/ % available for search in ERA Index to motion video30551/26/2009In ERA, being processed from WH Counsel572,0511,0571/26/2009In ERA, being processed Other Records>12,000,000>5,450Partial shipment 1/26/2009 Some in ERA, being processed. Remainder expected mid May

Processing Status - 1  All Bush e-records have been transferred to NARA’s custody.  Not all have been transferred to the ERA Data Center in WV.  EOP is maintaining copies until NARA successfully completes ingest.  Archives Operational Issues  Several sets of records were not transferred in the formats previously agreed by NARA and EOP o NARA required retransmission  Some records exhibited anomalies o Some ARMS records had binary data in the “To” field o Some metadata in the digital photography system did not have corresponding images. o Some entries in the Records Management System are missing some fields. o MS Exchange was not divided presidential from federal records or associated with EOP component, and contained numerous duplicates.  EOP is addressing these problems prior to transfer to ABL.  EOP has converted from proprietary to standard format.  NARA will preserve both the original files and the output of the EOP processing. o Encoding of date of birth in the Access system impeded searches on that field.  Viruses have been found in a small percentage of files. o Infected files have been successfully quarantined. LMC & NARA are working to produce clean copies.

Processing Status - 2  Technical Issues  Issues with COTS products: o Automatic indexing of a batch of records stops when errors are found in any of the records; e.g., binary data in headers of . o Erroneous results returned in certain conditions o Incomplete search results returned in other cases. o LMC underestimated storage space needed for the index. Additional hardware has been ordered.  Unanticipated software development needed to ensure complete and accurate mapping between ‘.eml’ produced by the EOP and the original MS ‘.pst’ files  NARA directed LMC to hire a subcontractor to perform actual ingest of records.

Status of Requests for Bush Records  28 Requests for access as of March 17, 2009  Primarily for paper records  NARA has responded using data about the paper records in the Records Management System  A few requests were for digital photographs.  Most requests were addressed using the two systems NARA set up under the Contingency Plan because processing of the records had not been completed at the time the requests were received.  Three requests fulfilled using records on temporary ERA storage.

Plans for Further Development

What’s in Store for the Future?  Increment 2  Preservation Framework o Introduction and use of a variety of tools for different preservation needs  Public access o Information about all types of records o Online access to electronic records  Initial system evolution  Increments  Incremental enhancements in capability & capacity  Continuing system evolution  Governmentwide expansion  Full Lifecycle Management Plans  Appraisal case management and workflow  Search Framework supporting different tools  FOIA and other access case management  Review and redaction of sensitive content

Shared Services ERA Functional View: Current Status System Management Help Desk Network Base Instance EOP Instance White HouseAgencies Enterprise Service Bus Data Management

Shared Services ERA Functional View: Planned System Management Preservation Framework Public Access Help Desk Network Base Instance EOP Instance White House Congressional Instance Committees Records Center Instance Agencies Public Enterprise Service Bus Current capability: solid fill Future capability: hashed fill Data Management

ERA Instances  Base Instance (June 2008)  Used by NARA and federal agencies  For management of all federal records  For transfer, inspection and management of federal electronic records  EOP instance (December 2008)  Used by NARA and Presidential Administrations  For transfer, inspection, and management of presidential electronic records  Congressional Instance (future)  Used by NARA for Congressional Committees  For transfer, inspection, and management of presidential electronic records  Federal Records Center Instance (future)  Used by NARA and other federal agencies  For transfer and storage of temporary and permanent federal electronic records that remain under the control of the originating agency

ERA Shared Services  System Management (current)  System operation and maintenance  Security  User account management  Deployment of new & updated software  Backup & other common services  Help Desk (current)  Respond to technical questions and issues from users  Network  Link to the Internet, NARANET (current)  Interfaces with other systems (future)  Data Management  Data about records and transactions related to them (current)  Description of NARA holdings (Increment 2)  Review and redaction of records with restricted content (future)  Preservation Framework (Increment 2)  Tools to overcome obsolescence of different digital formats (future)  Public Access (Inc. 2 +)  Search and retrieval of information about records, regardless of custody  Search and access to electronic records in NARA’s custody  Search and access to digitized records from NARA’s holdings  Freedom of Information Act for restricted records in NARA’s custody

Advantages of the Instances & Shared Services Approach  Instances enable different business rules and processes for different mission requirements:  Base Instance: Federal Records Act provisions on governmentwide records management and on the National Archives  EOP instance: Presidential Records Act  Congressional instance: House and Senate rules.  Federal Records Center Instance: Federal Records Act provisions on storage of temporary and permanent records under originating agencies’ authority.

Advantages of the Instances & Shared Services Approach  Shared services maximize utilization of resources, reduce redundancy and provide a stable foundation for system growth and evolution over time.  Shared services deliver capabilities and capacity wherever needed, regardless of differences in mission and business needs  E.g. the Preservation Framework can be used to preserve any electronic records, regardless of whether they came from Congress, the White House or a federal agency.  E.g., a citizen seeking access to information will be able to find it using a single web portal, regardless of whether o It is information about records or in the records, o the records are in NARA’s physical custody, o the records are electronic or hard copy, o they originated in the White House, Congress or an agency.

Preservation Electronic Record 2 Preservation Framework Record Identity Record Integrity Original Order Tool 1 Tool 2 Tool n … The Preservation Framework supports the introduction and use of an arbitrary number and variety of processes under the control of archival requirements for authenticity. Electronic Record n Electronic Record 1 Electronic Record 2’ Electronic Recordn’ Electronic Record 1’ …

Public Access  Information about all records  From Records Schedules  Archival Descriptions  Other NARA information  Online access to electronic records  Online access to scanned versions of hard copy records  Requests for copies of records  Freedom of Information Act requests for restricted records  Assistance from NARA staff

Increment 3 Work Status  Authority to Proceed Issued for Early Analysis  Architectural Framework  Preservation examination and prototyping  Search Engine examination and selection  Open Access examination and selection  Enhancements to address authorized user defined changes and software defects not addressed at IOC  Discussions begun on scope of work and technical details for full proposal  Target date for award: 7/09

Governmentwide Expansion  Initial Implementation  June 2008 – June 2009  Four collaborating agencies  NARA staff proxy for other agencies  Invitational Phase  June 2009 – February 2010  Additional agencies by invitation  Voluntary Phase  February 2010 – December 2010  Additional agencies who volunteer and meet critera  Mandatory Phase  January 2011  All agencies

The Development Timeline Full Operating Capability Initial Operating Capability) 6/08 Operation & Maintenance 9/059/069/079/089/099/109/11 Search & Access ERA Public Access & Preservation Framework Enhancement ERA Base