June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Submission Agreements: The role they play in supporting the relationship between the Data Producer/Provider and the Archive Dan Kowal, Data Administrator, NGDC

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal NRC Report National Research Council Highlighted Entrepreneurial Activities in data collection. Recommendations: NGDC can strengthen relationships with scientific data providers by working with them from the beginning of data collection projects to ensure that relevant data and metadata are obtained and archived properly.

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Agreement Circa 2004

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal OAIS to the Rescue ISO 14721:2003 ISO 20652:2006

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal OAIS Framework SIP = Submission Information Package AIP = Archival Information Package DIP = Dissemination Information Package PRODUCERPRODUCER CONSUMERCONSUMER MANAGEMENT SIP Ingest Descriptive Info. AIP Access AIP Data Management Archival Storage Descriptive Info. Preservation Planning Administration DIP queries result sets orders OAIS (archive)

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Submission Agreements Introduction Contacts SIP Description Ingest Archive Storage Archive Access “Letter of Intent” ~ 30 SA Docs in play at NGDC

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Sub. Agreements - The Who Contacts Section: –Organizations Involved –Roles (administrative, technical, data) –Designated Community

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Sub. Agreements - The What Introduction: –Scope of the document –Change Management –References supporting the SA. Submission Information Package –Citation Information –Format Descriptions and Documentation –Preservation Description Information

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal The Where, When and How Ingest: –Methods –Schedules –Buffering and Security Measures –SIP Inventory, structure and quantities. –Validation, Error Checking, Reporting Archive Storage –Growth Estimates –Retention Schedule –Contraints, Updates and Risks.

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Where, When and How cont. Archive Access: –Constraints –Discovery and Dissemination Tools. –Search, Display metadata definitions. –Metadata Availability –Designated Community Support. Appendices –Terminology, Acronyms –Document Review Cycle –Help Section –Additional details about the data arrangement.

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Is an SA Important? Without one: The digital objects received do not conform to what the Archive expects. What the Producer delivers has not been clearly defined. The ingest schedule is not fulfilled by the Producer. Errors in transfers are detected late by the Archive, or are not detected prior to use. Results could be: Increased workload Poor quality of archived information Negative effect on the relationship between Archive and Producer.

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Sub. Agreement Evolution Maintenance and Format: Submission Agreement Working Group (SAWG) Version Control within CLASS Knowledge Tree. ATRAC SA Creation Tools. Tactics and Issues: Old vs.New. Data Managers act as Proxies for Many Producers Producer consolidation into one SA - eg.: UNOLS-R2R Timely revisiting of the SA for monitoring updates. Handling the Katrinas and DWH.

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal “What to Archive” Documents The Complete Guide Informational Brochure Featured Best Practice at NARA Available at:

June 21, 2011EDMC Workshop in Silver Spring, MDDan Kowal Staff member confirms request is appropriate to NGDC and conducts Preliminary Appraisal Using R_to_A Questionnaire. Conduct Preliminary Appraisal Confirm appropriate Data Center and designate Appraisal Team Complete All Sections as Appropriate Legend Conduct Formal Appraisal Data Provider Data Center Receives Request to Archive Is a Formal Appraisal Required or CLASS support wanted? YesYes No Assemble Recommendation Package Preliminary Information Detailed Information Does Recommendation request use of CLASS? No Ye s Submit Final Recommendation Package to Data Center for Decision Data Center Makes Decision and Notifies Provider Recommendation Package Preliminary Agreement Requirements Documents Draft Preliminary Agreement with Provider and Requirements Documents (if needed and with CLASS if appropriate) Finalize Submission Agreement with Provider and Design Documents (with CLASS if appropriate) Design Documents Submission Agreement Metadata and Documentation Ingest and archive data Data to be archived NGDC Simple Case from Sea Level Draft Requirements Yes No, suggest alternatives 28 Questions 10 Dimensions COWG Evaluates and Makes Recommendation back to COPB COPB Tasks COWG to Evaluate Request COPB Makes Decision and Informs Data Center Data Center Makes Request to COPB for use of CLASS Resources Revise Recommendation Package as Necessary No or Conditional Yes Yes Cross-Data Center Activity Within-Data Center Activity Submission Agreement November 18, 2010NDIIPP Workshop at LOC on Geospatial DataDan Kowal