1. NSIDC DAAC Accessioning and “De-commissioning” Plans
Ruth Duerr

2. Outline DAAC Data Acceptance Plan Levels of Service
DAAC Data Acceptance – Status and Plans
DAAC Data “De-commissioning” Plan
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

3. DAAC Data Acceptance Plan - Purpose
“ensure that non-EOSDIS approved data and products submitted to the NSIDC DAAC […] are vetted and reviewed to be scientifically valid and viable for research applications.”
“NASA approved product review process is used to recommend data sets for inclusion into the EOSDIS supported collection of NSIDC DAAC Standard Data sets.”
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

4. DAAC Data Acceptance Plan - Heritage
NSIDC Data Management Policy
Discussions with EOSDIS and MeaSures program
Accessioning plans from other agencies and organizations
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

5. DAAC Data Acceptance Plan – Process Overview
Submission package completed and submitted to DAAC
DAAC product team reviews the submission
DAAC User Working Group assesses cost/benefits and makes a recommendation
NASA HQ approves (or not) recommendation
DAAC implements approved requests
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

6. DAAC Data Acceptance Plan – Submission Package
Proposed title
Description
Contact information
Product algorithm theoretical basis
Publication list
Justification
Data acquisition methods
Quality and accuracy information
Intended or appropriate product uses and limitations
Size, format, and ingest frequency
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

7. DAAC Data Acceptance Plan – Product Team Review
Accuracy and completeness checks
Comparison with similar data sets to estimate user base
e.g., users/year
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

8. DAAC Data Acceptance Plan – Product Team Review
DAAC Justification
Programmatic considerations
Scientific assessment
Determine appropriate levels of service
Estimate total effort required to achieve that LoS
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

9. DAAC Data Acceptance Plan – User Working Group
Reviews:
Scientific merit
Proposed LoS (and adjusts if necessary)
Anticipated level of effort required
Makes a recommendation
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

10. NSIDC Levels of Service - Categories
Archival – The relative amount of work required in order to ingest and archive a data set.
Metadata – The amount of work required to develop and maintain metadata not just for the data set as a whole; but also for any data element or service associated with the data set.
Documentation – The amount of work required to document the data set and any associated web pages.
Distribution – The amount of work required to support data distribution or distribution-related services.
USO Infrastructure – A one time setup required in order to provide any level of USO support.
USO Support – The amount of work required to support human-human requests for information about or help with the data set (via any mechanism – phone, , etc.).
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

11. NSIDC Levels of Service – How they work
LoS are a relative weighting of effort to achieve a certain state
A numeric score is given to each state
Scores can be affected by a number of multiplicative factors, for example, for archival some of the factors are:
Data provided on obsolete media - score is multiplied by 5
No format/structure modifications are needed - score is multiplied by .75
The adjusted scores for each category are summed to produce a single number for a data set
The difference in scores between two Levels of Service reflect the effort required/freed by moving a data set from one LoS to another
The difference in scores between LoS scores for two different data sets reflect the difference in the effort needed to accept each data set
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

12. NSIDC Levels of Service - Categories
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

13. NSIDC Levels of Service – Low Effort Example
A PI requests NSIDC to archive and distribute the data she acquired during a particular field campaign. The data is in a standard format and is available electronically (FTP pull) but is expected to be of interest to a limited set of users, so only a skinny catalog entry and readme file are scoped for development. Fortunately, the PI is willing to provide the metadata and information for the readme file. NSIDC expects to distribute the data via an FTP site and provide basic documentation support for end users.
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

14. NSIDC Levels of Service – Low Effort Example
Service Categories and Desired States
Score x Factors
Archival State - NSIDC 1-time ingest = Affecting factors: No format mods are needed =
5 x.75
Archive Total:
3.75
Metadata State - Skinny catalog = Affecting factors: None
5 x 1
Metadata Total: 5
Documentation State - Advertised + Readme Affecting factors: PI willing to provide the Readme file
10 x .5
Documentation Total:
Distribution State - FTP access Affecting factors: None
Distribution Total:
User Services Infrastructure Preparation
USO Infrastructure Prep Total:
User Services State - Documentation Support Affecting factors: Monthly user support expected
10 x.5
USO Services Total:
Grand Total Score:
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

15. NSIDC Levels of Service – High Effort Example
A PI requests NSIDC to ingest and distribute a 10-year time series of daily observations. The observations are on-going and the PI would like NSIDC to acquire them as soon as they are available after being QC’d by the PI. The PI had developed a set of OGC services with which to access the data and would like NSIDC to continue supporting these services. Given the sizeable user community the PI has been supporting NSIDC agrees to do so. However, the data set is complex, not all of which NSIDC has experience with, and the PI does not like writing documentation.
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

16. NSIDC Levels of Service – High Effort Example
Service Categories and Desired States
Score x Factors
Archival State - NSIDC ogoing ingest = Affecting factors: Need to develop ingest script =
15 x3
Archive Total: 45
Metadata State - Catalog + Inventory Plus = Affecting factors: None
20 x 1
Metadata Total: 20
Documentation State - Advertised + Full documentation Affecting factors: Limited PI Input Complicated Data Set
20 x 5 x 2
Documentation Total:
200
Distribution State - FTP +NSIDC advanced services Affecting factors: Automated pre-distribution code needed
15 x 3
Distribution Total:
User Services Infrastructure Preparation 5
USO Infrastructure Prep Total:
User Services State - NSIDC Technical Support Affecting factors: Daily user support expected Complex data set Multiple access paths
20 x3 x2 x1.5
USO Services Total:
180
Grand Total Score: 495
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

17. DAAC Data Acceptance Plan – Status and Plans
Roughly a half dozen datasets have been put through the process and the queue is long and getting longer
User working group attitudes:
Generally very supportive
Rather concerned that they will be inundated with requests
NSIDC may propose a modification to the process to more expeditiously handle small, low LoS requests without going through the UWG and NASA HQ steps
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

18. DAAC Data “De-commissioning” Plan – Status and Plans
Goal – To provide the NSIDC DAAC a mechanism to lower the LoS for a data set.
Graduate student from UIUC’s GSLIS data curation program interned at NSIDC last summer
Reviewed all decommissioning/deaccessioning/etc. plans and wrote a report summarizing her findings
Drafted a plan parallel to the DAAC Data Acceptance Plan
Plan has been internally reviewed by the DAAC manager and is awaiting updates for broader internal review
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

19. DAAC Data “De-commissioning” Plan – Overview
Any triggering event can start the process. Examples:
Systems obsolescence
Retiring of key personnel
Loss of funding support or termination of contractual requirement
Periodic review
Product team reviews existing materials, fills any gaps, and generates a recommendation (including potentially new LoS
Programmatic considerations
Scientific assessment (e.g., Uniqueness, literature references, etc.)
Data management considerations (e.g., Data file formats, hardware and software requirements)
DAAC UWG assesses DAAC materials and provides recommendation to NASA HQ
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC

20. DAAC Data “De-commissioning” Plan – And then what?
If data is to be moved to the lowest LoS possible, what should that LoS be?
If data is really to be decommissioned what processes for community notification, etc. need to be followed?
If data is to move to an “archive of last resort” what needs to accompany it and how should that be done?
What records must NSIDC keep?
DAAC Data Acceptance Plan and “De-commissioning” Plan
R. Duerr, 2010 winter ESIP Federation Meeting, Washington DC