Metadata requirements for archiving structured data Alice Born Statistics Canada Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (9-11 April 2008)

IMDB business model – archive focus Survey Survey instance Datasets Data elements Questionnaire Survey design Applications/ Software Products (COR) Value domains Frame and sample Datafiles in archive repository

Outline Overview of the integrated metadatabase (IMDB) in the survey life cycle Archive and disposal processes Metadata requirements for archiving Registration

Operations Management Quality Assurance AnalysisDissemination CollectEditEstimateTabulatePublish Operational Data Registers Survey Data Administrative Data Data Warehouses Operational Data Stores IMDB in the survey life cycle Design Metadata IMDB Archive + disposal IMDB

Processes for archiving and disposal 8. Archiving and disposal 8.1 Manage archive repository define archival format, format data, load data, record event dates, links to metadata registry (IMDB) 8.2 Preserve data and associated metadata Identify data/metadata, record archive request, transfer data/metadata, etc. 8.3 Dispose data and associated metadata Identify data to dispose, remove from archive, etc.

Types of metadata for archiving structured data (1) 1.Administrative metadata Maintaining and keeping track of the archived datafile Link to an IMDB record – SDDS or unique datafile identifier 2.Structural metadata Name of the datafile File format – record layout Software Storage media, location

Types of metadata for archiving structured data (2) 3. Survey and definitional metadata Already in the IMDB Survey description Questionnaires, questions, response choice Methodology Measures of data quality Variables (ISO 11179) Additional documentation – interviewer guide, coding tool

Administrative layer Organization Contact Documentation Identification Time Frame Keyword Theme Survey Universe Frame Survey instance Instrument Question Data file Methodology Instrument design Sampling Data source Error detection Imputation Estimation Quality evaluation Disclosure control Revisions and seasonal adjustment Data accuracy Data Element Data Element Concept Object Class Property Formula Conceptual Domain Value Domain Stewardship Identification Classification Statistical Activity Administered items

Registration status Steward Regular Registrar Standards Division Registrar Incomplete Candidate Recorded Qualified Standard Preferred standard Registration Authority Responsible Owner (Content) (Completeness, accuracy, adherence to quality and terminological description standards) Retired Historical Submitter Archived Disposed

Corporate Memory: Data and metadata for archive and disposal phases IMDB Operational data Registers Survey Data Administrative Data Clean Master Files Public Use Microdata Files Metadata for archive and disposal Archive repository + Disposal