DDI for the Uninitiated ACCOLEDS /DLI Training: December 2003 Ernie Boyko Statistics Canada Chuck Humphrey University of Alberta

Cataloguing Experiences How many have catalogued using MARC Dublin Core

Cataloguing Experiences Objectives of cataloguing Inventory control Location tool Access Distribution

Enter DDI Documentation in a standardized mark-up language o Data Documentation Initiative (DDI)

An Example American Public Opinion and U.S. Foreign Policy,

XML-DDI Benefits The display of data documentation through a variety of style sheets; Input for further processing, such as creating statistical package command files, conducting advanced searches, comparing variables across data files, driving data extraction engines, etc.

Data Documentation There is a need for comprehensive data documentation that allows easily o Finding variables By subject groupings By keywords, phrases or terms By response categories (value labels) Through linkages from the questionnaire

Data Documentation There is a need for comprehensive data documentation that allows easily o Tracing variables back to their origins To a question To a response category for a multiple response item To the variables from which it was computed for a derived variable.

Data Documentation There is a need for comprehensive data documentation that allows easily o Understanding the corrections that must be made because of the sampling methodology

Whats next? Lets assume we have compliant files … so whats next? What are the choices?

General Choices Feed your own system (input from a structured file) Look at systems using files directly Wait for SAS, SPSS, etc. to become XML enabled Wait and see

Projects Using DDI NESSTAR Health Canada -- DAIS SDA, Berkeley ICPSRs metadata University of Minnesota US Census Bureau Harvard Virtual Data Center

Global Access, Local Support Data Producers Data users NESSTAR Central Server

Data Observatory Workbench Text – Journal articles – User guides – Methodology instructions Tools – Finding and sorting – Browsing – Analysing – Publishing Data – Survey – Indicators – Administrative – Geographical People – – Conferences – Experts – Discussion lists Hyperlinks

Data Sharing - The NESSTAR Way (in 3 Steps) 1.Prepare your data using the Nesstar Publisher Microdata in SPSS, SAS, Stata, Statistica, ascii or other formats Table- or aggregated data in Excel, Ascii or other formats Documentation/metadata in various text-formats, including XML Data or metadata sitting in relational databases Import Import data and metadata from a variety of formats Cut and paste additional metadata from external sources Use templates to enforce structure and local best practice Organize your variables in groups and sub-groups Add local controlled vocabularies or thesauri Validate your data/metadata against the DDI and your local best practice Output DDI-instances and/or publish to a Nesstar server

2.Publish your data to a Nesstar server Publish over the Web or a local area network (LAN) Organize your data in folders and sub-folders Define the access conditions of your data Customize the user- interface to your data Publish Data Store Data Sharing - The NESSTAR Way (in 3 Steps) – (contd)

3. Share and explore your data through a variety of interfaces Nesstar Explorer – a feature rich data browser (Java application) Nesstar light – the standard web-browser interface to Nesstar resources and services Choose between a variety of customized interfaces Develop your own customized interface or integrate Nesstar services in an existing web- application Access Data Store

Demo URL:

Where do we go from here? Need to start producing files Need to create incentives for survey managers to create files Need to work cooperatively to convert legacy files

Whats ACCOLEDS role?