Dissemination to support Research & Analysis John Cornish

Global trends Users around the world are seeking access to more detailed data for research & analysis –More analytical/quantitative academic research –Evidence-based policy –More international comparisons and exchange of information amongst researchers, incl for monitoring MDGs More NSOs are finding ways of providing access to detailed data, including microdata –Better understanding of user needs –Improved usability & functionality of software interfaces –Improved database technology –Growth of Data Archives Trend towards ‘self help’ for most users 2003 Conference of European Statisticians agreed that more needed to be done to support research and analysis –But must not go beyond what is legal or acceptable to the public

Ways of supporting users doing research & analysis Have relevant data which is fit for use –Detailed aggregated data & Microdata Good documentation, metadata and archiving - users need to find what is available and all about it Access mechanisms Use standards, maintain consistency Partnerships Staff who understand researcher needs Involvement with professional organisations (eg population associations, economists)

Macrodata Data more detailed than in publications –Detailed tables –Datacubes –With or without tabulation software Detailed Indicators - ability to ‘drill down’ Thematic databases –Time series –Geographic/area based/international –industry

Indicators Web ideal for allowing users to drill down, draw graphs/maps More for assessing trends rather than reasons for trends

Microdata There are limits to analysis which can be done with macrodata Users want more flexibility with different cross-classifiers, different aggregations, etc Lots of models of analysis require working at the micro level Linking of datasets

International Guidelines Guidelines on “Managing Statistical Confidentiality and Microdata Access” by Trewin et al Contains many case studies on good practices Available on the UNSD website

Guideline Principles It is appropriate to use microdata for statistical research & analysis as long as confidentiality is protected Microdata should only be made available for statistical purposes Provision must be legal The procedures for researcher access to microdata, as well as the uses and users, should be transparent and publicly available

Confidentiality Usefulness & confidentiality risk are inversely related Usefulness increases with detail Confidentiality risk increases with detail Usefulness reduces with confidentiality adjustments (eg collapsing of detail) Confidentiality risk = f (sampling fraction + file detail + adjustments + management of access) also quality Greatest risk is with business records & families/households

Options for microdata access Release of confidentialised microdata files -direct to user or via data archive Remote access to more detailed confidentialised microdata files Microdata services - submit jobs, NSO runs & vets Data Laboratories Research collaborations

Managing the Risks Microdata Review Panel Provide a variety of access methods Legally enforceable undertaking is important Withdrawal of service Training of researchers before allowing access Audits of actual practices

Data Archives & Microdata Services International Census Microdata Service (IPUMS) provided by University of Minnesota SADA, Data First Key benefits to NSOs -Archiving and data recovery services -Harmonisation of variables to facilitate comparisons across countries and time -Allows access to international data sets -Documentation of microdata in English AASDA

Infrastructure issues Data warehouse/data management - consistency of outputs Metadata Standards - data and presentation Confidentiality knowledge & measures Interface tools Access facilities & vetting Archiving

Some success factors Good user engagement Sound data management Management strategies for risk - risk management, not risk avoidance Overall vision for website & a transition path Research & analysis done by NSO