Release of Microdata John Cornish

Global trends Users around the world are seeking access to more detailed data for research & analysis More international comparisons and exchange of information amongst researchers More NSOs are finding ways of providing access to detailed data, including microdata services Trend towards ‘self help’ for most users 2003 Conference of European Statisticians agreed that more needed to be done to support research and analysis But must not go beyond what is legal or acceptable to the public

International Guidelines Guidelines on “Managing Statistical Confidentiality and Microdata Access” by Trewin et al Contains many case studies on good practices Available on the UNSD website

Guideline Principles Appropriate to use microdata for research as long as confidentiality is protected Only use for statistical purposes Must be legal Transparency

Options for microdata access Release of confidentialised microdata files Remote access to more detailed files Microdata services Data Laboratories Research collaborations In addition to Special tabulations & Data cubes

Key concerns of NSOs Breaches of confidentiality may impact on reputation of NSO and public/respondent confidence Response rates Data quality may not stand up to scrutiny Legislative authority may not be clear Costs (documentation & confidentiality checking)

Key concerns of researchers NSOs are being unduly conservative Researchers can be trusted and have their own ethics There is too much bureaucracy involved with access Researchers are forced to conduct their own lesser quality surveys

How to manage the tension Risk management rather than risk aversion More clearly explain concerns of bureaucracy Shift some responsibility to researchers They should feel some pain for breaches

Australian arrangements Specified in legislation since 1983 Used widely - ~300 releases in last 12 months Confidentiality maintained through a mix of file modifications & access methods Microdata Review Panel makes independent assessment of confidentiality risk for the Statistician Users must sign legally enforceable undertaking Criticism comes from researchers (about limitations), not the public

Managing the Risks Microdata Review Panel Variety of access methods Legally enforceable undertaking is important Withdrawal of service is easiest to implement Training of researchers before allowing access Audits of actual practices

International Microdata Services International Census Microdata Service (IPUMS) provided by Uni of Minnesota Key benefits to NSOs Harmonisation of variables to facilitate comparisons across countries & time Allows access to international data sets Archiving & data recovery services Documentation of microdata in English https://international.ipums.org/international/