How to Achieve High Quality Data? Obtaining comparative results in multi-country surveys: is the devil in the details? A round table discussion Petra Holeckova, November 28 2017

What is behind the results Example of PIAAC survey

PIAAC Programme for International Assessment of Adult Competencies Project organized on international level by OECD „The survey measures adults’ proficiency in key information-processing skills: literacy, numeracy and problem solving in technology-rich environments – and gathers information and data on how adults use their skills at home, at work and in the wider community.“ (PIAAC website) Target group: 16-65 years old Length of interview: 120 minutes 6102 respondents in the Czech Republic Data Collection period: August 2011–April 2012 PIAAC Achieved education and work history Skills / competencies used at work and at home Series of tests Background questionnaire: possibility to compare achieved education of parents and children as well as their proffesion

Schedule – PIAAC (1st round) (Programme for International Assessment of Adult Competencies) Conceptual Framework Pilot – module Labor Market Requirements Preparation of the research instruments and PC platform Data Collection Agency Involved Data Collection – Pilot Analysis of Pilot s. Preparation for Main Survey Main Survey Analyses of the Main Survey International Report

PIAAC Technical Standards and Guidelines 192 pages Covered all issues: Qualification of the team Sample Design standards Survey instrument standards Translation and adaptation standards Data Collection staff training standards Data Collection standards Weighting standards Estimation standards National teams met several times and shared the „Sharepoint site“ All steps were closely monitored by Consortium of survey institutions

Conclusions Key points Key limits Professional and experienced team Well-translated and comprehensive questionnaire Representative sample High response rate in all sub-groups Weights People do not behave as in ideal model world Lack of time Lack of money