Training course on developing and using questionnaires for agricultural surveys Field Testing Post evaluation methods Marco Ballin Istanbul, 18-21 July 2011

Post evaluation methods These methods look to evaluate indirectly the quality of the questionnaire during or after the real data collection Under this item cab be classified most of the methods used to evaluate the quality of the surveys Most of times the results can be used to improve next editions of the same survey Istanbul, 18-21 July 2011 2

Post evaluation methods Main post-evaluation methods are the following: Analysis of item non response data Analysis of response distributions Analysis of editing and imputation phase Reinterview studies Istanbul, 18-21 July 2011 3

Analysis of item non response data Post evaluation methods Analysis of item non response data Item non response occurs when a respondent provides some, but not all, of the requested information The analysis should start by investigating the extend of item non response Questions with higher non response have tove deeply analysed. It could be important to identify the position of the question in the questionnaire or the profile of respondent Istanbul, 18-21 July 2011 4

Analysis of response distribution Post evaluation methods Analysis of response distribution This analysis can help in finding out problems of the categories used for closed questions, outliers. It helps to compare different version of the same question Istanbul, 18-21 July 2011 5

Analysis of editing and imputation phase Post evaluation methods Analysis of editing and imputation phase This analysis can help in finding out the question where inconsistencies or invalid data can be found Numerous edit failures may indicate problems with wording, definitions, instructions, etc. Istanbul, 18-21 July 2011 6

Reinterview studies Post evaluation methods These surveys are mainly aimed at estimating the measurement error arising from the method of data collection, the respondent or the questionnaire There are two main approaches: To carry out the new interview in the same conditions of the first one (mainly used to estimate the simple response variance) To carry out a more accurate interview on a sub sample of respondent to the survey (mainly used to estimate the systematic component of the response error) Istanbul, 18-21 July 2011 7

Post evaluation methods Discussion ….. Istanbul, 18-21 July 2011 8