Survey Training Pack Session 19 – Interpretation of Findings

Introduction Data interpretation is the process of assigning meaning to the collected information and determining the conclusions, significance, and implications of the findings Data interpretation goes beyond describing the results presented in the output table but rather draws the link between what the table says and the indicator you are trying to measure

Output table interpretation Reflect on the structure of the output table and your core indicators – The structure is the product of the specific indicators (e.g. disaggregation by sex of rice trainee) Focus on key points – large differences and/or striking similarities depending your core indicators If the data is disaggregated, it means there exists an interest in comparing subcategories – Comment on differences and/or similarities – What key points emerge from Table 15 (page 21)?

Output table interpretation Use a programmatic lens to identify key points For a simple table, you should probably aim for no more than 2 bullet points For a complex table, no more than 5 bullet points Individual exercise: What could you say about Table 13 (page 20)? What key points emerge from this table?

Common mistakes Not recognising the importance of confidence intervals when developing your bullet points as you compare groups – Table 13 – amongst rice trainees with IRM and non-IRM plots, 61.6% have applied 4 core practices. The 95% CI is 22.2% to 90.1%. What is misleading about the above statement? Always keep in mind bias – do not imply that the data are more reliable than they are in reality – Type I – sampling bias – Type II – non-sampling bias Association does not mean causation!

Example of a mistake Imagine that you draw a sample of 1,000 citizens in Geneva, of whom 20 have reported being the victim of a crime. You are interested in knowing if citizens who are crime victims are as satisfied as the rest of the citizens vis- à-vis police services. % very satisfied/ satisfied 95% confidence intervals Victim of a crime (n=20) 50%50% ± 22.5% Not victim of a crime (n=980) 60%60% ± 3.1% IS THERE A DIFFERENCE?

Practical examples From the rice analysis, pick 6 output tables. For each table, develop 1-2 key points by remembering the following: What is the denominator? What is the point estimate? Are there 95% CI? Is the data disaggregated? If so, is it meaningful? To what core indicator does that table respond? How would you convey these key points to programme staff?

What helps? Effective output monitoring is key to being able to make sense of the data? – Can you think of an example where reliable output data would have made survey data interpretation easier? Sufficient programmatic and contextual knowledge Exchanges and discussions with relevant programme staff A good understanding of the basic principles of statistics (e.g. what is a mean? what are its limitations?) A very good understanding of the sampling process Clear study objectives/core indicators

What hinders? Lack of any of the aforementioned points Poor quality of the information (i.e. bias): – Sampling errors: overrepresentation of a certain group – Non-sampling errors: poor supervision of data collection, enumerators fabricating information, etc. Limited statistical inferential capacity: – Small sample size which yield poor precision levels Data which is not linked to your study objectives/core indicators – a case where the data analysis plan was developed after the questionnaire

Summary Data interpretation extends beyond describing the table Only meaningful key points need to emerge when interpreting the output table – No more than 2 key points for a simple table – No more than 5 key points for a complicated table Remember that all data is bias – what is important is to understand how much bias affects the quality of your data Do not imply that your data is more reliable than it is in reality