Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Collection Tools and Methods How and when to use the tools

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Monitoring & Evaluation Processes Mid-term Evaluation Completion Evaluation Outcome Evaluation Impact Evaluation Project Start Design Summary Indicators & Targets Monitoring Mechanism Assumptions & Risks Data Gathering Survey or Observation Analyze Data Recommendation or Conclusion Prepare Report Design and Monitoring Framework

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Interviews Can be structures, semi- structures, or unstructured Involve asking specific questions aimed at getting information Can be open- ended or closed Can be a source of qualitative and quantitative information Anyone can use Can be done face-to-face or via telephone or Very flexible Requires some skill in interviewing

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Key Informant Interviews Carried out by specialists in the topic Very objective as most of the informants are not affiliated with the project Useful insights coming from Subject Matter Experts Need a skilled interviewer with a good understanding of the topic Information may be difficult to challenge being said by an expert

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Questionnaires Set of written questions used to get written responses Can save lots of time in gathering data from several respondents Give the respondents a feeling of anonymity Not effective for people who cannot read or write Cannot explore other information People get tired answering questionnaires Questions must be tested to ensure that it is understood

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Focus Groups A group of about 6 to 12 people are interviewed together by a skilled interviewer focused around a specific topic or issue Very useful way of getting opinions from quite a large sample of people Sometimes people influence one another Discussion need to be transcribed and may be time consuming

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Community Meetings Involves gathering a fairly large group of beneficiaries to whom questions, problems, situations are put for input to help in measuring indicators Useful for getting a broad response from many people on specific issues Involves beneficiaries directly in an evaluation process Difficult to facilitate Requires a very experienced facilitator

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Fieldworker Reports Structured report forms that ensure indicator- related questions are asked and answered Flexible, an extension of normal work, cheap and not time-consuming Relies on field workers being disciplined and insightful

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Ranking Involves getting people to say what they think is most useful, most important, least useful, etc. Can be used with individuals and groups as part of an interview or questionnaire Quite difficult concept to get across and requires very careful explanation to ensure that people understand what the are asking

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Visual/Audio Stimuli Includes pictures, movies, and role plays to illustrate problems or issues, past events or even future events Very useful when used with other tools, particularly with people who cannot read or write Need to have appropriate stimuli and the facilitator needs to be skilled in using such stimuli

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Rating Scales Make use of a continuum, along which people are expected to place their own feelings, observations, etc. Useful to measure attitudes, opinions, perceptions Need to test the statements very carefully to make sure that there is no possibility for misunderstand- ing

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Critical Event/Incident Analysis A way of focusing interviews with individuals or groups on particular events or incidents Used to get a full picture of what actually happened Very useful when something problematic has occurred and people feel strongly about it Evaluation team can end up with vast amount of contradictory information

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Participant Observation Involves direct observation of events, processes, relationships, and behaviors Useful way of confirming information Time consuming, sometime difficult to observe

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP Data Gathering Tools ToolDescriptionUsefulnessDisadvantages Self-drawings Involves getting participants to draw pictures, usually of how they feel or think about something Very useful particularly with younger children Can be difficult to explain or interpret

Monitoring and Evaluation in the GMS Learning Program 7 – 18 May 2012, Mekong Institute, Khon Kaen, Thailand Randy S. Balaoro, CE, MM, PMP The End