Copyright © 2014 by The University of Kansas Choosing Questions and Planning the Evaluation

Copyright © 2014 by The University of Kansas What do we mean by choosing questions? Evaluation questions are the questions your evaluation is meant to answer about your work. Evaluation questions help set the direction of the work, as well as assess its effectiveness. Ideally, choosing evaluation questions is part of the planning of the overall program.

Copyright © 2014 by The University of Kansas Questions to ask yourself as you choose: What do you want to know? Why are you interested? Is the issue you’re addressing important to the community or to society? How does the issue relate to the field? Is the issue general, rather than specific to your population or community? Who might use the results of your evaluation? Whose issue is it?

Copyright © 2014 by The University of Kansas Why is it necessary to choose evaluation questions carefully? It helps you understand what effects different parts of your effort are having. It helps you clearly define what it is you’re trying to do. It shows you where you need to make changes. It highlights unintended consequences. It guides your future choices. In participant evaluations, it involves stakeholders in setting the course of the program, thus making it more likely that it meets community needs. It provides focus for the evaluation and the program. It determines what needs to be recorded in order to gather data for evaluation.

Copyright © 2014 by The University of Kansas When should you choose questions and plan the evaluation? If possible, choosing questions and planning the evaluation should be an integral part of planning your program. If your reality makes that impossible, choosing questions and planning the evaluation should take place as soon as possible after the program starts.

Copyright © 2014 by The University of Kansas Who should be involved in the process? To the extent possible, the process should involve all stakeholders, including program participants/beneficiaries.

Copyright © 2014 by The University of Kansas How do you choose questions and plan the evaluation? Choosing questions: Describe the issue or problem you’re addressing. Describe the importance of the problem. Describe those who contribute to the problem. Assess the importance and feasibility of changing those behaviors. Describe the change objective. Make sure that the expected changes would constitute a solution or substantial contribution to the problem.

Copyright © 2014 by The University of Kansas Planning: Take into account the issues raised by multiple or very different settings. Take into account the issues raised by participant groups that differ in culture, ability to complete the program, geographical location, and other factors. How do you choose questions and plan the evaluation?

Copyright © 2014 by The University of Kansas For outside evaluators, specifically: Choose a setting. Learn as much as you can about the organization you’ve chosen. Contact the appropriate person(s) and request an interview. Plan and prepare for the initial meeting. How do you choose questions and plan the evaluation?

Copyright © 2014 by The University of Kansas For all evaluators: Find out all you can about the context. Establish trust with program administrators, staff, and participants. Aim for a participatory evaluation. Plan the evaluation, in collaboration with stakeholders. How do you choose questions and plan the evaluation?

Copyright © 2014 by The University of Kansas Consider in your planning all the elements of an evaluation: Information gathering and synthesis. Designing an observational system. Developing and testing a prototype intervention. Selecting an appropriate experimental design. Collecting and analyzing data. Gathering and interpreting ethnographic information. Collecting and using archival data. Encouraging participation throughout the research. Refining the intervention based on the evaluation Preparing the evaluation results for dissemination. How do you choose questions and plan the evaluation?