1. E. D. Bell

2.  Kapp & Anderson (2010) argue that the program evaluation process has eight steps that are fluid, dynamic, and interdependent, i.e., they are not linear – they influence and shape one another throughout the evaluation process.

3. 1. Identify the evaluation question 2. Conceptualize the evaluation question Conceptualize 3. Operationalize the evaluation question Operationalize 4. Pick and define the method 5. Select the sample 6. Collect the data 7. Analyze and manage the data 8. Utilize the information to improve program performance

4. Kapp & Anderson (2010) make this point The steps of the evaluation process need to be framed with an appreciation of two fundamental notions: (1). The use of information for service improvement is primary, and (2). The context for evaluation is an agency where evaluation must compete with many enterprises for attention. (p. 29)

5.  Given these notions, the agency needs to implement the following steps concurrently with the implementation of the program evaluation: 1. Define the program 2. Support users 3. Implement and report design of the evaluation (Kapp & Anderson, 2010)

6. “ The process of conducting research and the evaluation of programs and the outcomes associated with these activities have the potential to be helpful or harmful to individuals, groups, organizations, and the broader society” (Kapp & Anderson, 2010, p. 39).

7. So, if you were a participant in a study, what would you want to know? If you were a program director, how would you want the evaluators to conduct themselves? If the evaluation of your program found a number of faults, how would you want these to be reported? (Kapp & Anderson, 2010, p. 59)

8.  Guiding Principles for Evaluators Guiding Principles for Evaluators  Systematic inquiry  Competence  Integrity/Honesty  Respect for people  Responsibilities for general public welfare

9. “ The process of program evaluation does not take place in a social and cultural vacuum devoid of gender, race, ethnicity, and other consideration” (Kapp & Anderson, 2010, p. 107).

10. Kapp & Anderson (2010) describe four attributes: 1.Awareness and acceptance – that generates respectful treatment and no prejudgment of abilities, performance, and competency. 2.Self-awareness – that helps you understand how your culture shapes what you see and provides the ability to examine, control, and change assumptions, false beliefs, and stereotypes

11. 3. Dynamics of differences – historical and cultural differences influence how you see and define reality, e.g., see Emmett Till. This story that I saw in Jet magazine as a young person still influences how I see and understand reality. Click on the link “The Picture of Emmett Till”.Emmett Till 4. Knowledge of individual cultures – of the stakeholders who are involved with the program being evaluated.

12. 5. Adaption of skills – The evaluator must be able to determine if her or his methods are culturally appropriate and respectful of the persons and programs that are being evaluated.

13.  Review Kapp & Anderson (2010) pp. 111 – 131.

14. “ With most evaluation projects, one of the first things to address is the way the program is supposed to operate” ( Kapp & Anderson, 2010, p. 133).  Develop the logic model of how the program is supposed to work (Program Development and Evaluation, University of Wisconsin – Extension for resource tools)Program Development and Evaluation, University of Wisconsin – Extension

15.  Interview Stakeholders  Board of Directors  Clients  Community Partners  Funders  Management  Staff (Kapp & Anderson, 2010).

16.  Use Logic Model Questionnaire (Kapp & Anderson, 2010, p. 139)  Given the first draft of the logic model for the agency what areas of the model require additional information, i.e., what are the potential evaluation questions.  Select one of the areas as the evaluation question for your evaluation plan