1. Program evaluation Sebastian Uijtdehaage bas@mednet.ucla.edu 310.794.9009

2. Why do you evaluate?

3. Five reasons of evaluating 1.FORMATIVE informs program development 2.SUMMATIVE demonstrate program’s success and impact 3.UNDERSTANDING how and why did program work? 4.DISSEMINATION sharing best practices 5.RECOGNITION scholarship, academic credit, leadership Stufflebeam 2003

4. What type of evaluation should I conduct?

5. Outcome & Impact Did it work?

6. Dan Stufflebeam: CIPP Model of Program Evaluation Needs Assessment What should we do? Program Planning How shall we do it? Implementation Are we doing as planned? Outcome & Impact Does it work? INPUT CONTEXT PROCESS PRODUCT

7. What type of evaluation should I conduct? Formative or Summative?

8. formative & summative evaluation Formative Evaluation ContextWhat needs to be done? InputHow should it be done? ProcessIs it being done? ProductIs it working? Stufflebeam 2003

9. formative & summative evaluation Formative EvaluationSummative Evaluation ContextWhat needs to be done? Were important needs addressed? InputHow should it be done? Was program plan defensible? ProcessIs it being done?Was the plan well executed? ProductIs it working?Did the effort make a difference? Stufflebeam 2003

10. What type of evaluation should I conduct? Quantitative or Qualitative?

12. Quantitative or qualitative measures? Quantitative measures HOW MUCH…? How much did the participant change? How much did the community change? Qualitative measures WHY and HOW? Why did the program work? How did participants change as result?

13. What makes good evaluation?

14. Evaluation must be planned ahead Evaluation should be part of program planning Planning and resource allocation makes evaluation more successful – Example: long-term tracking of participants

15. Confidence being admitted in a medical school

16. What else makes good evaluation? Scientific rigor Evaluation produces useful information that helps make decisions – Assessment should be done with a particular purpose in mind Evaluation is tailored to your program objectives Evaluation is done with conceptual framework in mind

17. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact The Planned Work The Intended Results CONDITIONS and DRIVERS

18. LOGIC MODEL CONDITIONS and DRIVERS Current situation that precipitates the need for the program

19. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS Resources and inputs needed to operate the program: personnel, facilities, materials, equipment

20. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS Everything the program does if it had access to the resources

21. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS The “deliverables”—the events, products and services Number of participants, classes, brochures, advisors, etc The “deliverables”—the events, products and services Number of participants, classes, brochures, advisors, etc

22. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS If the program delivers, the participants will benefit in certain ways: Knowledge, skills, behavior, attitudes Immediate, mid-term and long-term outcomes If the program delivers, the participants will benefit in certain ways: Knowledge, skills, behavior, attitudes Immediate, mid-term and long-term outcomes

23. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact CONDITIONS and DRIVERS If these benefits to participants are achieved, then certain changes in organizations and communities might be expected to occur

24. LOGIC MODEL ResourcesActivitiesOutputOutcomesImpact The Planned Work The Intended Results CONDITIONS and DRIVERS

25. LOGIC MODEL Links processes and outcomes: – Funders love this! Great for program planning – Work backwards Evaluation is built in! Ain’t perfect

26. Your turn…