1. Learning from Practice: Continuous Improvement and Evaluation A-011B Candice Bocala, Ed.D. Spring 2015 Module 1

2. What is “improvement”? [2 min] Think about a time when you or a group you were a part of tried to improve something... This can be a practice, a routine, a product, or an outcome. [5 min] Share your experience with your neighbor. Switch after 2 minutes. [5 min] What common characteristics did each story have? 2

3. Two foundations for the course Improvement science Emphasis on thinking about systems and cycles Program evaluation Emphasis on formative (as opposed to summative) evaluation 3

4. Evaluation is the systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards as a means of contributing to the improvement of the program or policy. Carol Weiss (1998) 4

5. Course essential questions How can program theory be used in evaluation and improvement? What are the relationships among practitioners, researchers, and evaluators in various models of continuous improvement? How can improvement science and evaluation practices be used to build organizational capacity? 5

6. 6-week sequence Week 6: Other Models of Improvement Week 5: Evaluation Capacity Building & Supporting Continuous Improvement Week 4: Participatory & Empowerment Evaluation Week 3: Utilization-focused & Developmental Evaluation Week 2: Theory-based Evaluation & Logic Models Week 1: Introduction to improvement science 6

7. Assignments Task: Develop formative evaluation proposal Key skills & capacities: Communicate with program stakeholders Diagram program theory / logic modeling Write evaluation questions Align data collection with evaluation questions Consider pros / cons to different evaluation approaches Task: Reflect on your learning Key skills & capacities Demonstrate understanding and integration of course content Synthesize learning for your future work 7

8. Learning from practitioners Suggestions for evaluation use Approach to evaluation Data collection ideas Program theory / logic model Program description, key stakeholders, evaluation questions 8

9. A-011B partner organizations 9

10. Class pedagogy Class discussion of weekly study questions Rubrics Formative feedback and revision Workshop model using your own work 10

11. Logistics Grading: Attendance and class participation: 25% Progress on components of evaluation proposal: 25% Completed evaluation proposal: 25% Final reflective memo: 25% Can take class for a grade or pass/fail Final assignments are due March 13, 2015(1 week after class ends) No need for instructor permission to enroll No enrollment limit on the course 11

12. HOPE TO SEE YOU IN CLASS! Candice Bocala ceb193@mail.harvard.edu Office: Gutman 431 12