1. 1 Panel The Evaluator’s Role in Improving Accountability: Learning from Experience with Performance Indicators Presentation to the Canadian Evaluation Society/ Société Canadienne D’ Évaluation Vancouver, B.C., June 4, 2003, Session 302

2. 2 Presentations Improving Accountability by Understanding the Program and the Context Zoe Barley, McREL, Aurora, Colorado [zbarley@mcrel.org] Improving Accountability by Using Internal Quality Assurance and Peer Review Phyllis M. Thomas, McREL, Aurora, Colorado [pthomas@mcrel.org] Improving Accountability by Using External Audit and Verification of Data Kathleen Mooney, SERVE, Greensboro, North Carolina [kmooney@serve.org] Improving Accountability by Reporting to Multiple Audiences Ormond Hammond, PREL, Honolulu, Hawaii [hammondo@prel.org] Improving Accountability by Using Reflection and Self-assessment Ivana Zuliani, Lab at Brown University, Providence, Rhode Island [Ivana_Zuliani@brown.edu] (Presentations to be followed by audience participation and discussion.) Disclaimer: This document has been funded at least in part with federal funds from the U.S. Department of Education under contracts in the Regional Educational Laboratory Program. The content of this document does not necessarily reflect the views or policies of the Department of Education nor does mention of trade names, commercial products, or organizations imply endorsements by the U.S. Government.

3. 3 Session Agenda Purpose – To explore the evaluators’ role in improving accountability Holding ourselves accountable for quality, even when mandated Taking the high road/taking responsibility Not a top down view but middle up, down, & sideways – To share lessons learned from our indicators work using various strategies – To learn from the audience’s experience

4. 4 Session Agenda Format – Presentation of five strategies (use, benefits, limitations) – Audience participation Questions/comments Personal applications to evaluation work Overall lessons learned re: accountability

5. 5 Discussion Questions for CES Panel Audience Participation 1. Do you have any comments or questions for the panel? 2. Have you used any of the strategies presented today in your own work? a. If so, which ones? Please give an example. b. What have you found to be the benefits and limitations of the strategy ( i.e., with regard to enhancing or creating a culture of accountability)? 3. Have you used any other strategies for improving accountability or creating a culture of accountability? a. If so, which ones? Please give an example. b. What are the benefits and limitations of the strategy? 4. If you have not used any of the strategies discussed today: a. Which strategy, if any, seems like it could be useful in your own practice? b. In what ways might you apply this strategy to your work? Additional comments or questions for the panel or the audience?

6. 6 Lessons Learned from Our Experience 1. What accountability mechanisms are, or should be, in place to ensure that public expenditures are appropriately invested to achieve desired outcomes? 2. How does evaluation help to foster a culture of accountability within public organizations? 3. How can experience in other countries with various evaluation models inform our practices in improving accountability to the public? 4. How can we adapt evaluation practice to help empower the public to hold government accountable for program outcomes?