Session 853 Extending Organizational Capacity and Capability to Evaluate Federal Environmental Research Programs Howard Cantor U.S. EPA Office of Research and Development October 29, Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Session 853 Extending Organizational Capacity and Capability to Evaluate Federal Environmental Research Programs Introduction Howard Cantor U.S. EPA Office of Research and Development October 29, Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Session Focus  Question: How can we develop an evaluation framework that helps organizational leaders to  Value and use evaluation results;  Build evaluation capability & capacity and  Communicate about performance, outcomes, and evaluation to our clients partners, and stakeholders

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Why Is this Important?  Growing emphasis on evaluation & accountability for federal research programs: –Canada, European Union, Australia, New Zealand, Japan, Korea, United States

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Why Is this Important?  Challenges to evaluating federal R & D programs and to measuring performance:  Measuring new knowledge as it develops  Managing the pace at which research progresses  Coping with an absence of accurate methods to objectively evaluate research quality and impact  Reaching agreement about key evaluation definitions  Communicating research contributions to outcomes in language that the public understands

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Why Is this Important?  Within the United States, EPA program managers and agency leaders must communicate to many stakeholders about accountability:  The Government Performance and Results Act  The President’s Management Agenda  The OMB Program Assessment Rating Tool (PART)  The Research and Development Investment Criteria  The Information Quality Act

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Specific Evaluation Objectives for Research Programs  Within the U.S., the Research & Development Investment Criteria focus specifically on research program accountability  Within the U.S., the Research & Development Investment Criteria (relevance, quality, performance) focus specifically on research program accountability R&D investments must be planned to be relevant to national priorities, agency missions, strategic goads & customer needs Programs must maximize the quality of the research they invest in R&D programs must demonstrate performance by setting annual and long-term goals and demonstrating progress toward outcomes   OMB and OSTP also encourage federal research managers to characterize the scientific leadership of their research programs

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Today’s Presentations Focus on: Dale Pahl 15 minutes Lori Kowalski 20 minutes David Schmeltz 30 minutes 20 minutes 1.Integrating program design, management, accountability, & evaluation: conceptual framework 2.Independent expert evaluation with a FACA panel: the Board of Scientific Counselors 3.Research contributions to outcomes: the acid rain cap & trade program Discussion

2005 Joint Conference: Crossing Borders, Crossing Boundaries Canada Evaluation Society & American Evaluation Association Thanks to our Session Sponsors !  Research, Technology, and Development Evaluation Group  co-chairs: Gretchen Jordan and George Teather  Environmental Evaluation Group  co-chairs: Kara Crohn, Katherine Dawes, Allison Titcomb