1. 1 PERFORMANCE MEASURES: So WHY should I care? Remarks by Sherry Sterling Senior Advisor, OPP PREP Course 13 June 2005

2. 2 Critical Components of Program Success CLEAR GOALS Public Health & Ecological Protection Outcomes GOOD PLANNING Strategic Objectives EFFECTIVE IMPLEMENTATION Multiple Levels ADEQUATE RESOURCES Acquisition & Allocation MEANINGFUL ACCOUNTABILITY Measures & Tracking

3. 3 Accountability Drivers in EPA Statutes EPA 5 Year Strategic Plan 3 Year NPM Commitments Budget Formulation Annual Work Plan Performance Standards Budget Allocation Audits Tracking Systems/Reporting

4. 4 Government Performance and Review Act (GPRA) GPRA is a statute aimed at increasing accountability for all federal agencies by requiring measurement of program results & linking those results to resource allocation. Provides a statutory framework for strategic planning and reporting that requires federal agencies to: – Establish accountability measures to assess the outcomes of each program activity, and – Prepare performance plans that establish annual performance goals.

5. 5 GPRA: Auditing Function Office of Management & Budget (OMB) assumed the role of evaluating progress against GPRA, developing the Program Assessment Rating Tool (PART) OMB is committed to audit all federally funded programs against GPRA requirements on a 5 year cycle. This is the third year. Several Pesticide Programs have been audited.

6. 6 Principle focus has been on Outcome Measures. Short turn-around times for agencies to develop their data. Average scores have been very low; many classified as “Results Not Demonstrated”. “Re-PARTing” is the norm. Results are published each year with the federal budget. Programs not demonstrating results for successive years are subject to budget cuts or elimination. PART: Outcomes

7. 7 How do we measure? OUTPUTS An activity, effort, or work product related to a goal May be quantitative or qualitative Must be measurable during the funding period Examples: number of policies developed, number of REDs issued

8. 8 How do we measure? OUTCOMES The result, effect, or consequence that will occur from carrying out a program or activity related to a goal Must be quantitative May not be achievable during the funding period Examples: reduced occurrence of disease, fewer incidents

9. 9 Levels of Outcomes Short-Term – Most direct and measurable – 1 to 3 years Intermediate – Less measurable – 4 to 6 years Long-Term – Difficult to measure impacts – 7 to 10 years

10. 10 How do we measure? EFFICIENCY Ratio of outputs to cost Cost can be dollars or time Examples: cost per purchase order transaction, cost per seminar conducted

11. 11 Framework for Action in OPP Senior Level Steering Committee FOCUS: Set Direction; Manage Process; Resolve conflicts/issues Task Groups FOCUS: developing outcome and efficiency in specific areas ESA Water Quality PESPSAI Food Safety Worker Safety ??? Coordinating Committee FOCUS: developing outcome and efficiency measures for MISSION AREAS; coordinating work of task groups

12. 12 Process highlights… Inclusive – HQ, Regions, States, Tribes, Stakeholders Performance measures will serve as an important management tool Very tight timeframe

13. 13 Products expected from process… Protect Human Health Protect Environment Other Benefits from Use Worker Safety Yes Food/Agg. Exp. Yes ESA Yes Water Quality Yes SAI + PESP Yes

14. 14 What’s a Logic Model? A systematic and visual way to develop and present the causal links between the: – resources available to operate a program – activities conducted with those resources – products or services produced by the activities – intended changes or results

15. 15 Why the Logic Model? Simple to understand and use Links budget to goals Helps to identify what to measure Communication tool Endorsed by National Grants Office as a way to address EPA Order 5700.1 Endorsed by OMB (conference website)

16. 16 Logic Model Template ResourcesActivitiesOutputsOutcomes/ Results What do we need to operate this program well? What are we doing to achieve our intended results? How many, how often, over what duration? So what? What difference are we making?

17. 17 Criteria for Good Measures VALUE Meaningful Outcomes Beans Understandable Clear Complex Useful Sliced bread File 13 BURDEN # Measures Few Many Collecting Difficulty Simple Hard Data System Cost Low High Expertise required User-friendly Geeks

18. 18 Performance Measures Quadrant Analysis Value BurdenBurden OUTCOME NIRVANA OUTPUT HELL Low High Low

19. 19 Example of a good measure…. From OMB’s website Efficiency measure: Percent reduction in review time for registration of conventional pesticides

20. 20 Why is this a good measure? Properly assists the program in improving its administrative efficiency Important to stakeholders Reduction in time, taking into account various steps in the process, is a valuable method for improving efficiency and potentially cost effectiveness

21. 21 WHY do we care? Because… As public servants, we must assume responsibility We are always searching for the highest quality public service at the lowest cost We want to use whatever information is available to improve programs Remembering that we are the trustees for the public’s monies

22. 22 WHY do we care? Because… Meaningful Accountability = Good Management = Good Government