1. Reporting Results for Pesticide Programs Robin Powell, EM Pyramid Lake Paiute Tribe Environmental Department

2. Background General Performance Reporting Act (GPRA) Requirements General Performance Reporting Act (GPRA) Requirements Program Assessment Rating Tool (PART) Program Assessment Rating Tool (PART) General Accountability Office (GAO)-investigative arm of Congress charged with examining matters relating to the receipt and payment of public funds. General Accountability Office (GAO)-investigative arm of Congress charged with examining matters relating to the receipt and payment of public funds. Office of Management and Budget (OMB) Office of Management and Budget (OMB)

3. Passed in 1993 Passed in 1993 Phased implementation Phased implementation Focusing on ALL Federal programs Focusing on ALL Federal programs Goals & Purpose: Goals & Purpose: Improve program efficiency and effectiveness by focusing on results Improve program efficiency and effectiveness by focusing on results Holding federal agencies accountable and improving the confidence of the American people Holding federal agencies accountable and improving the confidence of the American people Improve Congressional rulemaking with accurate information Improve Congressional rulemaking with accurate information GPRA

4. GPRA As of 1997, all programs are required to submit an annual strategic plan, annual report with results, and a performance report to OMB As of 1997, all programs are required to submit an annual strategic plan, annual report with results, and a performance report to OMB I.e., EPA Strategic Plan and Goals I.e., EPA Strategic Plan and Goals Establish performance indicators Establish performance indicators Verification and validation of results Verification and validation of results

5. Outcome Measure “An assessment of the results of a program activity compared to its intended purpose”

6. Output Measure “The tabulation, calculation, or recording of activity or effort and can be expressed in a quantitative or qualitative manner”

7. Performance Goal “A performance goal means a target level of performance expressed as a tangible, measurable objective, against which actual achievement can be compared, including a goal expressed as a quantitative standard, value, or rate”

8. Performance Measure “A way to measure the activities of a program in relation to the performance goal set forth by the program.”

9. Performance Indicator “A performance indicator means a particular value or characteristic used to measure output or outcome”

10. PART “A systematic, consistent process for developing program performance ratings and then using that information to make budget decisions”

11. PART According to Bill Diamond/EPA: According to Bill Diamond/EPA: PART was “developed to assess and improve program performance so that the Federal government can achieve better results.” In 2004, Pesticide Field Programs (ES; Worker Protection; WQ; SAI) were assessed and received a score of 36 (results not demonstrated) Poor score was largely attributable to lack of performance outcome measures and quantifiable data on contribution of field program activities to strategic agency goals.

12. Reporting Results: Why? Will be necessary to remain “aggressive” in budget allocation process Will be necessary to remain “aggressive” in budget allocation process Regionally and nationally Regionally and nationally Maintain funding levels Maintain funding levels Improve ability to defend the need for additional funding Improve ability to defend the need for additional funding

13. Reporting Results: How? No single output or outcome can measure an entire program No single output or outcome can measure an entire program Integrated approach Integrated approach All outputs and outcomes All outputs and outcomes Quantitative/Qualitative Quantitative/Qualitative Statistical Statistical Narrative Narrative Aggregated and disaggregated Aggregated and disaggregated

14. Reporting Results: What? Outputs Outputs Any numerical data & statistical analyses Any numerical data & statistical analyses Outcomes Outcomes Did the X # of inspections prevent X# of unlicensed applicators applying RUPs? Did the X # of inspections prevent X# of unlicensed applicators applying RUPs? Narrative Narrative Qualitative and Quantitative Qualitative and Quantitative

15. Pesticides: General Can be data rich or deficient depending on program Can be data rich or deficient depending on program Assessment results Assessment results Number of spray events Number of spray events Comparison of applied GUPs & RUPs Comparison of applied GUPs & RUPs Quantity of pesticides used on tribal lands Quantity of pesticides used on tribal lands Analyses of any gathered sampling data Analyses of any gathered sampling data NOT raw data NOT raw data

16. Pesticides: C&T Number of C&T events Number of C&T events Number of attendees at each event Number of attendees at each event Comparison of C&T attendees Comparison of C&T attendees Pass vs. Fail Pass vs. Fail

17. Pesticides: Enforcement Number of spray violations Number of spray violations Compliance Assistance activities (limited) Compliance Assistance activities (limited) Number of inspections performed vs. number of violations Number of inspections performed vs. number of violations Number of licensed off-reservation applicators who apply on tribal lands Number of licensed off-reservation applicators who apply on tribal lands Number of cases initiated Number of cases initiated

18. Pesticides: WPS Number of violation events Number of violation events Number of citations Number of citations Comparison of violation types Comparison of violation types Number of inspections Number of inspections

19. Pesticides: Limitations Difficult to measure compliance Difficult to measure compliance Very limited outputs and performance indicators Very limited outputs and performance indicators Very difficult to assess the progress in achieving environmental goals Very difficult to assess the progress in achieving environmental goals Environmental Educational indicators Environmental Educational indicators Difficulty in assessing the compliance assistance outcomes Difficulty in assessing the compliance assistance outcomes Goal of EPA Goal of EPA How to measure the pollution reduction through these activities How to measure the pollution reduction through these activities

20. Pesticides: Possibilities Can improve strategic planning Can improve strategic planning Forecast needs Forecast needs Identify needs and goals Identify needs and goals Budgetary process Budgetary process Defensible requests for additional funding Defensible requests for additional funding Provides guidelines on funding needs Provides guidelines on funding needs Forecasting future funding requirements to maintain program integrity Forecasting future funding requirements to maintain program integrity

21. Reporting Results Example: Pesticides

22. Overall Report Example: Pesticides

23. Other Media: Grants Management Demonstrate program capacity building Demonstrate program capacity building Demonstrates the need for additional support for programs that are deficit in funding Demonstrates the need for additional support for programs that are deficit in funding Enables the program and EPA to identify tribal needs for additional programs Enables the program and EPA to identify tribal needs for additional programs Can forecast needs Can forecast needs

24. Reporting Results Example: Grants Management

25. Overall Report Example: Grants Mgmt

26. Not Recommended It is not needed to include full documents or reports with the results or progress report It is not needed to include full documents or reports with the results or progress report Do not have more than 3 pages of narrative per project or program Do not have more than 3 pages of narrative per project or program It is not recommended to include ordinances or codes within the report. It is not recommended to include ordinances or codes within the report. Reference within the report the documents Reference within the report the documents You can have a good results report while keeping it simple!!!

27. Conclusion It has been stated repeatedly that if programs are not producing verifiable “results” through illustration of outputs and/or outcomes, those programs will NOT be funded in the future

28. Thank You. Robin Powell, Environmental Manager Environmental Department Pyramid Lake Paiute Tribe rpowell@plpt.nsn.us 775-574-0101 x12