1. Building Evaluation Capacity for Prescription Opioid Overdose Prevention: Lessons From a Multi-state Initiative
Session Chair: Sarah Bacon, PhD
Presenters: Kari Cruz, MPH, Bradley Biggers, MPH,
Emilia Pasalic, MPH, and Aleta Christensen, MPH
Presentation to the American Evaluation Association
Annual Conference
October 29, 2016

2. 500,000 4x as many opioid deaths drug overdose deaths since 2000
in 2014 as 1999

3. For every Rx opioid overdose death in 2011,
there were...
SAMHSA NSDUH, DAWN, TEDS data sets.

5. Overdose Prevention In States (OPIS)
Three competitive cooperative agreements, all to state health departments or their bona fide agents
Prevention for States
Data Driven Prevention Initiative
Enhanced Surveillance for Opioid Morbidity and Mortality

6. 1 2 4 3 Prevention for States Program Enhance and Maximize PDMPs
Move toward universal PDMP registration and use
Make PDMPs easier to use and access
Move toward a real-time PDMP
Expand and improve proactive reporting
Conduct public health surveillance with PDMP
Enhance and Maximize PDMPs
Community or Health System Interventions
State Policy Evaluation
Rapid Response Projects
Implement or improve opioid prescribing interventions for insurers, health systems, or pharmacy benefit managers. This includes:
Prior authorization, prescribing rules, academic detailing, CCPs, PRRs,
Enhance adoption of opioid prescribing guidelines 1 2
Prevention for States Program 4 3
Allow states to move on quick, flexible projects to respond to changing circumstances on the ground and move fast to capitalize on new prevention opportunities.
Build evidence base for policy prevention strategies that work like pain clinic laws and regulations, or naloxone access laws

7. Evaluation Capacity Building: Tools for State Partners
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

8. Evaluation Officers

9. Current Funded States IL TX NY IA CA SD ND NE MN MI ID MS KS IN LA WY NM AL WI NC OK GA VA CO KY HI MT UT SC OH OR TN PA WV WA AZ ME FL MO NV AK NH VT MA RI CT NJ DE MD DC AR
Funded State: Prevention for States (PfS)
Funded State: Data-Driven Prevention Initiative (DDPI)

10. CDC State Support Teams
Program Consultant
Evaluation Officer
Science Officer

11. State Prescription Drug Overdose Teams
Program Manager
Evaluator
Epidemiologist

12. Technical Assistance
Tailored assistance that is purposeful, responsive, science-based, and systematic in building their infrastructure and capacity around implementation and evaluation to improve outcomes associated with opioid misuse, abuse and overdose prevention programming. 

13. Evaluation Tools
Monitoring and Evaluation Tool (MET) Evaluation plans Indicators

14. Capacity Building: Monitoring and Evaluation Tool (MET)
National Center for Injury Prevention and Control
Division of Analysis, Research, and Practice Integration

15. Monitoring and Evaluation Tool (MET) Background
Microsoft Access Database
Started in 2011
Developed primarily by Deloitte
Supports Prevention for States (PfS) and three other CDC cooperative agreements

17. Improve responsiveness of CDC staff in support of PfS awardees
Identify capacity of state prescription drug overdose programs
Identify emerging needs and gaps so CDC can deliver timely technical assistance

18. Capacity Building: Developing Evaluation Plans with State Partners
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

19. Evaluation Plan Development Process
Template developed
States develop evaluation plans
Evaluation Plans submitted by state to CDC
Review criteria
Review worksheets
Gems!
Evaluation Plan reviewed by CDC Evaluation Officer
Feedback form
Formal discussion TA
COP
CDC provides feedback to state
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

20. Evaluation Plan Template
Template sections:
Project specific logic model
Detailed evaluation plan for project years one and two
Narrative description of evaluation plan for years three and four

21. Evaluation Plan Template Development
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

22. Evaluation Plan Review Processes
Common criteria worksheets
Logic model review worksheet
Evaluation Plan review worksheet
Evaluation Plan Narrative review worksheet
Gathering resources: Evaluation Gems!
Evaluation Plan Feedback Template
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

23. Logic Model Review Worksheet
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

24. Evaluation Plan review worksheet

25. Evaluation Plan Narrative Review Worksheet

26. Evaluation Gems!
An internal tracking spreadsheet prompts Evaluation Officers to document:
Details of exemplary evaluation plans
Resources received from the state
Additional Comments

27. Evaluation Plan Feedback Template

28. Evaluation Profiles

29. Lessons Learned
Both reviewers and reviewees appreciate clear criteria!
Limit the amount of mandatory forms
Systematic approach = Proactive technical assistance

30. Capacity Building: Indicator Development
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention

31. Evaluation Capacity and Data Use
Data Sources:
Vital statistics death data
Emergency department data
Hospital discharge data
Prescription drug monitoring program (PDMP) data

32. Developing the Indicators – Challenges
Definition
How to calculate (numerator & denominator)
Limitations

33. Prevention for States Indicator Support Toolkit
Contents:
Background, Introduction, and Purpose
Indicator Considerations
Preparing the datasets
Indicator quick guide
One-pagers for each indicator
Sample SAS code

34. Developing the Indicators – Engaging Stakeholders

35. Indicator and Toolkit Pilot
2 piloting phases
Funded state partners
ASTHO
Pilot though and webinars

36. Progress and Next Steps

37. 1600 Clifton Road NE, Atlanta, GA 30333
For more information please contact Centers for Disease Control and Prevention
1600 Clifton Road NE, Atlanta, GA
Telephone: CDC-INFO ( )/TTY:
Web:
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.