1. The Pew-MacArthur Results First Initiative: Improving Outcomes Through Evidence-Based Policymaking
August 4, 2014

2. The Policy Challenge
Though policymakers want to make strategic choices, the process often relies on inertia and anecdote
Limited data on:
What programs are funded
What each costs
What programs accomplish
How they compare
Solution: bring systematic evidence into the system

3. Evidence-Based Policymaking
Focuses on “What Works” – target funds to programs shown to be effective by rigorous research
Uses lists of ‘proven’ and ‘promising’ programs identified by clearinghouses
Outcome-oriented approach
Asks whether programs’ benefits justify their costs

4. The Results First Approach
Inventory programs and compare to evidence
Target funds to evidence-based programs
Conduct cost-benefit analysis to assess returns on investment
Goal: Achieve dramatic improvements without increased spending
The solution to the policy challenge is to bring evidence into the process. Fill the information gap.
--IDENTIFY where state funds are going – and what we know about each program we’re buying. In most of our states, the majority of the program budget is spent on programs that we know nothing about – we don’t know if they’re working; we don’t know what we’re buying with taxpayer money.
--TARGET funds using evidence. At the very least, move money away from programs that you know are ineffective (whether through your agencies’ evaluations or because the national literature tells us they don’t produce results) and toward programs that you know are effective. Barring more evidence, move money toward programs that you THINK are effective, but make sure the agencies have what they need to evaluate them to make sure. That means research staff or contracting with research entities, that means rigorous evaluation designs, time to collect and analyze needed data – it also means attention to quality control - ensuring programs are implemented well.

5. Inventory & Assess Programs
Identify currently funded programs
Compare to lists of evidence-based programs
Results First portal of programs assessed by 8 clearinghouses
Data on over 950 interventions, categorized by evidence of impact
January 13, 2019January 13, 2019

6. Assess Current Programs
Currently funded Programs
Evidence-based programs
This one slide will explain what it is and why we do it.
Allows to you answer three main questions/inform policymakers
What programs are being funded? Allows you to understand where your money is going.
Are they evidence-based? Establish a baseline of evidence based programs– what % of funding goes towards them now.
Are they effective?
Match programs in the RF model and conduct CBA on them.

7. Compare Programs to Evidence
ADULT PROGRAMS
FY14 BUDGET
FY14 BUDGET %
RATING
Moral Reconation Therapy
$120,000 2%
Evidence-based
Thinking for a Change
$65,000
1.0%
Education programs
$300,000 5%
Transcendental Meditation
$75,000 1%
Stayin’ Clean
$180,000 3%
Changing Course
$90,000
Promising
Adult Boot Camp
$250,000 4%
No effects
Domestic Violence program
$650,000
11%
Poor outcomes
Sober Living Environment
$375,000 6%
???
Others
$3,850,000
64%
Total Program Budget
$6,025,000
12%: Evidence-based
1.5%: Promising
15%: Negative
Determine which are programs for which there is evidence that it works – through Results First program categories included in the model software AND the clearinghouse database
70%: No Evidence

8. Conduct Cost-Benefit Analysis

9. Impact Analysis of Functional Family Therapy
Recidivism Rate
RECIDIVISM RATES REDUCED BY 22%
Without FFT (actual baseline)
With FFT
Follow-up Years
Source: Washington State Institute for Public Policy

10. Cost-Benefit Analysis of Functional Family Therapy
BENEFITS PER FAMILY
WA STATE DOLLARS
SOURCE OF BENEFITS
Reduced crime
$31,745
Lower state & victim costs
Increased high school graduation
$5,686
Increased earnings
Reduced health care costs
$307
Lower public costs
Total Benefits Per Family
$37,739
Cost Per Family
$3,190
Net Present Value
$34,549
Benefits Per Dollar of Cost
$11.86

11. Compare ROI Between Programs
POLICY/PROGRAM
COST
LONG-TERM BENEFITS
COST/BENEFIT RATIO
Intensive supervision (only)
$4,140
-$578
-$0.14
Mental health court
$2,935
$20,424
$6.96
Community drug treatment
$1,602
$17,711
$11.05
Correctional education in prison
$1,128
$21,426
$19.00
Work release
$661
$7,117
$10.77
Cognitive behavioral therapy
$412
$9,695
$23.55
Community job training & aid
$135
$5,501
$40.76
JUVENILE PROGRAMS
Functional Family Therapy
$3,262
$70,370
$11.86
Aggression replacement training
$1,508
$62,947
$41.75
Coordination of services
$395
$13.94
Scared Straight
$65
-$4,949
-$76.35
*Washington State 2012 dollars

12. Participation in Results FirstHI WA TN MT OR ID WY CO UT NV CA AZ NM NE KS OK TX ME ND SD MN LA AR MO IA WI MI IL IN OH PA NY WV KY MS AL GA SC NC FL VA AK VT NH MA RI CT NJ DE MD AK

13. Results First Can Drive System Improvement
Appropriations Investment Advice
Research Test New Programs
Implementation Ensure Fidelity
Need to discuss narrative for this slide.
Oversight Monitor Outcomes

14. Key Results First Outcomes 2013 & 2014 (to date)
9 States
Completed implementing the model and presented results to legislators and stakeholders
4 States
Enacted Legislation incorporating Results First into their policymaking process
2 States
Used models to analyze legislation
6 States: CT, IA, MA, NM, NY, and VT
3 States: CT, MA, VT
Impact
Models used to target $80 million in funding

15. Iowa
Determined Cognitive Behavioral Therapy had high ROI, expanding in prisons
Replacing ineffective domestic violence treatment program with new pilot program
Expanding analysis to other policy areas
DV Program: Were losing $3 for every $ spent on old program. Now working with the University of Iowa to pilot an alternative program known as Achieving Change Through Value-Based Behavior, also known as ACTV.

16. New Mexico Implemented in all available policy areas
Assessed state spending:
“Cost of Doing Nothing”
Impact of State Budget Cuts
Used Results First to target $57M to evidence based programs in early education and criminal justice

17. Results First Can Be Used For A Growing Number of Policy Areas
K-12 Education
Prevention Programs
Criminal Justice
Child Welfare
Mental Health
Substance Abuse
Early Education

18. Gary VanLandingham, DIRECTOR