The Pew-MacArthur Results First Initiative: Improving Outcomes Through Evidence-Based Policymaking August 4, 2014

The Policy Challenge Though policymakers want to make strategic choices, the process often relies on inertia and anecdote Limited data on: What programs are funded What each costs What programs accomplish How they compare Solution: bring systematic evidence into the system

Evidence-Based Policymaking Focuses on “What Works” – target funds to programs shown to be effective by rigorous research Uses lists of ‘proven’ and ‘promising’ programs identified by clearinghouses Outcome-oriented approach Asks whether programs’ benefits justify their costs

The Results First Approach Inventory programs and compare to evidence Target funds to evidence-based programs Conduct cost-benefit analysis to assess returns on investment Goal: Achieve dramatic improvements without increased spending The solution to the policy challenge is to bring evidence into the process. Fill the information gap. --IDENTIFY where state funds are going – and what we know about each program we’re buying. In most of our states, the majority of the program budget is spent on programs that we know nothing about – we don’t know if they’re working; we don’t know what we’re buying with taxpayer money. --TARGET funds using evidence. At the very least, move money away from programs that you know are ineffective (whether through your agencies’ evaluations or because the national literature tells us they don’t produce results) and toward programs that you know are effective. Barring more evidence, move money toward programs that you THINK are effective, but make sure the agencies have what they need to evaluate them to make sure. That means research staff or contracting with research entities, that means rigorous evaluation designs, time to collect and analyze needed data – it also means attention to quality control - ensuring programs are implemented well.

Inventory & Assess Programs Identify currently funded programs Compare to lists of evidence-based programs Results First portal of programs assessed by 8 clearinghouses Data on over 950 interventions, categorized by evidence of impact January 13, 2019January 13, 2019

Assess Current Programs Currently funded Programs Evidence-based programs This one slide will explain what it is and why we do it. Allows to you answer three main questions/inform policymakers What programs are being funded? Allows you to understand where your money is going. Are they evidence-based? Establish a baseline of evidence based programs– what % of funding goes towards them now. Are they effective? Match programs in the RF model and conduct CBA on them.

Compare Programs to Evidence ADULT PROGRAMS FY14 BUDGET FY14 BUDGET % RATING Moral Reconation Therapy $120,000 2% Evidence-based Thinking for a Change $65,000 1.0% Education programs $300,000 5% Transcendental Meditation $75,000 1% Stayin’ Clean $180,000 3% Changing Course $90,000 Promising Adult Boot Camp $250,000 4% No effects Domestic Violence program $650,000 11% Poor outcomes Sober Living Environment $375,000 6% ??? Others $3,850,000 64% Total Program Budget $6,025,000 12%: Evidence-based 1.5%: Promising 15%: Negative Determine which are programs for which there is evidence that it works – through Results First program categories included in the model software AND the clearinghouse database 70%: No Evidence

Conduct Cost-Benefit Analysis

Impact Analysis of Functional Family Therapy Recidivism Rate RECIDIVISM RATES REDUCED BY 22% Without FFT (actual baseline) With FFT Follow-up Years Source: Washington State Institute for Public Policy

Cost-Benefit Analysis of Functional Family Therapy BENEFITS PER FAMILY WA STATE 2010 DOLLARS SOURCE OF BENEFITS Reduced crime $31,745 Lower state & victim costs Increased high school graduation $5,686 Increased earnings Reduced health care costs $307 Lower public costs Total Benefits Per Family $37,739 Cost Per Family $3,190 Net Present Value $34,549 Benefits Per Dollar of Cost $11.86

Compare ROI Between Programs POLICY/PROGRAM COST LONG-TERM BENEFITS COST/BENEFIT RATIO Intensive supervision (only) $4,140 -$578 -$0.14 Mental health court $2,935 $20,424 $6.96 Community drug treatment $1,602 $17,711 $11.05 Correctional education in prison $1,128 $21,426 $19.00 Work release $661 $7,117 $10.77 Cognitive behavioral therapy $412 $9,695 $23.55 Community job training & aid $135 $5,501 $40.76 JUVENILE PROGRAMS Functional Family Therapy $3,262 $70,370 $11.86 Aggression replacement training $1,508 $62,947 $41.75 Coordination of services $395 $13.94 Scared Straight $65 -$4,949 -$76.35 *Washington State 2012 dollars

Participation in Results First HI WA TN MT OR ID WY CO UT NV CA AZ NM NE KS OK TX ME ND SD MN LA AR MO IA WI MI IL IN OH PA NY WV KY MS AL GA SC NC FL VA AK VT NH MA RI CT NJ DE MD AK

Results First Can Drive System Improvement Appropriations Investment Advice Research Test New Programs Implementation Ensure Fidelity Need to discuss narrative for this slide. Oversight Monitor Outcomes

Key Results First Outcomes 2013 & 2014 (to date) 9 States Completed implementing the model and presented results to legislators and stakeholders 4 States Enacted Legislation incorporating Results First into their policymaking process 2 States Used models to analyze legislation 6 States: CT, IA, MA, NM, NY, and VT 3 States: CT, MA, VT Impact Models used to target $80 million in funding

Iowa Determined Cognitive Behavioral Therapy had high ROI, expanding in prisons Replacing ineffective domestic violence treatment program with new pilot program Expanding analysis to other policy areas DV Program: Were losing $3 for every $ spent on old program. Now working with the University of Iowa to pilot an alternative program known as Achieving Change Through Value-Based Behavior, also known as ACTV.

New Mexico Implemented in all available policy areas Assessed state spending: “Cost of Doing Nothing” Impact of State Budget Cuts Used Results First to target $57M to evidence- based programs in early education and criminal justice

Results First Can Be Used For A Growing Number of Policy Areas K-12 Education Prevention Programs Criminal Justice Child Welfare Mental Health Substance Abuse Early Education

Gary VanLandingham, DIRECTOR gvanlandingham@pewtrusts.org www.pewstates.org/ResultsFirst