Policy and Evidence: An Uneasy but Essential Partnership Mark E. Courtney Fred H. Wulczyn Chapin Hall at the University of Chicago

The policy realm is where conflicting values play out most directly  Practice concerns individual-level effects on outcomes that are generally agreed upon  Administration is organized around achieving the goals of policy  Social welfare policy is often the result of highly contested and shifting values  Examples: Willingness to pay  Child protection versus family preservation  Benefit-cost analysis of extending foster care past 18

Types of Evidence Needed for Policy Making  Characteristics/needs of the population  Characteristics and behaviors of systems  Capacities  Political environment  Organizational cultures  System dynamics

System Dynamics: Patterns Identify Targets of Policy

Limits of randomized experiments for informing policy  Some policy changes cannot be assessed at all using experiments  Mandatory child maltreatment reporting  Extending foster care past 18  Constellations of policies are difficult to assess due to design complexities  Extending care, health insurance extension, and education vouchers  Time required leads to changes in context that can compromise experiments, but actually benefit non- experimental methods …but, experiments are VERY valuable!

Principles and Lessons  Sound information to guide policy will come as much or more from sound data on populations and services as from evaluation research per se  Experiments are best implemented in the context of sophisticated knowledge of populations, service contexts, and system dynamics  At best, research can often only help illustrate the tradeoffs of policy choices  Do not oversell the implications of findings for policy, lest you get what you asked for  Illustrating tradeoffs helps policymakers understand when and how ideology guides their decision making