The Cause…or the “What” of What Works? David S. Cordray Vanderbilt University IES Research Conference Washington, DC June 16, 2006

Achieved Relative Strength =.15 (85)-(70) = Outcome Infidelity “Infidelity” T C Treatment Strength C t Expected Relative Strength =.25

Intervention Fidelity Intervention, as conceptualized (T and C) Intervention, as realized or implemented (t and c) Does the intervention, as implemented, faithfully represents the intervention “theory” or model? Intervention Fidelity

Dimensions Intervention Fidelity Adherence to the intervention model: –Essential components (activities, processes) –Necessary, but not unique, activities, processes and structures (supporting the essential components of T) –Ordinary features of the setting (shared with C); Quality of delivery (adherence); and Receipt essential components.

Fidelity and IES Goals Goal 2 (Development) –Need explicit model of the final intervention model –Products: Evidence of potential effects, an intervention model, all relevant materials, and detailed indices for assessing fidelity Goals 3 (Efficacy) and 4 (Scale-up) –Need detailed indices of fidelity for: training and program research (e.g., effects of core components) and –Need efficient indices for scaling implementation in the unit of delivery –Need efficient indices of fidelity in all conditions (T and C) “Goal 6” – Managing the tension between fidelity and inevitable adaptation

Substantial Challenges Developing explicit enough program models so the fidelity can be meaningfully assessed; Measuring fidelity entails: –Multiple indicators –Mixed methods; Constructing indexes: –Differential weighting of components –Use of broader range of scale construction procedures (cumulative and rubric-based); Validation studies; Because fidelity plays many roles, we’ll need multiple versions of fidelity scales (e.g, “scale-down”)

The Cause…and “What Works?” In experimental research, the causal agent is a relative difference between ingredients in T and C conditions; The achieved relative strength of the contrast is affected by infidelity; and With adequate indices of fidelity, we can move to questions of what works, under what circumstances, and why.