1. Evaluating Service Delivery and Patient Safety Interventions
Richard Lilford. Boston 2007 QC

2. Two Key Evaluation Issues
Study design
End points

3. Before and After Data-collection Control Intervention
Post-intervention
Pre-intervention
Example: Blood stream infections in ICU. Pronovost et al, 2006

4. Controlled Cross-Sectional
Data-collection Control Intervention
Post-intervention
Pre-intervention
Impact of dedicated medication nurses. Greengold et al 2003

5. The Concept of Terminal Event

6. Controlled Before and After
Data-collection Control Intervention
Post-intervention
Pre-intervention
Example: Nationally mediated drug reviews: nursing homes v assisted living.
Briesacher et al,2005.

7. Advantage Controlled Before and After
1) Reduce variance (improve precision)
2) Reduce bias (especially non-randomised studies)

8. Two Key Evaluation Issues
Study design
End points

9. Management
Processes
(Fidelity)
Clinical
Processes
/errors
Outcomes
and
throughput
Structure
Intervening
variables

10. Management
Processes
(Fidelity)
Clinical
Processes
/errors
Outcomes
and
throughput
Structure
Intervening
variables
Specific interventions

11. Management
Processes
(Fidelity)
Clinical
Processes
/errors
Outcomes
and
throughput
Structure
Intervening
variables
Generic Interventions
Specific interventions

12. Corollaries for Service Interventions
1) Multiple end-points
2) a) Controlled before and after
b) Compare change in outcomes
from base-line to post-intervention

13. CASE STUDY: MERIT TRIAL
Control Met p Value
1) Multiple end-points
Calls: non-arrest
<0.001
2) Change from base-line
Cardiac arrest
Unexpected death

14. Control + MET hospitals combined
Base-line Post P
Cardiac arrest
Unexpected death

15. Combined Pre-Post Effect
Possible Reasons for
Combined Pre-Post Effect
1) General Secular Trend
2) Study effect
a) Mediated directly
b) Indirect effect

16. A Design for all Seasons
Control
Intervention
Time

17. Advantages of Step Wedge Design
Ethical, political and logistic
2) Scientific

18. A Design for all Seasons
Control
Intervention
Time

20. Scientific Arguments for Step Wedge Cluster Designs
Parallel Step-Wedge
Intervention effect
Sub-groups
Time-effects
General
Sustainability
(loss of power)
Time of intervention/Hawthorne - +

21. Design 1: Uncontrolled Data-collection Control Intervention
Post-intervention