1. 1 EPI235: Epi Methods in HSR April 17, 2007 L5 Program Evaluation with Longitudinal Data 1: Applications (Dr. Schneeweiss) Various examples of applications in Health Services Research. Strengths and limitations of time series analysis. Background reading: Soumerai SB, Avorn J, Gortmaker S, Ross ‑ Degnan D. Payment restrictions for prescription drugs in Medicaid: Effects on therapy, cost, and equity. New Engl J Med 1987;317:550 ‑ 556. Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin TJ. Effects of Medicaid drug-payment limits on admission to hospitals and nursing homes. N Engl J Med 1991;325:1072-1077

2. 2

3. 3

4. 4

5. 5 What is Program Evaluation?

6. 6 Program Evaluation

7. 7

8. 8

9. 9

10. 10

11. 11

12. 12 Observational Design: Simple pre-post comparisons Time Intervention Assumptions for causal inference: 1. The pre experience represents the post experience had there been no intervention Outcome rate

13. 13 Observational Design: Simple pre-post comparisons Time Intervention Threat to causal inference: Single pre-post estimates are averages of an underlying trend independent of the Intervention Outcome rate

14. 14 25% cost sharing in Quebec Tamblyn, JAMA 2000

15. 15 Observational Design: no concurrent controls Time Intervention Assumptions for causal inference: 1. Close temporal relation 2. Extrapolation of baseline trend is equal to the counterfactual experience Schneeweiss, Health Policy 2000

16. 16 Fixed ¢ 50 cost sharing in SC Nelson, Med Care 1984

17. 17 Observational Design: With concurrent controls Assumptions for causal inference: 1. … 2. Control trend is equal to the counterfactual experience of intervention group Time Intervention Intervention group Control group Schneeweiss, J Clin Epi 2002

18. 18 3 prescription caps in NH: 80% 85% 90% 95% 100% BaselineCapAfter Cap % outside nursing home New Jersey New Hampshire Soumerai NEJM 1991

19. 19

20. 20 … or randomization? Time Intervention Intervention group Control group R Assumptions for causal inference: 1. Subjects comply with their assigned ‘treatment’ = policy

21. 21 Complex statistical analyses are less convincing for decision makers

22. 22 Formulary delisting in BC

23. 23 More Examples:

24. 24 More Examples:  Prescription drug use  Surgical site infections  Contraindicated drug use

25. 25 Medicaid prior authorization: Use Smalley NEJM 1995

26. 26 Medicaid prior authorization: $$ Smalley NEJM 1995

27. 27 Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery) Solid diamonds = surgical site infection rate after cesarean section Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods. Weinberg et al.: Arch Intern Med 2001 Reducing surgical site infections after C-section: Hospital A

28. 28 Open squares = utilization indicator (cesarean sections receiving perioperative antibiotic prophylaxis) Open circles = timing indicator (antibiotic within 1 hour of delivery) Solid diamonds = surgical site infection rate after cesarean section Period 1 was a baseline period. Periods 2 and 3 were successive intervention periods. Weinberg et al.: Arch Intern Med 2001 Reducing surgical site infections after C-section: Hospital B

29. 29 The Intervention: 'Dear Doctor' letters concerning interactions between cisapride and a series of drugs. A letter in 1995 described a risk of prolonged QT intervals and serious ventricular arrhythmia in patients who received macrolide antibiotics and imidazole antifungals in conjunction with cisapride. A June 1998 letter that expanded the list of contraindicated comedications had wider distribution than an earlier one, was accompanied by substantial Internet and media coverage, and was complemented by an effort to inform large pharmacy dispensing information organizations of the warnings against concurrent use of the named drugs. FDA Risk Management: Effectiveness of a ‘Dear Doctor’ letter

30. 30 Weatherby LB et al. PharmEpi Drug Safety 2001

31. 31

32. 32 Lecture on LPUs: Weatherby LB et al. Clin Pharm Ther 2002

33. 33 The Policy Model

34. 34 Policy Model: reference pricing

35. 35 The Clinical Model

36. 36 More on the Clinical Model:

37. 37 Clinical Model: reference pricing

38. 38 Policy Model vs. Clinical Model