1. Accounting for Clinical Heterogeneity in Comparative Effectiveness Research How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)? The Example of the BARI trial for CABG vs PTCA September 28, 2010 Carlos Weiss, MD, MHS

2. AHRQ DEcIDE Project: Methods to Study the Heterogeneity of Treatment Effects in Comparative Effectiveness Research PI: Ravi Varadhan, PhD Co-I: Jodi Segal, MD, MPH; Cynthia Boyd, MD, MPH; Al Wu, MD, MPH Consultant: David Kent, MD, MPH Technical Experts: Curt Furberg, MD, PhD; Bruce Psaty, MD, PhD Task Order Officer: Parivash Nourjah, PhD

3. N=1,829

4. BARI Clinical Question Population targeted: “Multivessel disease” with severe angina or ischemia Intervention: PTCA (a form of PCI) Comparator: CABG Outcome: 5-yr Mortality

5. Questions to Audience - Set 1 What are sources of HTE? How would pre-specification of analyses affect interpretation of results?

6. BARI Design for HTE Protocol pre-specified 4 subgroup analyses: angina severity

7. BARI Design for HTE Protocol pre-specified 4 subgroup analyses: angina severity left ventricular function number of diseased vessels complex lesions

8. BARI Clinical Question: Sources of HTE in CABG vs PTCA

9. BARI Clinical Question: Sources of HTE in PTCA v CABG Patients –baseline risk –competing risks –risk of treatment harms –treatment responsiveness >>Ideas drawn from Kravitz, Duan & Braslow, 2004, Milbank Quarterly

10. BARI Clinical Question: Sources of HTE in PTCA v CABG Patients –baseline risk –competing risks –risk of treatment harms –treatment responsiveness Treatment Providers Environments

11. PATIENTS PROVIDERSENVIRONMENTS TREATMENT

12. BARI Results 5-yr Mortality: Overall, no clinically significant nor statistically significant difference

14. CABG,+ treated diabetes PTCA,+ treated diabetes PTCA,- treated diabetes CABG,- treated diabetes

15. Questions to Audience - Set 2 When should one be worried that a subgroup result is an error (Type I or Type II) ? What can be done to lower error probabilities?

16. Proposed General Approach to Examining a Trial for HTE 1. HTE hypotheses pre-specified? 2. Design and measurement quality? 3. Modeling pre-specified?

17. Proposed General Approach to Examining a Trial for HTE 1. HTE hypotheses pre-specified? 2. Design and measurement quality? 3. Modeling pre-specified? 4. If No to 1, 2 or 3: Validation study available?

18. Proposed General Approach to Examining a Trial for HTE 1. HTE hypotheses pre-specified? 2. Design and measurement quality? 3. Modeling pre-specified? 4. If No to 1, 2 or 3: Validation study available? 5.a. If frequentist, test of interaction performed? 6.b. If Bayesian, pre-specified priors and variance acceptable?

20. Extra Slides

21. What is Heterogeneity of Treatment Effect? Non-random variability in the direction or magnitude of a treatment effect

22. Risk if Treated, events per 100 p-y 5 10 Risk if UnTreated or “Baseline Risk”, events per 100 p-y 10 5 Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

23. Risk if Treated, events per 100 p-y 5 10 ─ Average Absolute Treatment Effect (ARR) Risk if UnTreated or “Baseline Risk”, events per 100 p-y 10 5 AR Open circles - HTE absent Closed circles - HTE present Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

24. Risk if Treated, events per 100 p-y 5 10 Risk if UnTreated or “Baseline Risk”, events per 100 p-y 10 5 - - Average Relative Treatment Effect (RRR) Δy/Δx = RR Open circles - HTE present Closed circles - HTE absent Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale