1. Heterogeneity and Comparative Effectiveness: Implications for Studies, Reviews, and Policy David Atkins, MD, MPH Health Services Research and Development Quality Enhancement Research Initiative Dept. of Veterans Affairs

2. Session Overview: How to Examine for Heterogeneity and Its Implications Within an individual studyWithin an individual study Within a systematic review or meta- analysisWithin a systematic review or meta- analysis In Policy DecisionsIn Policy Decisions ConclusionsConclusions

3. Heterogeneity of treatment effect Non-random variation in direction or magnitude of individual treatment effects within a populationNon-random variation in direction or magnitude of individual treatment effects within a population May be due to factors specific to patient, provider, environment, or treatmentMay be due to factors specific to patient, provider, environment, or treatment Also referred to as clinical heterogeneityAlso referred to as clinical heterogeneity

4. “ Clinical trials are good experiments but poor surveys.” – N. Longford –From Kravitz et al.. Milbank Q. 2004;82:661-687

5. Heterogeneity vs. Applicability HTE – How great is variation in the treatment effect for individuals within the study population relative to the “average” effect in that population?HTE – How great is variation in the treatment effect for individuals within the study population relative to the “average” effect in that population? Applicability – Do the results for the study population apply to a specific patient, group or setting of interest (outside of the study)?Applicability – Do the results for the study population apply to a specific patient, group or setting of interest (outside of the study)?

6. “ Clinical trials are good experiments but poor surveys.” – N. Longford –From Kravitz et al.. Milbank Q. 2004;82:661-687

7. Relevance in Policy Debates Concern that “average” effects don’t apply to individual patientsConcern that “average” effects don’t apply to individual patients “Cookbook medicine”“Cookbook medicine” Decisions based on CER that ignore heterogeneity might deny clinicians ability to individualize treatment decisionsDecisions based on CER that ignore heterogeneity might deny clinicians ability to individualize treatment decisions

8. Is alendronate (Fosamax) better than calcium and vitamin D to prevent fracture? What factors might cause heterogeneity in the net benefits of alendronate vs. calcium/vitamin D?What factors might cause heterogeneity in the net benefits of alendronate vs. calcium/vitamin D? AUDIENCE RESPONSEAUDIENCE RESPONSE

9. Sources of Heterogeneity in Net Benefits of Treatment Responsiveness to treatment:Responsiveness to treatment: –E.g., Calcium intake, vitamin D status Vulnerability to harmsVulnerability to harms –? Risks from bisphosphonates Baseline risk in absence of treatmentBaseline risk in absence of treatment –Age, health status, bone density Values attached to outcomesValues attached to outcomes –Convenience of regimens

10. Scenario 63 year old woman with diabetes and ischemic heart disease, evaluated for cardiac procedure to address increasing exertional chest pain despite optimal medical therapy. She has two-vessel disease with preserved LV function. She is pondering whether to have PCA with stent or bypass.63 year old woman with diabetes and ischemic heart disease, evaluated for cardiac procedure to address increasing exertional chest pain despite optimal medical therapy. She has two-vessel disease with preserved LV function. She is pondering whether to have PCA with stent or bypass.

11. Question Is bypass surgery or percutaneous coronary intervention (PCI) superior for treating coronary artery disease?Is bypass surgery or percutaneous coronary intervention (PCI) superior for treating coronary artery disease? Does the answer depend on patient or intervention factors (i.e. is there heterogeneity of the treatment effects )?Does the answer depend on patient or intervention factors (i.e. is there heterogeneity of the treatment effects )?