Inferring the Effects of Cancer Treatment: Divergent Results from the Early Breast Cancer Trialists’ Collaborative Group Meta-analyses of Randomized Trials and Observational Data from SEER Registries. Henson KE, Jagsi R, Cutter D, McGale P, Taylor C, Darby SC Journal of Clinical Oncology. Published ahead of print on January 19, doi: /JCO

Background - 1 Comparative effectiveness research is the direct comparison of existing health care interventions to determine which work best for which patients and which pose the greatest benefits and harms.

Background - 2 – As part of comparative effectiveness research, there is currently much enthusiasm for using observational data to determine the impact of cancer treatments. – eg, a literature search revealed that 58 published papers estimating the effect of radiotherapy in breast cancer had been published using just data from the SEER cancer registries. Many of these papers were in high impact journals.

Purpose of study To compare the effect of breast cancer radiotherapy, as estimated from observational data, with findings from randomized trials. Methods Selected results for 13,932 women randomised in trials included in meta-analyses by the Early Breast Cancer Trialists’ Collaborative Group were collated. Estimates of exactly the same quantities for 393,840 women with breast cancer in the US SEER cancer registries were derived. Henson et al, JCO e-pub 19 Jan 2016

Breast- conserving surgery Mastectomy in node positive disease 2p for heterogeneity EBCTCG*0.82 ( )0.84 ( )0.75 SEER Basic stratification†0.64 ( )1.34 ( )< Additional stratification‡0.63 ( )1.32 ( )< Results Breast Cancer Mortality, Radiotherapy vs No radiotherapy Ratios of annual death rates (95% Cls ) * Stratification: trial, year of follow-up, age at randomization, nodal status. † Years since breast cancer, age at breast cancer, calendar year of breast cancer, ethnicity. ‡ Additional stratification: stage, tumor size, number of involved nodes, grade, estrogen receptor status, quadrant, axillary clearance. Data from 1990 only: n=329,235. Henson et al, JCO e-pub 19 Jan 2016

Conclusions This study found strikingly divergent results between the EBCTCG randomized trial data and the SEER observational data. Efforts to remove confounding and selection biases did not diminish these differences. Non-randomized comparisons are liable to provide misleading estimates of treatment effects. Henson et al, JCO, e-pub 19 Jan 2016