Unmeasured Covariates and the Need for Randomization Alec Walker Epi 221 September 2016

T D U

U T D An additional appearance of an effect TD, due to confounding by U.

T D An additional appearance of an effect TD, due to confounding by U. T D

Blocking the effect of confounders

Blocking the effect of confounders

Blocking the effect of confounders Randomization Self-matching Self-matching Proxies Proxies Instruments

A “Classic” Example: Cimetidine and Gastric Cancer

Does cimetidine cause stomach cancer? Case reports of de novo appearance in 1982 Colin-Jones et al looked at data from ongoing work Persons treated with cimetidine in a 12-month window Matched to a comparison person General practitioner Age Sex Seen for another condition Examined the incidence of stomach cancer

Excess cases during follow-up Diagnosed before cimetidine treatment started Diagnosed within six months of starting cimetidine treatment Diagnosed more than six months after starting cimetidine treatment Controls Cases of “early” cancer Before study |  Study period  | After study Number of cases Colin-Jones et al. Cimetidine and gastric cancer: preliminary report from post-marketing surveillance study. Brit Med J 1982;285:1311-1313

Hypotheses to account for excess cancers Colin-Jones and his coauthors suggested that Stomach cancer incidence was only an artifact of treatment having come before diagnosis in disease that was already present – The as-yet undetected disease caused the use of cimetidine and led to detected disease. They hypothesized that the effect would disappear with longer follow-up.

Excess of stomach cancer persisted for two years Deaths from Year 1 2 3 4 Ex-pected Malignant neoplasm of the stomach 45 12 5 3.5 Colin-Jones DG. Postmarketing surveillance of the safety of cimetidine: mortality during second, third, and fourth years of follow up. Brit Med J 1985;291:1084-8

Excess of lung cancer persisted for at least four years Deaths from Year 1 2 3 4 Ex-pected Malignant neoplasm of the stomach 45 12 5 3.5 Malignant neoplasm of the trachea, bronchus and lung 35 25 17 22 12.5 Colin-Jones DG. Postmarketing surveillance of the safety of cimetidine: mortality during second, third, and fourth years of follow up. Brit Med J 1985;291:1084-8

Hypotheses to account for excess deaths Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years. Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time.

Hypotheses to account for excess deaths Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years. Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time. Under each of these hypotheses, cimetidine use was driven by unmeasured factors that also led to the outcomes. The argument was that cimetidine did not cause the deaths from stomach cancer or lung cancer, but confounding created associations and the false appearance of causal relations.

A confounded relationship Cimetidine Cancer Risk factors

Reassuring hypothesis Cimetidine Cancer Risk factors

Is the hypothesis of confounding correct? Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years. Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time. Colin-Jones and colleagues could not know for sure. Had they used a different study design, an answer might have been clear.

Mechanisms for “Confounding by Indication” Shared risk factors What causes the disease also leads through other mechanisms to the drug therapy Protopathic bias The “event” is not disease but diagnosis of disease Treatment for symptoms of an undiagnosed disease appears to be causal Prognostic bias A variation on protopathic bias: with recognized early disease expectation- of-a-poor-outcome can be a confounding factor if it affect treatment choice Doctors act in patients’ interest to give most aggressive therapy to disease with worst prognosis

Randomized controlled trials 1 Eliminate confounding by unmeasured baseline characteristics

Random assignment of treatment “Coin flip” metaphor Mechanical process Assignment not systematically associated with any patient characteristics Effects Expectation of similar outcomes between groups under the Null Hypothesis Justification for the calculation of p-values

An expectation of zero correlation Randomization Treatment allocation is determined by a process That generates An expectation of zero correlation between treatment and predictors of outcome. The Predictors may be Known or unknown to the experimenter Measured or unmeasured Measured poorly or well

Baseline balance All characteristics other than treatment are balanced in expectation Measured and unmeasured Predictors and correlates of predictors The intermediate states that later arise from these

Baseline balance All characteristics other than treatment are balanced in expectation Measured and unmeasured Predictors and correlates of predictors The intermediate states that later arise from these Unadjusted estimates are unbiased estimates of treatment effect Differences, ratios, more complex functions of Risk, rates, hazards, survival, … Costs, QoL, … Even of dependent happenings, like epidemics (provided that exposure groups are not intermixed)

N Engl J Med. 2010 Apr 1;362(13):1192-202

Evidence for balance The measured characteristics were very similar. The reader infers that “the randomization was successful,” so that unmeasured characteristics were in balance too. N Engl J Med. 2010 Apr 1;362(13):1192-202

Delta = Treatment Effect

Delta = Treatment Effect

What did the trial show? Treatment with Dutasteride caused decreases in diagnoses of ▼ Acute urinary retention ▼ BPH-related surgery ▼ Urinary tract infection ▼ Cancer of the prostate overall We’re sure that alternate explanations of hidden confounding have been ruled out.

Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7 8-10 Dutasteride 417 18 223 12 Placebo 558 17 274 1 The relation with cancer is complex RCTs eliminate confounding as an explanation of the relation between the intervention and the measured outcome of detected cancer

Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7 8-10 Dutasteride 417 18 223 12 Placebo 558 17 274 1 The relation with cancer is complex RCTs eliminate confounding as an explanation of the relation between the intervention and the measured outcome of detected cancer But the effect appears to be variable by cancer grade and over time

Was there differential cancer detection? Protocol provided similar surveillance Scheduled visits Scheduled biopsies Nonetheless, the work-up of symptoms was inevitably differential because the symptomatic outcomes were different Urinary retention, urinary tract infection, BPH surgery Possibly other symptoms

Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7 8-10 Dutasteride 417 18 223 12 Placebo 558 17 274 1 The authors raised the possibility that more early detection in the placebo group may have reduced the risk of more advanced prostate cancer in those patients in later years. What should we consider to be the treatment effect? Did dutasteride increase the risk of advanced prostate cancer?

Screen-detected cancer Randomization blocks confounding of intermediates The increased risk of invasive cancer associated with Dutasteride was not due to confounding. It may have been due to less effective screening. Invasive cancer U Randomization Screen-detected cancer Dutasteride

Did dutasteride suppress early detection? Despite per protocol similar surveillance Scheduled visits Scheduled biopsies Evaluation was inevitably differential because the symptomatic outcomes were differential Dutasteride reduced urinary retention, urinary tract infection, BPH surgery Dutasteride may have therefore prevented the early diagnosis of some cancer that would have been incidental to the work-up of other genitourinary tract symptoms. This same problem will often be part of the mix of uncertainty in observational studies.

Problems solved with randomization, other problems revealed Randomization in RCTs provides the gold standard for inference No hypothesis of confounding Complex interactions between Treatment Intermediate outcomes and their treatment Outcomes Revealing the effect of dutasteride to increase the risk of advanced prostate cancer was a demonstration of the advantage of randomization. Knowing that confounding is not part of the relation we can focus on the true causal mechanism.

Randomized controlled trials 2 Other elements to remember in comparison with observational studies, elements to emulate

Preplanned treatment Dose Route Frequency Duration Timing Discontinuation Rules for supplementary (“rescue”) therapies

Entry criteria Treated disease Definition Severity, Prognosis Other health conditions Concomitant diseases Others Demographics – age, sex, race Implicit Criteria Populations served by participating clinical sites See also Informed Consent

Informed consent Introduced for ethical reasons Patients should be aware that they are participating in an experiment Actively agree to enter A subtle selection criterion Language skills Education Trust in the medical care system Inclination to follow directions

Preplanned data collection Baseline Characteristics of each participant’s history Concomitant illnesses Diagnostic tests and procedures Medical examination Treatment Dose, route, frequency, duration, timing, discontinuation, rescue meds as actually given Endpoints Symptoms and tests required for diagnosis Safety Adverse outcomes Contemporaneous assessment of causality

Treatment adherence Commitment from patients Encouragement from staff Monitoring Pill counts Blood level

Preplanned analysis Goals Primary Secondary Strategy for unanticipated results Study size Statistical power Stopping rules

However – Limited follow-up For chronic conditions, no amount of follow-up will reproduce ultimate conditions of use Surrogate outcomes Examples Control of blood pressure or HbA1c Patient-reported outcomes Desiderata Well established correlates of clinically important Generally not important clinically in themselves Manifest earlier Real clinical outcomes can be addressed later

Summing up Randomized Controlled Trials Randomization eliminates many competing explanations Other aspects of RCT design ought to inspire us when we think of creating observational studies Treatment definition and adherence Eligibility Data collection planned in advance Analysis per protocol Limited follow-up and sometime ethical and practical concerns means there will always be a place for observational studies