1. Unmeasured Covariates and the Need for Randomization
Alec Walker Epi 221
September 2016

2. T D U

3. U T D An additional appearance of an effect TD,
due to confounding by U.

4. T D An additional appearance of an effect TD,
due to confounding by U. T D

5. Blocking the effect of confounders

6. Blocking the effect of confounders

7. Blocking the effect of confounders
Randomization
Self-matching
Self-matching
Proxies
Proxies
Instruments

8. A “Classic” Example:
Cimetidine and Gastric Cancer

9. Does cimetidine cause stomach cancer?
Case reports of de novo appearance in 1982
Colin-Jones et al looked at data from ongoing work
Persons treated with cimetidine in a 12-month window
Matched to a comparison person
General practitioner
Age
Sex
Seen for another condition
Examined the incidence of stomach cancer

10. Excess cases during follow-up
Diagnosed before cimetidine treatment started
Diagnosed within six months of starting cimetidine treatment
Diagnosed more than six months after starting cimetidine treatment
Controls
Cases of “early” cancer
Before study |  Study period  | After study
Number of cases
Colin-Jones et al. Cimetidine and gastric cancer: preliminary report from post-marketing surveillance study. Brit Med J 1982;285:

11. Hypotheses to account for excess cancers
Colin-Jones and his coauthors suggested that
Stomach cancer incidence was only an artifact of treatment having come before diagnosis in disease that was already present – The as-yet undetected disease caused the use of cimetidine and led to detected disease.
They hypothesized that the effect would disappear with longer follow-up.

12. Excess of stomach cancer persisted for two years
Deaths from Year 1 2 3 4
Ex-pected
Malignant neoplasm of the stomach 45 12 5
3.5
Colin-Jones DG. Postmarketing surveillance of the safety of cimetidine: mortality during second, third, and fourth years of follow up. Brit Med J 1985;291:1084-8

13. Excess of lung cancer persisted for at least four years
Deaths from Year 1 2 3 4
Ex-pected
Malignant neoplasm of the stomach 45 12 5
3.5
Malignant neoplasm of the trachea, bronchus and lung 35 25 17 22
12.5
Colin-Jones DG. Postmarketing surveillance of the safety of cimetidine: mortality during second, third, and fourth years of follow up. Brit Med J 1985;291:1084-8

14. Hypotheses to account for excess deaths
Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years.
Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time.

15. Hypotheses to account for excess deaths
Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years.
Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time.
Under each of these hypotheses, cimetidine use was driven by unmeasured factors that also led to the outcomes. The argument was that cimetidine did not cause the deaths from stomach cancer or lung cancer, but confounding created associations and the false appearance of causal relations.

16. A confounded relationship
Cimetidine
Cancer
Risk factors

17. Reassuring hypothesis
Cimetidine
Cancer
Risk factors

18. Is the hypothesis of confounding correct?
Stomach cancer – The as-yet undetected disease caused the use of cimetidine and led to detected disease within a few years.
Lung cancer – Shared determinants. Cigarette smoking predisposed to persistence of stomach ulcer, which in turn led to cimetidine use. The smoking (years before) also caused lung cancer during the observation time.
Colin-Jones and colleagues could not know for sure. Had they used a different study design, an answer might have been clear.

19. Mechanisms for “Confounding by Indication”
Shared risk factors
What causes the disease also leads through other mechanisms to the drug therapy
Protopathic bias
The “event” is not disease but diagnosis of disease
Treatment for symptoms of an undiagnosed disease appears to be causal
Prognostic bias
A variation on protopathic bias: with recognized early disease expectation- of-a-poor-outcome can be a confounding factor if it affect treatment choice
Doctors act in patients’ interest to give most aggressive therapy to disease with worst prognosis

20. Randomized controlled trials 1 Eliminate confounding by unmeasured baseline characteristics

21. Random assignment of treatment
“Coin flip” metaphor
Mechanical process
Assignment not systematically associated with any patient characteristics
Effects
Expectation of similar outcomes between groups under the Null Hypothesis
Justification for the calculation of p-values

22. An expectation of zero correlation
Randomization
Treatment allocation is determined by a process
That generates
An expectation of zero correlation
between
treatment and predictors of outcome.
The Predictors may be
Known or unknown to the experimenter
Measured or unmeasured
Measured poorly or well

23. Baseline balance
All characteristics other than treatment are balanced in expectation
Measured and unmeasured
Predictors and correlates of predictors
The intermediate states that later arise from these

24. Baseline balance
All characteristics other than treatment are balanced in expectation
Measured and unmeasured
Predictors and correlates of predictors
The intermediate states that later arise from these
Unadjusted estimates are unbiased estimates of treatment effect
Differences, ratios, more complex functions of
Risk, rates, hazards, survival, …
Costs, QoL, …
Even of dependent happenings, like epidemics (provided that exposure groups are not intermixed)

25. N Engl J Med Apr 1;362(13):

26. Evidence for balance The measured characteristics were very similar.
The reader infers that “the randomization was successful,” so that unmeasured characteristics were in balance too.
N Engl J Med Apr 1;362(13):

27. Delta = Treatment Effect

28. Delta = Treatment Effect

29. What did the trial show?
Treatment with Dutasteride caused decreases in diagnoses of
▼ Acute urinary retention
▼ BPH-related surgery
▼ Urinary tract infection
▼ Cancer of the prostate overall
We’re sure that alternate explanations of hidden confounding have been ruled out.

30. Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7
8-10
Dutasteride
417 18
223 12
Placebo
558 17
274 1
The relation with cancer is complex
RCTs eliminate confounding as an explanation of the relation between the intervention and the measured outcome of detected cancer

31. Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7
8-10
Dutasteride
417 18
223 12
Placebo
558 17
274 1
The relation with cancer is complex
RCTs eliminate confounding as an explanation of the relation between the intervention and the measured outcome of detected cancer
But the effect appears to be variable by cancer grade and over time

32. Was there differential cancer detection?
Protocol provided similar surveillance
Scheduled visits
Scheduled biopsies
Nonetheless, the work-up of symptoms was inevitably differential because the symptomatic outcomes were different
Urinary retention, urinary tract infection, BPH surgery
Possibly other symptoms

33. Most serious outcomes Years 1 & 2 Years 3 & 4 Treatment | Grade 5-7
8-10
Dutasteride
417 18
223 12
Placebo
558 17
274 1
The authors raised the possibility that more early detection in the placebo group may have reduced the risk of more advanced prostate cancer in those patients in later years.
What should we consider to be the treatment effect?
Did dutasteride increase the risk of advanced prostate cancer?

34. Screen-detected cancer
Randomization blocks confounding of intermediates
The increased risk of invasive cancer associated with Dutasteride was not due to confounding. It may have been due to less effective screening.
Invasive cancer U
Randomization
Screen-detected cancer
Dutasteride

35. Did dutasteride suppress early detection?
Despite per protocol similar surveillance
Scheduled visits
Scheduled biopsies
Evaluation was inevitably differential because the symptomatic outcomes were differential
Dutasteride reduced urinary retention, urinary tract infection, BPH surgery
Dutasteride may have therefore prevented the early diagnosis of some cancer that would have been incidental to the work-up of other genitourinary tract symptoms.
This same problem will often be part of the mix of uncertainty in observational studies.

36. Problems solved with randomization, other problems revealed
Randomization in RCTs provides the gold standard for inference
No hypothesis of confounding
Complex interactions between
Treatment
Intermediate outcomes and their treatment
Outcomes
Revealing the effect of dutasteride to increase the risk of advanced prostate cancer was a demonstration of the advantage of randomization. Knowing that confounding is not part of the relation we can focus on the true causal mechanism.

37. Randomized controlled trials 2 Other elements to remember in comparison with observational studies, elements to emulate

38. Preplanned treatment Dose Route Frequency Duration Timing
Discontinuation
Rules for supplementary (“rescue”) therapies

39. Entry criteria Treated disease Definition Severity, Prognosis
Other health conditions
Concomitant diseases
Others
Demographics – age, sex, race
Implicit Criteria
Populations served by participating clinical sites
See also Informed Consent

40. Informed consent Introduced for ethical reasons
Patients should be aware that they are participating in an experiment
Actively agree to enter
A subtle selection criterion
Language skills
Education
Trust in the medical care system
Inclination to follow directions

41. Preplanned data collection
Baseline
Characteristics of each participant’s history
Concomitant illnesses
Diagnostic tests and procedures
Medical examination
Treatment
Dose, route, frequency, duration, timing, discontinuation, rescue meds as actually given
Endpoints
Symptoms and tests required for diagnosis
Safety
Adverse outcomes
Contemporaneous assessment of causality

42. Treatment adherence Commitment from patients Encouragement from staff
Monitoring
Pill counts
Blood level

43. Preplanned analysis Goals Primary Secondary
Strategy for unanticipated results
Study size
Statistical power
Stopping rules

44. However – Limited follow-up
For chronic conditions, no amount of follow-up will reproduce ultimate conditions of use
Surrogate outcomes
Examples
Control of blood pressure or HbA1c
Patient-reported outcomes
Desiderata
Well established correlates of clinically important
Generally not important clinically in themselves
Manifest earlier
Real clinical outcomes can be addressed later

45. Summing up Randomized Controlled Trials
Randomization eliminates many competing explanations
Other aspects of RCT design ought to inspire us when we think of creating observational studies
Treatment definition and adherence
Eligibility
Data collection planned in advance
Analysis per protocol
Limited follow-up and sometime ethical and practical concerns means there will always be a place for observational studies