1. Sensitivity Analysis for Residual Confounding
Sebastian Schneeweiss MD, ScD
Division of Pharmacoepidemiology and Pharmacoeconomics
Department of Medicine, Harvard Medical School,

2. Outline Residual Confounding and what we can do about it
Simple sensitivity analysis: Array Approach
Study-specific analysis: Rule Out Approach
Using additional information: External Adjustment

3. Unmeasured (residual) Confounding:
[smoking,healthy lifestyle, etc.] CU
OREC CM
RRCO
Drug exposure
Outcome
RREO

4. Unmeasured Confounding in Claims Data
Database studies are criticized for their inability to measure clinical and life-style parameters that are potential confounders in many pharmacoepi studies
OTC drug use
BMI
Clinical parameters: Lab values, blood pressure, X-ray
Physical functioning, ADL (activities of daily living)
Cognitive status

5. Strategies to Minimize Residual Confounding
Choice of comparison group
Alternative drug use that have the same perceived effectiveness and safety
Multiple comparison groups
Crossover designs (CCO, CTCO)
Instrumental Variable estimation
High dimensional proxy adjustment

6. Strategies to Discuss Residual Confounding
Qualitative discussions of potential biases
Sensitivity analysis
SA is often seen as the ‘last line of defense’
A) SA to explore the strength of an association as a function of the strength of the unmeasured confounder
B) Answers the question “How strong must a confounder be to fully explain the observed association”
Several examples in Occupational Epi but also for claims data
Greenland S et al: Int Arch Occup Env Health 1994
Wang PS et al: J Am Geriatr Soc 2001

7. Dealing with confounding
Unmeasured Confounders
Measured Confounders
Design
Restriction
Matching
Analysis
Standardization
Stratification
Regression
Unmeasured, but measurable in substudy
2-stage sampl.
Ext. adjustment
Imputation
Unmeasurable
Design
Analysis
Cross-over
Active comparator (restriction)
Instrumental variable
Proxy analysis
Sensitivity analysis
Propensity scores
Marginal Structural Models
Schneeweiss, PDS 2006

8. A simple sensitivity analysis
The apparent RR is a function of the adjusted RR times ‘the imbalance of the unobserved confounder’
After solving for RR we can plug in values for the prevalence and strength of the confounder:

9. A made-up example
Association between TNF-a blocking agents and NH lymphoma in RA patients
Let’s assume an observed RR of 2.0
Let’s assume 50% of RA patients have a more progressive immunologic disease
… and that more progressive disease is more likely to lead to NH lymphoma
Let’s now vary the imbalance of the hypothetical unobserved confounder

10. Bias by residual confounding

11. drugepi.org

12. Pros and cons of “Array approach”
Very easy to perform using Excel
Very informative to explore confounding with little prior knowledge
Problems:
It usually does not really provide an answer to a specific research question
4 parameters can vary -> in a 3-D plot 2 parameter have to be kept constant
The optical impression can be manipulated by choosing different ranges for the axes

13. Same example, different parameter ranges

14. Conclusion of “Array Approach”
Great tool but you need to be honest to yourself
For all but one tool that I present today:
Assuming conditional independence of CU and CM given the exposure status
If violated than residual bias may be overestimated
Hernan, Robins: Biometrics 1999 CU ?
OREC CM
RRCO
Drug exposure
Outcome
RREO

15. More advanced techniques
Wouldn’t it be more interesting to know
How strong and imbalanced does a confounder have to be in order to fully explain the observed findings?
RRCO
OREC

16. OREC RRCO Example: Psaty et al: JAGS 1999;47:749 CCB use and acute MI.
The issue:
Are there any unmeasured factors that may lead to a preferred prescribing of CCB to people at higher risk for AMI?
OREC
ARR = 1.57
ARR = 1.30
RRCO

17. drugepi.org

18. Caution!
Psaty et al. concluded that it is unlikely that an unmeasured confounder of that magnitude exists
However, the randomized trial ALLHAT showed no association between CCB use and AMI
Alternative explanations:
Joint residual confounding may be larger than anticipated from individual unmeasured confounders
Not an issue of residual confounding but other biases, e.g. control selection?

19. Pros and cons of “Rule Out Approach”
Very easy to perform using Excel
Meaningful and easy to communicate interpretation
Study-specific interpretation
Problems:
Still assuming conditional independence of CU and CM
“Rule Out” lacks any quantitative assessment of potential confounders that are unmeasured

20. External Adjustment One step beyond sensitivity analyses
Using additional information not available in the main study
Often survey information

21. Strategies to Adjust residual con-founding using external information
Survey information in a representative sample can be used to quantify the imbalance of risk factors that are not measured in claims among exposure groups
The association of such risk factors with the outcome can be assess from the medical literature (RCTs, observational studies)
Velentgas et al: PDS, 2007
Schneeweiss et al: Epidemiology, 2004

22. From Survey data in a subsample From medical literature
In our example:
OREC
RRCO
[smoking,aspirin, BMI, etc.] CU
From Survey data in a subsample
From medical literature CM
Rofecoxib
Acute MI
RREO

23. More contrasts

24. Sensitivity of Bias as a Function of a Misspecified RRCD :
Obesity (BMI >=30 vs. BMI<30)

25. Sensitivity towards a misspecified RRCO from the literature:
OTC aspirin use (y/n)

26. drugepi.org

27. Limitations Example is limited to 5 potential confounders
No lab values, physical activity, blood pressure etc.
What about the ‘unknow unknowns’?
To assess the bias we assume an exposure–disease association of 1 (null hypothesis)
The more the truth is away from the null the more bias in our bias estimate
However the less relevant unmeasured confounders become
Validity depends on representativenes of sampling with regard to the unmeasured confounders
We could not consider the joint distribution of confounders
Limited to a binary world

28. Solving the Main Limitations
Need a method
That addresses the joint distribution of several unmeasured confounders
That can handle binary, ordinal or normally distributed unmeasured confounders
Lin et al. (Biometrics 1998):
Can handle a single unmeasured covariate of any distribution
But can handle only 1 covariate
Sturmer, Schneeweiss et al (Am J Epidemiol 2004):
Propensity score calibration
Can handle multiple unmeasured covariates of any distribution

29. Summary
Sensitivity analyses for residual confounding are underutilized although they are technically easy to perform
Excel program for download (drugepi.org)
The real challenge is the interpretation of your findings
This is all summarized in Schneeweiss PDS 2007