1. Joint Statistical Meetings, Vancouver, August 1, 2018
Weighting-Based Sensitivity Analysis in Causal Mediation Studies: Interactive Tools for Analysts
Guanglei Hong University of Chicago
Xu Qin University of Pittsburgh
Fan Yang University of Colorado Denver

2. What Is Sensitivity Analysis?
A causal conclusion is valid only when the identification assumptions hold.
Sensitivity Analysis: An attempt to determine whether a plausible violation of an identification assumption could qualitatively change an analytic conclusion
Logical Reasoning: Analytic conclusions that are harder to alter by such a violation are expected to add a higher value to scientific knowledge

3. Did You Conduct a Sensitivity Analysis?
Scientists always rightfully voice skepticisms toward causal claims.
A study is incomplete without a sensitivity analysis.
However, Preach  Practice
Sensitivity analysis is rarely done (except in economics)
Discussions mostly limited to the statistics community
Never in guidelines for grant application or publication

4. WHY? Sensitivity analysis is challenging
Often requires additional assumptions
like a tail-chasing game that never ends…

5. Weighting-based sensitivity analysis
The new approach is consistent with the logic of using propensity score-based weighting to reduce selection bias
In a nutshell
The discrepancy between a new weight that adjusts for an omitted confounder and an initial weight that omits the confounder captures the role of the confounder that contributes to the bias

6. A Re-evaluation of the National Welfare-to-Work Strategies (Hong, Deutsch, & Hill, 2015)
Employment M
Under LFA +
Under control +
Treatment Z
Depression Y
Did the treatment increase employment rate? -- YES (40%  65%)
Did the treatment alleviate or aggravate maternal depression two years later? -- APPARENTLY NO
Did employment mediate the treatment impact on depression? -- SEEMINGLY

7. Definition of the Causal Effects
Definition of the Causal Effects (Pearl, 2001; Robins & Greenland, 1992) 𝑀 𝑍 𝑌
Z Effect on M: 𝐸 𝑀 1 −𝑀 0
Z Effect on Y: 𝐸 𝑌 1 −𝑌 0
Natural Indirect Effect: 𝐸 𝑌 1,𝑀 1 −𝑌 1,𝑀 0
Natural Direct Effect: 𝐸 𝑌 1,𝑀 0 −𝑌 0,𝑀 0

8. Major advantage: No need to specify an outcome model!
Weighting-Based Identification of the Causal Effects (Hong, 2010, 2015)
𝐸 𝑌 1,𝑀 0 =𝐸 𝑊𝑌|𝑍=1
where
𝑊= 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥
NIE: 𝐸 𝑌 1,𝑀 1 −𝑌 1,𝑀 0 =𝐸 𝑌|𝑍=1 −𝐸 𝑊𝑌|𝑍=1
NDE: 𝐸 𝑌 1,𝑀 0 −𝑌 0,𝑀 0 =𝐸 𝑊𝑌|𝑍=1 −𝐸 𝑌|𝑍=0
Major advantage: No need to specify an outcome model!

9. Identification under Sequential Ignorability (Imai et al, 2010a, 2010b)X
X: Observed pretreatment covariates 𝑀 𝑍 𝑌
Assignment of Z is strongly ignorable given X
Assignment of M is strongly ignorable given Z=z and X (i.e., there are no omitted pretreatment and post-treatment confounders of the M-Y relationship)

10. Potential Sources of Omitted Confounding
Three Types of Omission in causal mediation analysis:
Observed pretreatment covariates and/or their higher order functions
Observed posttreatment covariates
Unobserved pretreatment or posttreatment covariates
(Sensitivity analysis for type C omission requires scientific reasoning and empirical evidence indicating the comparability of an unobserved covariate to some of the observed ones)

11. Bias due to an Omitted Pretreatment Confounder P𝑋 𝑀 𝑍 𝑌 𝑃
e.g., preference for taking care of family rather than working

12. 𝛿 𝑁𝐼𝐸 ≡𝐸 𝑌 1,𝑀 1 −𝑌 1,𝑀 0 Weighting-based adjustment for X
𝛿 𝑁𝐼𝐸 ≡𝐸 𝑌 1,𝑀 1 −𝑌 1,𝑀 0
Weighting-based adjustment for X
𝑑 𝑁𝐼𝐸 =𝐸 𝑌|𝑍=1 −𝐸 𝑊𝑌|𝑍=1
𝑊= 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥
To additionally remove bias associated with the omitted P
𝑑 𝑃.𝑁𝐼𝐸 =𝐸 𝑌|𝑍=1 −𝐸 𝑊 𝑃 𝑌|𝑍=1
𝑊 𝑃 = 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥,𝑃=𝑝 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥,𝑃=𝑝

13. Bias due to an omitted pretreatment confounder P
𝐵𝑖𝑎𝑠 𝑁𝐼𝐸 = 𝑑 𝑁𝐼𝐸 − 𝑑 𝑃.𝑁𝐼𝐸 =𝑐𝑜𝑣 𝑊 𝑃 −𝑊,𝑌|𝑍=1
𝐵𝑖𝑎𝑠 𝑁𝐷𝐸 = −1 × 𝐵𝑖𝑎𝑠 𝑁𝐼𝐸
𝐸𝑆 of 𝐵𝑖𝑎𝑠 𝑁𝐼𝐸 =𝜎𝜌
𝜎= 𝑣𝑎𝑟 𝑊 𝑃 −𝑊|𝑍=1 ; 𝜌=𝑐𝑜𝑟𝑟 𝑊 𝑃 −𝑊,𝑌|𝑍=1
𝜎=0 if P does not conditionally predict M; monotonic between 𝜎 and the P-M association
𝜌=0 if P does not conditionally predict Y; monotonic between 𝜌 and the P-Y association

14. Bias due to an omitted posttreatment confounder Q𝑋 𝑀 𝑄 𝑍 𝑌
e.g., ever been out of welfare and out of work during the first year after randomization

15. 𝑁𝐼𝐸≡𝐸 𝑌 1,𝑄 1 ,𝑀 1,𝑄 1 −𝑌 1,𝑄 1 ,𝑀 0,𝑄 0 𝑋 𝑀 𝑄 𝑍 𝑌

16. 𝑁𝐷𝐸≡𝐸 𝑌 1,𝑄 1 ,𝑀 0,𝑄 0 −𝑌 0,𝑄 0 ,𝑀 0,𝑄 0 𝑋 𝑀 𝑄 𝑍 𝑌

17. Q and M as Consecutive Mediators
Identification Assumptions
Assignment of Z is strongly ignorable given X
Assignment of Q is strongly ignorable given Z=z and X
Assignment of M is strongly ignorable given Z=z, X, and Q
𝐸 𝑌 1,𝑄 1 , 𝑀 0,𝑄 0 =𝐸 𝑊 𝑄 𝑌|𝑍=1
where
𝑊 𝑄 = 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥,𝑄=𝑞 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥,𝑄=𝑞

18. 𝐵𝑖𝑎𝑠 𝑁𝐼𝐸 = 𝑑 𝑁𝐼𝐸 − 𝑑 𝑄.𝑁𝐼𝐸 =𝑐𝑜𝑣 𝑊 𝑄 −𝑊,𝑌|𝑍=1
𝑁𝐼𝐸≡𝐸 𝑌 1,𝑄 1 ,𝑀 1,𝑄 1 −𝑌 1,𝑄 1 ,𝑀 0,𝑄 0
Weighting-based adjustment for X
𝑑 𝑁𝐼𝐸 =𝐸 𝑌|𝑍=1 −𝐸 𝑊𝑌|𝑍=1
𝑊= 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥
To additionally remove bias associated with Q
𝑑 𝑄.𝑁𝐼𝐸 =𝐸 𝑌|𝑍=1 −𝐸 𝑊 𝑄 𝑌|𝑍=1
𝑊 𝑄 = 𝑝𝑟 𝑀=𝑚|𝑍=0,𝑋=𝑥,𝑄=𝑞 𝑝𝑟 𝑀=𝑚|𝑍=1,𝑋=𝑥,𝑄=𝑞
𝐵𝑖𝑎𝑠 𝑁𝐼𝐸 = 𝑑 𝑁𝐼𝐸 − 𝑑 𝑄.𝑁𝐼𝐸 =𝑐𝑜𝑣 𝑊 𝑄 −𝑊,𝑌|𝑍=1
𝐵𝑖𝑎𝑠 𝑁𝐷𝐸 = −1 × 𝐵𝑖𝑎𝑠 𝑁𝐼𝐸

19. Two Sensitivity Parameters
Use 𝑊 # as a general form of the new weight, 𝑊 as the initial weight
Effect Size of Bias in Identifying NIE: 𝜎𝜌
Effect Size of Bias in Identifying NDE: −𝜎𝜌
𝜎= 𝑣𝑎𝑟 𝑊 # −𝑊|𝑍=1
(reflecting the degree to which the omitted confounder is associated with M given Z and X)
𝜌=𝑐𝑜𝑟𝑟 𝑊 # −𝑊,𝑌|𝑍=1
(reflecting the degree to which the omitted confounder predicts Y in the experimental group given X)

20. Weighting-Based Sensitivity Analysis Procedure
Obtain the initial weight
Obtain a new weight
If type A or type B omission, then include the omitted observed covariates or their high-order forms
If type C omission, then exclude a comparable observed covariate
Compute the values of the sensitivity parameters 𝜎 and 𝜌 each as a function of the weight discrepancy
Compute the effect size of bias 𝜎𝜌 and −𝜎𝜌
Compute the adjusted point estimate and interval estimate of the effect size of NIE and NDE

21. Sensitivity Analysis for NIE
Initial estimate of effect size of NIE: ; 95% CI: (-0.227, 0.006)
Reference values from X1, X2, X3, X1X2, X1X3, X2X3, X1X2X3, X4 (posttreatment)

22. Sensitivity Analysis for NDE
Initial estimate of effect size of NDE: 0.158; 95% CI: (-0.054, 0.371)

23. Relative Advantages of Weighting-Based Sensitivity Analysis
Necessary and coherent for weighting-based causal investigations
No additional simplifying assumptions (nearly a free lunch!)
No increase in sensitivity parameters despite changes in data generation functions
Unconstrained by measurement scales of M, Y, P, and Q
Easy to assess the aggregate bias due to omitting multiple confounders
For assessing nonresponse bias in addition to mediation selection bias
Easy to combine the bias from different sources
Easy to assess potential bias associated with posttreatment as well as pretreatment covariates

24. Extensions to ATT, ATE, Multisite studies…
Hong, G., Qin, X., & Yang, F. (2018). Weighting-based sensitivity analysis in causal mediation studies. Journal of Educational and Behavioral Statistics, 43(1),
“rmpw” package in R:
Make the tool “interactive” with users for assisting with analytic decision-making by asking for:
Omitted pretreatment confounders
Omitted posttreatment confounders
Observed confounders comparable to unobserved ones in terms of plausible values of 𝜎 and 𝜌
Extensions to ATT, ATE, Multisite studies…