Statistical Methods for Observational Studies Lecture 3 (of 4) Steve Fienberg Memorial Lectures Series in Advanced Analytics November 2018 Statistical Methods for Observational Studies Lecture 3 (of 4) Dylan Small University of Pennsylvania Slides posted at my web site: www-stat.wharton.upenn.edu/~dsmall

Unmeasured Confounders IV Assumptions: IV affects the treatment; IV is independent of unmeasured confounders; IV has no direct effect on the outcome (exclusion restriction) Outcome (Mortality) Treatment (C-section vs. vaginal delivery) 3 Measured Confounders 1 IV (Hospital rate of C-section) 2 Unmeasured Confounders

Unmeasured Confounders Subgroup Unaffected by Treatment: Suppose there is a subgroup known to be unaffected by the treatment. If IV affects outcome in this subgroup, then it provides evidence that IV assumptions are violated. Outcome (Mortality) ? Treatment (C-section vs. vaginal delivery) Measured Confounders IV (Hospital rate of C-section) ? Unmeasured Confounders

Testing the Exclusion Restriction Presuming the literature’s claim that C-section has no effect for 30-34 week babies, we tested the exclusion restriction by redoing our study for 30-34 week babies. If literature’s claim is true and exclusion restriction is true, IV should have no effect on mortality for 30-34 week babies. We matched 30-34 babies delivered in high C-section rate and low C-section rate hospitals on the measured covariates as before, producing 23,631 pairs of 30-34 week babies.

Matching Balanced the Measured Covariates for 30-34 Week Babies

Effect of IV on Mortality for 30-34 Week Babies Mortality rates are higher in the low C-section hospitals. Odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital for 30-34 week babies: 0.62 (95% CI: 0.58, 0.69) [Compare with for 23-24 week babies, odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital: 0.51 (95% CI: 0.34, 0.69)].

Aporia Our results: Using hospital C-section rate as an IV, C-sections increase mortality for both 23-24 week babies and older 30-34 week babies. Previous literature, based on adjusting for measured confounders, claims that C-sections do not increase mortality for 30-34 week babies. Each of these claims sound plausible and reasonable if taken one at a time, but they cannot all be correct. We have produced an aporia. (Oxford American Dictionary): “an aporia is an irresolvable [without more data], internal contradiction…in a text, argument or theory” with aporetic as an adjective. A collection of propositions is an aporia if each is plausible on its own but they are jointly inconsistent, that is, is false or implausible.

A special case of an aporia is used when doing a proof by contradiction: One proves by showing that are certainly true and are an aporia.

In contrast, in a typical aporia, the identity of the culpable proposition(s) is unknown. In Plato’s early dialogues, Socrates would invalidate the views of his opponents by showing that those views were an aporia.

Value of Finding an Aporia Continued Socrates, in Plato’s Meno, says of a befuddled young interlocutor who he put in an aporia: At first he did not know what [he thought he knew], and he does not know even now: but at any rate he thought he knew then, and confidently answered as though he knew, and was aware of no difficulty; whereas now he feels the difficulty he is in, and besides not knowing does not think he knows…. [W]e have certainly given him some assistance, it would seem, towards finding out the truth of the matter: for now he will push on in the search gladly, as lacking knowledge; whereas then he would have been only too ready to suppose he was right….   [Having] been reduced to the perplexity of realizing that he did not know… he will go on and discover something.

Value of Finding an Aporia To recognize that one’s beliefs contain an aporia, is an advance in understanding, albeit an uncomfortable one. When one recognizes that one’s beliefs contains an aporia, one recognizes that one harbors at least one false belief, is motivated to identify what the false belief is, and is hesitant in deducing consequences from . This is certainly better than believing without recognizing that they are an aporia. One can avoid an aporia by arbitrarily discarding propositions until the remaining propositions are inconsistent, but there is nothing to ensure that one has discarded false proposition and kept true ones. One has just narrowed one’s beliefs sufficiently to be safe from accusations of inconsistency. For example, one can avoid an aporia in testing the assumptions of an IV by defining those assumptions so narrowly that they become untestable.

Value of Finding Aporia for C-section study The aporia tells us that our beliefs that hospital C-section rate is a valid IV and that C-sections do not affect 30-34 week babies cannot both be true. We are motivated to find out which belief(s) isn’t true. Some Next Steps: Investigate how high C-section rate hospitals and low C-section rate hospitals differ in other aspects of care besides C-sections. Try to better understand important confounders of C-section-mortality relationship and improve the analyses that adjust for measured confounders.

Strong inference and quasi-experiments

Bracketing in the Comparative Interrupted Time-Series Design to Address Concerns about History Interacting with Group: Evaluating Missouri’s Handgun Purchaser Law Raiden Hasegawa University of Pennsylvania Daniel Webster Johns Hopkins University Dylan Small

Repeal of Missouri’s Background Check Law U.S. federal gun law requires background checks and record keeping for gun sales by federally licensed firearm dealers but exempts these regulations for private sales (e.g., gun show sales). However, some states have laws requiring all purchasers of handguns from licensed dealers and private sellers to acquire a permit-to-purchase (PTP) license that verifies the purchaser has passed a background check. Missouri passed a PTP law in 1921, requiring handgun purchasers to obtain a license from the local sheriff’s office which facilitated the background check, but repealed the law on August 28, 2007. Webster et al. (2014) conducted a difference-in-difference study comparing Missouri to border states (Arkansas, Illinois, Iowa, Kansas, Kentucky, Oklahoma, Nebraska and Tennessee).

Difference-in-Difference: Causal Inference Based on the Parallel Trends Assumption

History Interacting with Group History interacting with group: Even if trends are parallel in before period, there could be historical events in the after period that affect the treated and control group differently. History interacting with group may be related to the level of the outcome in the before period. Missouri study: Missouri had higher gun homicide rates than average border state in before period (1999-2007). Recession starting in 2008. Recession followed by downturn in gun homicide rates. Effect of recession on gun homicide rate might depend on starting level of gun homicide rate.

Bracketing for Difference-in-Difference Concern: History interacting with group related to starting level of group. Ideal control would be equal in before period to treated group. In the absence of ideal control, construct two control groups Lower Control Group: lower expected outcome than treated group in before period Upper Control Group: higher expected outcome than treated group in before period. Under assumptions about history interacting with group being related to starting level, expectations of (i) difference-in-difference comparing treated to upper control group and (ii) difference-in-difference comparing treated to lower control group bracket true causal effect.

Model

Assumptions for Bracketing Method

Assumptions for Bracketing Method Continued

Bracketing Proposition

Constructing Upper and Lower Control Groups

Missouri Study

Placebo Study

Summary of Findings from Missouri Study Difference-in-difference finds Missouri PTP repeal associated with increase in gun homicides. Provides evidence for causal effect of Missouri PTP repeal on gun homicides under parallel trends assumption. Bracketing method eliminates history interacting with group due to time-invariant unmeasured confounder having an increased or decreased effect in the after period as a plausible rival explanation to a causal effect.

Strong inference and quasi-experiments

Limitations and Partially Addressing the Limitations

Final Lecture Elaborate Theories and Evidence Factors.