1. Statistical Methods for Observational Studies Lecture 3 (of 4)
Steve Fienberg Memorial Lectures Series in Advanced Analytics
November 2018
Statistical Methods for Observational Studies Lecture 3 (of 4)
Dylan Small
University of Pennsylvania
Slides posted at my web site: www-stat.wharton.upenn.edu/~dsmall

2. Unmeasured Confounders
IV Assumptions:
IV affects the treatment;
IV is independent of unmeasured confounders;
IV has no direct effect on the outcome
(exclusion restriction)
Outcome (Mortality)
Treatment
(C-section vs.
vaginal delivery) 3
Measured Confounders 1
IV (Hospital rate of
C-section) 2
Unmeasured Confounders

3. Unmeasured Confounders
Subgroup Unaffected by Treatment:
Suppose there is a subgroup known to be unaffected by the treatment.
If IV affects outcome in this subgroup, then it provides
evidence that IV assumptions are violated.
Outcome (Mortality) ?
Treatment
(C-section vs.
vaginal delivery)
Measured Confounders
IV (Hospital rate of
C-section) ?
Unmeasured Confounders

4. Testing the Exclusion Restriction
Presuming the literature’s claim that C-section has no effect for week babies, we tested the exclusion restriction by redoing our study for week babies.
If literature’s claim is true and exclusion restriction is true, IV should have no effect on mortality for week babies.
We matched babies delivered in high C-section rate and low C-section rate hospitals on the measured covariates as before, producing 23,631 pairs of week babies.

5. Matching Balanced the Measured Covariates for 30-34 Week Babies

6. Effect of IV on Mortality for 30-34 Week Babies
Mortality rates are higher in the low C-section hospitals.
Odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital for week babies: 0.62 (95% CI: 0.58, 0.69)
[Compare with for week babies, odds of dying from being delivered at a high C-section hospital compared to a low C-section hospital: 0.51 (95% CI: 0.34, 0.69)].

7. Aporia
Our results: Using hospital C-section rate as an IV, C-sections increase mortality for both week babies and older week babies.
Previous literature, based on adjusting for measured confounders, claims that C-sections do not increase mortality for week babies.
Each of these claims sound plausible and reasonable if taken one at a time, but they cannot all be correct.
We have produced an aporia. (Oxford American Dictionary): “an aporia is an irresolvable [without more data], internal contradiction…in a text, argument or theory” with aporetic as an adjective.
A collection of propositions is an aporia if each is plausible on its own but they are jointly inconsistent, that is, is false or implausible.

8. A special case of an aporia is used when doing a proof by contradiction: One proves by showing that are certainly true and
are an aporia.

9. In contrast, in a typical aporia, the identity of the culpable proposition(s) is unknown.
In Plato’s early dialogues,
Socrates would invalidate
the views of his
opponents by showing
that those views were an
aporia.

10. Value of Finding an Aporia Continued
Socrates, in Plato’s Meno, says of a befuddled young interlocutor who he put in an aporia:
At first he did not know what [he thought he knew], and he does not know even now: but at any rate he thought he knew then, and confidently answered as though he knew, and was aware of no difficulty; whereas now he feels the difficulty he is in, and besides not knowing does not think he knows…. [W]e have certainly given him some assistance, it would seem, towards finding out the truth of the matter: for now he will push on in the search gladly, as lacking knowledge; whereas then he would have been only too ready to suppose he was right….   [Having] been reduced to the perplexity of realizing that he did not know… he will go on and discover something.

11. Value of Finding an Aporia
To recognize that one’s beliefs contain an aporia, is an advance in understanding, albeit an uncomfortable one.
When one recognizes that one’s beliefs contains an aporia, one recognizes that one harbors at least one false belief, is motivated to identify what the false belief is, and is hesitant in deducing consequences from
This is certainly better than believing without recognizing that they are an aporia.
One can avoid an aporia by arbitrarily discarding propositions until the remaining propositions are inconsistent, but there is nothing to ensure that one has discarded false proposition and kept true ones. One has just narrowed one’s beliefs sufficiently to be safe from accusations of inconsistency.
For example, one can avoid an aporia in testing the assumptions of an IV by defining those assumptions so narrowly that they become untestable.

12. Value of Finding Aporia for C-section study
The aporia tells us that our beliefs that hospital C-section rate is a valid IV and that C-sections do not affect week babies cannot both be true.
We are motivated to find out which belief(s) isn’t true.
Some Next Steps:
Investigate how high C-section rate hospitals and low C-section rate hospitals differ in other aspects of care besides C-sections.
Try to better understand important confounders of C-section-mortality relationship and improve the analyses that adjust for measured confounders.

13. Strong inference and quasi-experiments

14. Bracketing in the Comparative Interrupted Time-Series Design to Address Concerns about History Interacting with Group: Evaluating Missouri’s Handgun Purchaser Law
Raiden Hasegawa
University of Pennsylvania
Daniel Webster
Johns Hopkins University
Dylan Small

15. Repeal of Missouri’s Background Check Law
U.S. federal gun law requires background checks and record keeping for gun sales by federally licensed firearm dealers but exempts these regulations for private sales (e.g., gun show sales).
However, some states have laws requiring all purchasers of handguns from licensed dealers and private sellers to acquire a permit-to-purchase (PTP) license that verifies the purchaser has passed a background check.
Missouri passed a PTP law in 1921, requiring handgun purchasers to obtain a license from the local sheriff’s office which facilitated the background check, but repealed the law on August 28, 2007.
Webster et al. (2014) conducted a difference-in-difference study comparing Missouri to border states (Arkansas, Illinois, Iowa, Kansas, Kentucky, Oklahoma, Nebraska and Tennessee).

17. Difference-in-Difference: Causal Inference Based on the Parallel Trends Assumption

19. History Interacting with Group
History interacting with group: Even if trends are parallel in before period, there could be historical events in the after period that affect the treated and control group differently.
History interacting with group may be related to the level of the outcome in the before period.
Missouri study:
Missouri had higher gun homicide rates than average border state in before period ( ).
Recession starting in Recession followed by downturn in gun homicide rates.
Effect of recession on gun homicide rate might depend on starting level of gun homicide rate.

20. Bracketing for Difference-in-Difference
Concern: History interacting with group related to starting level of group.
Ideal control would be equal in before period to treated group.
In the absence of ideal control, construct two control groups
Lower Control Group: lower expected outcome than treated group in before period
Upper Control Group: higher expected outcome than treated group in before period.
Under assumptions about history interacting with group being related to starting level, expectations of
(i) difference-in-difference comparing treated to upper control group and
(ii) difference-in-difference comparing treated to lower control group
bracket true causal effect.

21. Model

22. Assumptions for Bracketing Method

23. Assumptions for Bracketing Method Continued

24. Bracketing Proposition

25. Constructing Upper and Lower Control Groups

26. Missouri Study

29. Placebo Study

31. Summary of Findings from Missouri Study
Difference-in-difference finds Missouri PTP repeal associated with increase in gun homicides.
Provides evidence for causal effect of Missouri PTP repeal on gun homicides under parallel trends assumption.
Bracketing method eliminates history interacting with group due to time-invariant unmeasured confounder having an increased or decreased effect in the after period as a plausible rival explanation to a causal effect.

32. Strong inference and quasi-experiments

33. Limitations and Partially Addressing the Limitations

34. Final Lecture
Elaborate Theories and Evidence Factors.