CAUSAL INFERENCE Presented by: Dan Dowhower Alysia Cohen H 615 Friday, October 4, 2013

Goals of Presentation  Review the purpose of Experimentation  Discuss Key Concepts  Discuss types of experiments and their role in science  Group Activity  Campbell Causal Model  The Nature of Good Design

Key Concepts  Cause  Effect and the Counterfactual Model  Logic of Causal Relationships  Criteria for Causal Inference  Approaches to Understanding Causal Relationships  Experiment  Goals of Experiments

Cause  Cause  “is that which makes other things, either simple idea, substance, or mode, begin to be”  It is not deterministic but only increase the probability that an effect will occur  It is context dependent and therefore has implications for generalization  “Inus condition”  an insufficient but non-redundant part of an unnecessary but sufficient condition”  Insufficient – cannot make other things happen without other conditions  Non-redundant – adds something uniquely different from what other factors can contribute  Sufficient – can make other things happen in combination with other conditions

Effect and Counterfactual Model  Effect  “is that, which had its beginning from some other thing”  Is the difference between what did happen and what would have happened.  Question:  Is it possible to develop an accurate estimate of what would have happened in the absence of an intervention?  Counterfactual Model  The process of to comparing the observed results to those you would expect if the intervention had not been implemented.

Logic of Causal Relationship 1.Manipulate the presumed cause and observe an outcome afterward 2.Determine whether variations in the cause is related to variation in the effect 3.Use various methods during the experiment to reduce the plausibility of other explanations for the effect

Criteria for Causal Inference 1.Causal Relativity 2.Causal Manipulation 3.Temporal Ordering 4.Elimination of Alternative Explanations

The Experiment According to Campbell and Stanley (1965) is:  The only means for settling…disputes  The only way of verifying…improvement  The only way of establishing a cumulative tradition for introducing improvement without the…discard of old wisdom

The Goals of Experimentation  To establish the validity of a hypothesis  To develop a well-vetted research design  To create procedures amenable to repetition  To explore a cause that can be manipulated  To contribute knowledge illuminating the nature of a causal relationship

Approaches to Understanding Causal Relationships  Simple Observation  “the action or process of observing something or someone carefully in order to gain information”  Correlational Study  Natural Experiment  Quasi-Experiment  Randomized Control Design

Non-Experimental Approaches  Key informant: asking experts to predict what would have happened in the absence of the intervention.  Establishing a baseline: using the baseline as an estimate of the counterfactual.

Quasi-Experimental Approach  Pre-Post test: comparing the before-and-after difference for the group receiving the intervention to those who did not (no random assignment)  Matching: matching participants (individuals, organizations or communities) with a non-participant on variables that are thought to be relevant.  Propensity Scores: statistically creating comparable groups based on an analysis of the factors that influenced people’s propensity to participate in the program (propensity scores).  Regression Discontinuity: comparing the outcomes of individuals just below the cut-off point with those just above the cut-off point.  Statistical Modeling: develop a model to estimate what would have happened in the absence of an intervention.

Randomized Control Design  Create a control group and compares this to one or more treatment groups to produce an unbiased estimate of the net effect of the intervention.

Randomized Experiment Activity  Previously, you were randomly selected for participation in a research study for Vitamin D.  You have since provided information about the study and informed consent.  You have agreed to the protocol and informed to return next week for your first session.  In the meantime, the investigators have compiled the sample and are ready to perform randomized selection and assignment of participants to the Group A (control) or Group B (treatment).

 How closely do the two groups relate in regards to the participant demographics? Did the relationship improve/worsen with each round of assignment?  What can be inferred about construct validity for the unit construct (people)?  What can we infer about external validity based on the two populations of the study (and the various attempts to randomize the sample)? Randomized Experiment Activity

 Groups have been formed and the study has begun. Consider the following common events and their impact on causal inference: (1) Two people from one Group B (treatment) drop out after completing 3 of the 15 scheduled treatments. (2) Later an additional participant (control) missed the 3 rd and 4 th sessions then return for the 5 th session. (3) Communication between participants in the different groups occurs in which the participant in the control group requests to receive the treatment. Randomized Experiment Activity

 Randomized Experiment considered a “gold standard” but not always the best or most appropriate design.  Campbell’s Causal Model (CCM)  Design a study to reduce the number of plausible rival hypotheses.  Minimize as many from the start by generating a strong study design then after a study assess the remaining threats. Is there one best design?

“CCM stresses careful selection and addition of design features that can reduce the plausibility of a contextually important threat to validity, that can increase the comparability of treatment and comparison groups, and that can replace assumptions with data.” (Shandish, 2007) Campbell’s Causal Model

 Part I: CCM Core  Validity typology Internal Validity External Validity Statistical Conclusion Validity Construct Validity  Part II: Threats to Validity  Rival Hypotheses  Part III: Use of validity types and threats to analyze and prevent likely inferential problems in the design of cause- probing studies. CCM Components

The Nature of Good Design  Theory-Grounded.  Situational -reflect the settings of the investigation.  Feasible – can be carried out with fidelity  Redundant. multiple replications of a treatment.  Efficient – cost effective and rules out threats to validity  Research is like sewing together patches in a quilt and supports our professional obligation to “prevent disillusionment with experimentation”