The analysis of individual and average causal effects: Basic principles and some applications EUROPEAN SCIENCE FOUNDATION Programme ‘Quantitative Methods.

Slides:



Advertisements
Similar presentations
Multilevel Models with Latent Variables Daniel J. Bauer Department of Psychology University of North Carolina 9/13/04 SAMSI Workshop.
Advertisements

Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Your Name Your Institution.
Holland on Rubin’s Model Part II. Formalizing These Intuitions. In the 1920 ’ s and 30 ’ s Jerzy Neyman, a Polish statistician, developed a mathematical.
The World Bank Human Development Network Spanish Impact Evaluation Fund.
Review of Chapter 11 Team One: Leigh Anne, Javid, Kevin, Mantie and Hugh.
Regression Discontinuity Design Thanks to Sandi Cleveland and Marc Shure (class of 2011) for some of these slides.
Experimental Research Independent variable Dependent variable Causation.
1 Exploring Quasi-Experiments Lab 5: May 9, 2008 Guthrie, J.T., Wigfield, A., & VonSecker, C. (2000). Effects of integrated instruction on motivation and.
Gifts in the treasure chest of Methodology: A personal view Rolf Steyer Friedrich Schiller University Jena Institute of Psychology Department of Methodology.
Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved. Slide 1 Chapter 11 Experimental and Quasi-experimental.
Copyright c 2001 The McGraw-Hill Companies, Inc.1 Chapter 8 Quantitative Research Designs.
1 Exploring Quasi-experiments Lab Assignment #5 Due May 16, 2008 Locate an empirical journal article that employs a quasi-experimental design. Closely.
Application of Ethical Principles During the Informed Consent Process for Clinical Trials Barbara E. Barnes, MD, MS Joanne Russell, MPPM Maurice Clifton,
Designs that allow testing of hypotheses.  Describe pre-experimental, experimental and quasi-experimental research designs.  Explain the types of conclusions.
Human Resources Training and Individual Development February 11: Training Evaluation.
L1 Chapter 11 Experimental and Quasi- experimental Designs Dr. Bill Bauer.
S-005 Types of research in education. Types of research A wide variety of approaches: –Theoretical studies –Summaries of studies Reviews of the literature.
Chapter 2 Research Methods. The Scientific Approach: A Search for Laws Empiricism: testing hypothesis Basic assumption: events are governed by some lawful.
1 Classroom-Based Research: How to Be a Researcher in Your Classroom Basic Skills Initiative Teaching and Learning Workshop October 2009 Darla M. Cooper.
Chapter 2: The Research Enterprise in Psychology
Difference Two Groups 1. Content Experimental Research Methods: Prospective Randomization, Manipulation Control Research designs Validity Construct Internal.
Chapter 2: The Research Enterprise in Psychology
Chapter 2 Research Methods. The Scientific Approach: A Search for Laws Empiricism: testing hypothesis Basic assumption: events are governed by some lawful.
Chapter 2 The Research Enterprise in Psychology. n Basic assumption: events are governed by some lawful order  Goals: Measurement and description Understanding.
Research methods in clinical psychology: An introduction for students and practitioners Chris Barker, Nancy Pistrang, and Robert Elliott CHAPTER 8 Foundations.
Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,
Chapter 11 Experimental Designs
Quasi Experimental Methods I Nethra Palaniswamy Development Strategy and Governance International Food Policy Research Institute.
Non-experimental Quantitative Research Designs (NEQDs)
1 Research Methodology Model. 2 Hypothesis a prediction of what is the case (fact) based on theory Conclusions Observation (s): Phenomena; Problem (Tree)
The Scientific Method in Psychology.  Descriptive Studies: naturalistic observations; case studies. Individuals observed in their environment.  Correlational.
LECTURE 2 EPSY 642 META ANALYSIS FALL CONCEPTS AND OPERATIONS CONCEPTUAL DEFINITIONS: HOW ARE VARIABLES DEFINED? Variables are operationally defined.
Chapter 21 Basic Statistics.
Experimentation in Computer Science (Part 1). Outline  Empirical Strategies  Measurement  Experiment Process.
Quantitative and Qualitative Approaches
What is Computer Science?  Three paradigms (CACM 1/89) Theory (math): definitions, theorems, proofs, interpretations Abstraction (science): hypothesize,
Second Language Classroom Research (Nunan, D. 1990) Assoc. Prof. Dr. Sehnaz Sahinkarakas.
Research MethodsinPsychology The Scientific Method an organized way of using experience and testing ideas to increase knowledge.
Rural Economy Research Centre Understanding farmers’ intentions to convert to organic farming An application of the theory of planned behaviour using structural.
Learning Objectives In this chapter you will learn about the elements of the research process some basic research designs program evaluation the justification.
NSC 440 RESEARCH IN NURSING 4 UNITS DEPARTMENT OF NURSING SCIENCE FACULTY OF BASIC MEDICAL SCIENCES 1.
Impact Evaluation Sebastian Galiani November 2006 Causal Inference.
S-005 Types of research in education. Types of research A wide variety of approaches: –Theoretical studies –Summaries of studies Reviews of the literature.
Chapter Eight: Quantitative Methods
EXPERIMENTS Lecture 5. Administrative STATA Course Mailing List Info:   No subject  In body.
1 Joint meeting of ESF Evaluation Partnership and DG REGIO Evaluation Network in Gdańsk (Poland) on 8 July 2011 The Use of Counterfactual Impact Evaluation.
Repetition I  Causes of biases in contingency and causal judgments:  Pattern recognition capabilities.  Belief in the law of small numbers.  Hot hand.
Rerandomization in Randomized Experiments Kari Lock and Don Rubin Harvard University JSM 2010.
Research Design Quantitative. Quantitative Research Design Quantitative Research is the cornerstone of evidence-based practice It provides the knowledge.
Kolmogorov (1933) Stochastic dependence of events [ ( , A, P), P(A  B)] Set (of possible outcomes)  -Algebra (set of possible events) A is a set of.
AP Stats Chapter 7 Review Nick Friedl, Patrick Donovan, Jay Dirienzo.
Dr. Wang Xingbo QQ: Homepage: wxbstudio.home4u.china.com Fall , 2005 Mathematical & Mechanical Method in Mechanical.
Chapter 17 STRUCTURAL EQUATION MODELING. Structural Equation Modeling (SEM)  Relatively new statistical technique used to test theoretical or causal.
Chapter 2: The Research Enterprise in Psychology.
Plausible Values of Latent Variables: A Useful Approach of Data Reduction for Outcome Measures in Pediatric Studies Jichuan Wang, Ph.D. Children’s National.
Chapter 11 Experimental Designs PowerPoint presentation developed by: Sarah E. Bledsoe & E. Roberto Orellana.
Chapter 9 Scrutinizing Quantitative Research Design.
Part Two.
Quasi Experimental Methods I
Quasi Experimental Methods I
Group Quantitative Designs
عنوان عنوان فرعی.
بسم الله الرحمن الرحیم.
Impact evaluation of actions for jobseekers under the current OP ESF- Flemish Community : beyond classical parameters for success Expert Hearing.
Introduction to Econometrics
Impact Evaluation Methods
The Use of Counterfactual Impact Evaluation Methods in Cohesion Policy
Research Design Quantitative.
How to Effectively Communicate Commonly misinterpreted Statistical terms in experimentation AMAZON IPC LAB.
Interpreting & Summarizing
Presentation transcript:

The analysis of individual and average causal effects: Basic principles and some applications EUROPEAN SCIENCE FOUNDATION Programme ‘Quantitative Methods in the Social Sciences’ (QMSS) Seminar: ‘Theory-Driven Evaluation and Intervention Studies in the Social Sciences’, September 2006, Nicosia, Cyprus Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Rolf Steyer University of Jena (Germany) Institute of Psychology Department of Methodology and Evaluation Research

2 Overview Individual and average causal effects (Neyman, Rubin) Motivation: The Simpson Paradox Pre-Post Design with Control Group for the Analysis of Intervention Effects Designs for the Analysis of Individual Causal Effects - Example with an Intervention - Example with Method Effects Conclusions

3 The Simpson Paradox

4

5

6

7

8 The single-unit trial Sample a person u, register her assignment to one of the treatment conditions and observe her outcome y. In this single-unit trial U, X, and Y have a joint distribution u1u1 treatment y1y1 y2y2 y3y3 y4y4 y1y1 y2y2 y3y3 y4y4 control u2u2 treatment control

9 Individual and average causal effects (Neyman, Rubin) Define the true-outcome variables as follows:  0 (u) := E(Y 0  U = u) and  1 (u) := E (Y 1  U = u)  1-0 (u) :=  1 (u)   0 (u) = individual causal effect of unit u ACE 1-0 = E (  1-0 ) = E(  1 ) – E(  0 ) =: average causal effect

10 Individual, conditional and average causal effects (Neyman, Rubin)

11 Bias Theorem

12 Three Design Types Between-Group Designs Pre-post Designs (not used at all in the Neyman-Rubin tradition) Between-Group Designs with Pre-Post Measures (only the between group comparisons are used in the Neyman-Rubin tradition)

13 Utilizing Pre-Post Designs for the Analysis of Individual Effects Pre-post Designs and Between-Group Designs with Pre-Post Measures can be used to analyze not only average but also individual causal effects. The crucial asssumption is that the individual pretest distribution is the same as the individual posttest (outcome variable) distribution under control (no treatment).

14 Theorem 1 (Sufficient conditions for unbiasedness of the (conditional) prima facie effects)

15 Theorem 2

16 Standard research questions in the analysis of causal effects

17 Generalization to J + 1 Treatment Conditions

18 Average effects in the analysis of causal effects

19 Types of covariates

20 Example (using EffectLite): Intelligence training

21 Pre-Post Design with Control Group for the Analysis of Intervention Effects

22 Between-Group Design, no Pretest Y 0 = τ 0 +  0 Y 1 =  0 +  1  τ 0 +  1

23 Pre-Post Design, no Control Group

24 Identified Individual Effects Model with pretests Y 11 and Y 21

25 Identified Individual Effects Model with pretests Y 11 and Y 21

26 Design for the Analysis of Method Effects

27 Introducing Individual Method Effects

28 Introducing Individual Method Effects τ 22  τ 12 = τ 21  τ 11 = IME 2-1

29 Introducing Individual Method Effects τ 22  τ 12 = τ 21  τ 11 = IME 2-1

30 An identifíed Individual-Method-Effects Model τ 22  τ 12 = τ 21  τ 11 = IME 2-1

31 Model in treatment group Treatment

32 Model in control group

33 Model in treatment group (t-values)

34 Model in control group (t-values)

35 Correlation Matrix of ETA in Control group IQ ICE IME IQ 1.00 ICE IME

36 Correlation Matrix of ETA in Experimental Group IQ ICE IME IQ 1.00 ICE IME

37 The effects of negativ item formulation

38 The effects of negativ item formulation

39 The effects of negativ item formulation (standardized)

40 Summary and Conclusion We can analyze individual (and average) causal effects in Pre-post Designs The causal interpretation rests on assumptions These assumptions can be tested Latent variables can be constructed from true-scores Not a single path in our SEM models represented a causal effect

41 Want More? Steyer, R. & Partchev, I.. (2006). Causal Effects in Experiments and Quasi-Experiments: Theory (Chapters 1 -5 are available at effects.de) Symposium on causality in Jena July 7 to 9, 2006 with Tom Cook, Steve West, Don Rubin … (videos available: see Online video of workshop on the analysis of causal effects (same home page) Software „EffectLite“ (see:

42 Thanks to: Sven Hartenstein Ulf Kröhne Benjamin Nagengast Ivailo Partchev Steffi Pohl