What is randomization and how does it solve the causality problem? 2.3.

Slides:

Advertisements

Similar presentations

Introduction to Psychology

Advertisements

Presented by: Tom Chapel Focus On…Thinking About Design.

Significance Testing.  A statistical method that uses sample data to evaluate a hypothesis about a population  1. State a hypothesis  2. Use the hypothesis.

#ieGovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 Measuring Impact 1 Non-experimental methods 2 Experiments Vincenzo Di Maro Development.

The World Bank Human Development Network Spanish Impact Evaluation Fund.

Experimental Design making causal inferences. Causal and Effect The IV precedes the DV in time The IV precedes the DV in time The IV and DV are correlated.

AGEC 608 Lecture 11, p. 1 AGEC 608: Lecture 11 Objective: Provide overview of how “demonstrations” are typically used in deriving benefits and costs of.

Impact Evaluation: The case of Bogotá’s concession schools Felipe Barrera-Osorio World Bank 1 October 2010.

Quantitative Methods Chapter 4 Fall 20101Community Research Methods (Chapter 4)

Types of Studies Observational Study –Observes and measures characteristics without trying to modify the subjects being studied Experiment –Impose a treatment.

Association vs. Causation

Cross-Country Workshop for Impact Evaluations in Agriculture and Community Driven Development Addis Ababa, April 13-16, 2009 AIM-CDD Using Randomized Evaluations.

3-6 6 th grade math Sampling Methods. Objective To understand how the method of sampling determines how representative the sample is of the population.

Learning Objectives 1 Copyright © 2002 South-Western/Thomson Learning Primary Data Collection: Experimentation CHAPTER eight.

Impact Evaluation in the Real World One non-experimental design for evaluating behavioral HIV prevention campaigns.

AADAPT Workshop Latin America Brasilia, November 16-20, 2009 Non-Experimental Methods Florence Kondylis.

Measuring Impact: Experiments

Global Workshop on Development Impact Evaluation in Finance and Private Sector Rio de Janeiro, June 6-10, 2011 Mattea Stein Quasi Experimental Methods.

Quasi Experimental Methods I Nethra Palaniswamy Development Strategy and Governance International Food Policy Research Institute.

S-005 Intervention research: True experiments and quasi- experiments.

Chapter 1 DATA AND PROBLEM SOLVING. Section 1.1 GETTING STARTED.

Experimental Design All experiments have independent variables, dependent variables, and experimental units. Independent variable. An independent.

Evaluating Job Training Programs: What have we learned? Haeil Jung and Maureen Pirog School of Public and Environmental Affairs Indiana University Bloomington.

Decision Making with Data Section 8.4. Evaluate data collection procedures Sample size Random assignment Validity –Did the test measure what it was supposed.

Collection of Data Chapter 4. Three Types of Studies Survey Survey Observational Study Observational Study Controlled Experiment Controlled Experiment.

Section 5.1 Designing Samples Malboeuf AP Statistics, Section 5.1, Part 1 3 Observational vs. Experiment An observational study observes individuals.

Africa Impact Evaluation Program on AIDS (AIM-AIDS) Cape Town, South Africa March 8 – 13, Causal Inference Nandini Krishnan Africa Impact Evaluation.

Rigorous Quasi-Experimental Evaluations: Design Considerations Sung-Woo Cho, Ph.D. June 11, 2015 Success from the Start: Round 4 Convening US Department.

A Survey is a study of one or more characteristics of a group. A Survey is a study of one or more characteristics of a group.

Experiments Main role of randomization: Assign treatments to the experimental units. Sampling Main role of randomization: Random selection of the sample.

AFRICA IMPACT EVALUATION INITIATIVE, AFTRL Africa Program for Education Impact Evaluation David Evans Impact Evaluation Cluster, AFTRL Slides by Paul J.

Nigeria Impact Evaluation Community of Practice Abuja, Nigeria, April 2, 2014 Measuring Program Impacts Through Randomization David Evans (World Bank)

EXPERIMENTAL EPIDEMIOLOGY

EXPERIMENT VS. CORRELATIONAL STUDY. EXPERIMENT Researcher controls all conditions Experimental group – 1 or more groups of subjects Control group – controlled.

Applying impact evaluation tools A hypothetical fertilizer project.

Module 3: Research in Psychology Learning Objectives What is the scientific method? How do psychologist use theory and research to answer questions of.

Introduction to Social Psychology

An introduction to Impact Evaluation and application to the Ethiopia NFSP Workshop on Approaches to Evaluating The Impact of the National Food Security.

Measuring Impact 1 Non-experimental methods 2 Experiments

Diagram of a quasi-experimental design with two groups

Africa Program for Education Impact Evaluation Dakar, Senegal December 15-19, 2008 Experimental Methods Muna Meky Economist Africa Impact Evaluation Initiative.

1 General Elements in Evaluation Research. 2 Types of Evaluations.

Implementing an impact evaluation under constraints Emanuela Galasso (DECRG) Prem Learning Week May 2 nd, 2006.

Randomized Assignment Difference-in-Differences

Bilal Siddiqi Istanbul, May 12, 2015 Measuring Impact: Non-Experimental Methods.

Social Experimentation & Randomized Evaluations Hélène Giacobino Director J-PAL Europe DG EMPLOI, Brussells,Nov 2011 World Bank Bratislawa December 2011.

Global Workshop on Development Impact Evaluation in Finance and Private Sector Rio de Janeiro, June 6-10, 2011 Using Randomized Evaluations to Improve.

1 YOUTHBUILD EVALUATION Building evidence about the effect of YouthBuild on the young people it serves August 17, 2011 Washington, D.C.

4.1 Statistics Notes Should We Experiment or Should We Merely Observe?

Impact Evaluation for Evidence-Based Policy Making Arianna Legovini Lead Specialist Africa Impact Evaluation Initiative.

Measuring causal impact 2.1. What is impact? The impact of a program is the difference in outcomes caused by the program It is the difference between.

S-005 Intervention research: True experiments and quasi- experiments.

Psychology Research Methods. Characteristics of Good Psychological Research © 2002 John Wiley & Sons, Inc.

The Evaluation Problem Alexander Spermann, University of Freiburg 1 The Fundamental Evaluation Problem and its Solution SS 2009.

Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 1 Overview of non- experimental approaches: Matching and Difference in Difference Estimators.

Cross-Country Workshop for Impact Evaluations in Agriculture and Community Driven Development Addis Ababa, April 13-16, Causal Inference Nandini.

1 An introduction to Impact Evaluation (IE) for HIV/AIDS Programs March 12, 2009 Cape Town Léandre Bassolé ACTafrica, The World Bank.

Kenya Evidence Forum - June 14, 2016 Using Evidence to Improve Policy and Program Designs How do we interpret “evidence”? Aidan Coville, Economist, World.

Chapter 11: The Nuts and Bolts of one-factor experiments.

Quasi Experimental Methods I

Quasi Experimental Methods I

©G Dear 2008 – Not to be sold/Free to use

Evaluating research Is this valid research?.

Impact Evaluation Methods: Difference in difference & Matching

Do Now On google classroom.

Scientist 3: John Snow (continued…)

Design of Experiments Jim Bohan Manheim Township School District

Principles of Experimental Design

Title Team Members.

Types of Statistical Studies and Producing Data

Presentation transcript:

What is randomization and how does it solve the causality problem? 2.3

Randomization and causation Participants and nonparticipants are chosen at random There is no reason, other than chance, that they are selected into the program On average, participants and nonparticipants have the same characteristics – they would, on average, have the same outcomes Any difference at the end is due to the program (or chance, to which we will return)

Randomization creates groups with similar characteristics Study sample Treatment group 1 Comparison group Treatment group 2

Random assignment vs. random sampling Random assignment – Units (people, schools etc.) are randomly assigned to different groups (e.g. treatment and comparison) – Creates two or more comparable groups – Basis of randomized evaluation Random sampling – Want to measure the characteristics of a group (e.g. average height) – Measure a random sample of the group – Often used during randomized evaluations, especially group level randomization

Randomly sample from area to get study sample Random sampling

Randomly sample from area to get study area Randomly assign Communities to treatment and comparison Random sampling and Random Assignment Randomly sample Individuals to survey from both treatment and comparison

Steps in random assignment Define those eligible for a program Randomly assign which units are in the treatment and the comparison group Implement the policy or program for the treatment group Compare outcomes for those in treatment and comparison groups

Steps to randomization: 1

Steps to randomization: 2

Steps to randomization: 3

Steps to randomization: 4