The Fundamentals of Political Science Research, 2nd Edition

Slides:



Advertisements
Similar presentations
Copyright © 2010, 2007, 2004 Pearson Education, Inc. Chapter 13 Experiments and Observational Studies.
Advertisements

Who are the participants? Creating a Quality Sample 47:269: Research Methods I Dr. Leonard March 22, 2010.
Chapter 13: Experiments and Observational Studies
Chapter 5 Research Methods in the Study of Abnormal Behavior Ch 5.
Chapter 1: Introduction to Statistics
Copyright © 2010 Pearson Education, Inc. Chapter 13 Experiments and Observational Studies.
Experiments and Observational Studies. Observational Studies In an observational study, researchers don’t assign choices; they simply observe them. look.
Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 13 Experiments and Observational Studies.
Module 4 Notes Research Methods. Let’s Discuss! Why is Research Important?
AP Statistics.  Observational study: We observe individuals and measure variables of interest but do not attempt to influence responses.  Experiment:
STUDYING BEHAVIOR © 2009 The McGraw-Hill Companies, Inc.
Chapter 3.1.  Observational Study: involves passive data collection (observe, record or measure but don’t interfere)  Experiment: ~Involves active data.
METHODS IN BEHAVIORAL RESEARCH NINTH EDITION PAUL C. COZBY Copyright © 2007 The McGraw-Hill Companies, Inc.
The Practice of Statistics, 5th Edition Starnes, Tabor, Yates, Moore Bedford Freeman Worth Publishers CHAPTER 4 Designing Studies 4.2Experiments.
Methodology: How Social Psychologists Do Research
Statistics notes 8/10/14 Bell Ringer: How could you organize an experiment so that the results showed something completely different from the truth?
Research in Psychology Chapter Two 8-10% of Exam AP Psychology.
Experiments Textbook 4.2. Observational Study vs. Experiment Observational Studies observes individuals and measures variables of interest, but does not.
Copyright ©2011 Brooks/Cole, Cengage Learning Gathering Useful Data for Examining Relationships Observation VS Experiment Chapter 6 1.
Psy B07 Chapter 1Slide 1 BASIC CONCEPTS. Psy B07 Chapter 1Slide 2  Population  Random Sampling  Random Assignment  Variables  What do we do with.
Copyright © 2009 Pearson Education, Inc. Chapter 13 Experiments and Observational Studies.
Chapter 1 Section 1.
Get out your homework and materials for notes!
Chapter 11: The Nuts and Bolts of one-factor experiments.
CHAPTER 4 Designing Studies
CHAPTER 4 Designing Studies
Selecting the Best Measure for Your Study
The Science of Social Psychology
Reasoning in Psychology Using Statistics
MODULE 2 Myers’ Exploring Psychology 5th Ed.
Statistics: Chapter 1.
Political Science 30 Political Inquiry
Take-home quiz due! Get out materials for notes!
CHAPTER 4 Designing Studies
CHAPTER 11 Inference for Distributions of Categorical Data
Observational Studies and Experiments
Chapter 13- Experiments and Observational Studies
What are the key elements of maths that you need to focus on
2 independent Groups Graziano & Raulin (1997).
Making Causal Inferences and Ruling out Rival Explanations
Experiments and Observational Studies
Tuesday, October 25, 2016 Warm-up
The Fundamentals of Political Science Research, 2nd Edition
CHAPTER 4 Designing Studies
Chapter 9: The Political Process Section 4: The Electoral Process.
Designing Experiments
Experimental Design.
Experimental Design.
Research in Psychology
CHAPTER 11 Inference for Distributions of Categorical Data
CHAPTER 4 Designing Studies
Do Now- Identify the sampling method (Cluster or stratified)
Two-Sample Between-Subjects Experiments and Independent-Samples t-Tests So far, we’ve talked about experiments in which we needed to take only one sample.
Statistical Reasoning December 8, 2015 Chapter 6.2
Experimental Design: The Basic Building Blocks
CHAPTER 4 Designing Studies
CHAPTER 11 Inference for Distributions of Categorical Data
Chapter 3 The Idea Of Causation on Social Research
CHAPTER 4 Designing Studies
CHAPTER 4 Designing Studies
CHAPTER 11 Inference for Distributions of Categorical Data
CHAPTER 4 Designing Studies
Research Methods & Statistics
1.1 – Social Science Research Methods
CHAPTER 11 Inference for Distributions of Categorical Data
CHAPTER 4 Designing Studies
Reasoning in Psychology Using Statistics
CHAPTER 4 Designing Studies
If you have your parent letter, please turn in at my desk (scissors on my desk). Get out your homework and materials for notes!
Chapter Ten: Designing, Conducting, Analyzing, and Interpreting Experiments with Two Groups The Psychologist as Detective, 4e by Smith/Davis.
Presentation transcript:

The Fundamentals of Political Science Research, 2nd Edition Chapter 4: Research Design

Chapter 4 Outline Comparison as the key to establishing causal relationships Experimental research designs Observational studies (in two flavors)

What is being compared to what? Recall from our discussions of the examples in Chapter 3 that making good comparisons is one of the keys of doing social science. The simple bivariate comparison of participants in a Head Start program to those who are not in the program, despite its initial appeals, can be very misleading. (Why?) If the comparisons we make are faulty, then our conclusions about whether or not a causal relationship is present are also likely to be faulty. (But we're never sure one way or the other.) What to do?

Research design So let's say that we have a theory that says that some X causes some Y . We don't know whether, in reality, X causes Y . We may be armed with a theory that suggests that X does, indeed, cause Y , but theories can be (and often are) wrong or incomplete. So how do scientists generally, and political scientists in particular, go about testing whether X causes Y ? There are several strategies, or research designs that researchers can employ toward that end. The goal of all types of research designs are to help us evaluate how well a theory fares as it makes its way over the four causal hurdles--that is, to answer as conclusively as is possible the question about whether X causes Y .

Two approaches We will talk about two broad approaches to designing research. The first is called an experimental design, and it is the benchmark of scientific research. The second is meant to emulate the first, and is called an observational study. We'll take them in turn.

An example from day-to-day political life Suppose that you were a candidate for political office locked in what seems to be a tight race. You're deciding whether or not to make some television ad buys for a spot that sharply contrast your record with your opponent's. The campaign manager has had a public-relations firm craft the spot, and has shown it to you in your strategy meetings. You like it, but you look to your staff and ask the bottom-line question: “Will the ad work with the voters?”

Can you see the causal question? Exposure to a candidate's negative ad (X) may, or may not, affect a voter's likelihood of voting for that candidate (Y ). And it is important to add here that the causal claim has a particular directional component to it; that is, exposure to the advertisement will increase the chances that a voter will choose that candidate. How might researchers in the social sciences evaluate such a causal claim?

Think about the comparison How can we most effectively make a comparison to answer our causal question? It is very important, and not at all surprising, to realize that voters may vote for or against you for a variety of reasons (Zs) that have nothing to do with exposure to the ad--varying socioeconomic statuses, varying ideologies, and party identifications can all cause voters to favor one candidate over another. So how can we establish whether or not, among these other influences (Z), the advertisement (X) also causes voters to be more likely to vote for you (Y )?

The possibly confounding effects of political interest in the advertisement-vote intention relationship

An experiment The word “experiment” has many uses in the English language, but in this class we'll use it in a rather precise (picky?) way. An experiment is a research design in which the researcher both controls and randomly assigns values of the independent variable to the subjects. These two components--control and random assignment--form a necessary and sufficient definition of an experiment.

Control What does it mean to say that a researcher “controls” the value of the independent variable that the subjects receive? It means, most importantly, that the values of the independent variable that the subjects receive are not determined either by the subjects themselves, or by nature. In our campaign-advertising example, this requirement means that we cannot compare people who, by their own choice, already view the ad to those who do not (in this case the choice of whether or not to view the ad is a Z variable that may exert an influence on Y separate from X). It means that we, the researchers, have to decide which of our experimental subjects will view the and and which ones will not.

Random assignment We, the researchers, not only must control the values of the independent variable, but we must also assign those values to subjects randomly. In the context of our campaign-ad example, this means that we must toss coins, draw numbers out of a hat, use a random-number generator, or some other such mechanism to ensure that our subjects are divided into a treatment group (who will view the ad) and a control group (who will not view it, but will instead presumably watch something innocuous).

Experiments and internal validity How do experiments help us cross the four hurdles? Take them one at a time. 1 Is there a credible causal mechanism that connects X to Y ? 2 Can we rule out the possibility that Y could cause X? 3 Is there covariation between X and Y ? 4 Have we controlled for all confounding variables Z that might make the association between X and Y spurious? Because experiments deal with the fourth hurdle so effectively, they are said to have high degrees of internal validity -- that is, the inferences we make about whether X causes Y or not are likely to be correct.

Drawbacks to experiments 1 Can we really assign X to subjects? 2 What about external validity? Samples of convenience and replication External validity of the stimulus 3 Are there ethical considerations? 4 The mistake of emphasis

If not an experiment, then what? If we cannot evaluate causal theories in a controlled setting like an experiment, we have to take the world as it already is, and use what are called observational studies. Definition: An observational study is a research design in which the researcher does not have control over values of the independent variable, which occur naturally. However, it is necessary that there be some degree of variability on the independent variable between cases, as well as variation in the dependent variable. Some maintain that, in the absence of experiments, we cannot demonstrate causality with any degree of confidence, but only correlation. We disagree.

Observational studies and the four causal hurdles How do the four causal hurdles change? Not at all. Hurdles 1 and 3 are rather similar between experiments and observational studies. Hurdles 2 and 4 are a bit different, though. How so?

Two types of data, two types of observational studies What is the unit of observation? (individuals vs. aggregates) Two types of data sets Two types of observational studies, focusing on two different types of variation

Variation through space at one point in time

Variation through time in one spatial unit

Cross-sectional observational studies A cross-sectional observational study examines a cross-section of social reality, focusing on variation between individual spatial units -- again, like citizens, elected officials, voting districts, or countries--and explaining the variation in the dependent variable across them. Example: What, if anything, is the connection between the preferences of the voters from a district (X) and a representative's voting behavior (Y )? We could compare the aggregated preferences of voters from a variety of districts (X) to the voting records of the representatives (Y ). This particular X is not at all subject to experimental manipulation.

Time-series observational studies In the time-series observational study, political scientists typically examine the variation within one spatial unit over time. For example, how, if at all, do changes in media coverage about the economy (X) affect public concern about the economy (Y )? That is, when the media spend more time talking about the potential problem of inflation, does the public show more concern about inflation; and when the media spend less time on the subject of inflation, does public concern about inflation wane? We need to focus hard on that fourth causal hurdle. Are there any other variables (Z) that are related to the varying volume of news coverage about inflation (X) and public concern about inflation (Y )?

How many variables do I need to control for? All of them, or substantial problems arise. How is this done? By the use of statistical controls.