Research Design: Alan Monroe Chapter 3

The Concept of Causality (31) Casuality The types of research designs reviewed here are all intended to test whether one variable causes another or causes variation in another.

Three (3) Requirements of Causality (31-32) Correlation: two things tend to occur at the same time (not sufficient to est. causation) Examples: Whenever there is a foreign policy crisis, presidential popularity increases. If Catholic, then more likely to oppose abortion. Time Order: cause has to happen before the effect. Non-spuriousness: to make sure any correlation we observe between the independent and dependent variables is not caused by other factors.

Types of Research Designs (32) Experimental Design It involves a group of subjects (units of analysis), which is divided into two groups (randomly, to assure they are identical on the DV). Experimental and Control Groups The first group is the experimental group, the second is the control group. The experimental group receives a stimulus (the Independent Variable), the control does not. Post-Test A Post-Test is then given to both groups to test the effect (DV) of the stimulus (IV). You then compare the results. …

Obama Ad Strategy AFL-CIO Example: Experimental Research Design: Agenda Mailer Ohio Union Members IV: (Cause): Mailer Sample: Units of Analysis (randomly assigned) Union Mailer No Mailer Support McCain (38%) Post-test: DV: (Effect): Opinion (Poll) Support McCain (37.6%) Compare Results Experimental Group Control Group Time (Controlled Environment)

Experimental Design

Obama Ad Strategy AFL-CIO Example: Experimental Research Design: Agenda Mailer 1) Correlation: a post-test of both the experimental grp and the control grp would capture any variation on the DV (Support for McCain) that may have been caused by the IV (Mailer). 2) Time Order: Given that it is a controlled experiment, we could be certain that the IV, or cause was experienced before the post-test on the DV was administered 3) Non-Spuriousness: Given that it is a controlled experiment, and we can limit the impact of any outside variables as well as reasonably assume that the randomly selected experimental and control grp were essentially the same prior to experiment. Ohio Union Members IV: (Cause): Mailer Sample: Units of Analysis (randomly assigned) Union Mailer No Mailer Support McCain (38%) Post-test: DV: (Effect): Opinion (Poll) Support McCain (37.6%) Compare Results Experimental Group Control Group Time (Controlled Environment)

Obama Ad Strategy AFL-CIO Example: Experimental Research Design: Agenda Mailer 1) Correlation: a post-test of both the experimental grp and the control grp would capture any variation on the DV (Support for McCain) that may have been caused by the IV (Mailer). 2) Time Order: Given that it is a controlled experiment, we could be certain that the IV, or cause was experienced before the post-test on the DV was administered 3) Non-Spuriousness: Given that it is a controlled experiment, and we can limit the impact of any outside variables as well as reasonably assume that the randomly selected experimental and control grp were essentially the same prior to experiment. Ohio Union Members IV: (Cause): Mailer Sample: Units of Analysis (randomly assigned) Union Mailer No Mailer Support McCain (38%) Post-test: DV: (Effect): Opinion (Poll) Support McCain (37.6%) Compare Results Experimental Group Control Group Time (Controlled Environment)

Obama Ad Strategy AFL-CIO Example: Experimental Research Design: Agenda Mailer 1) Correlation: a post-test of both the experimental grp and the control grp would capture any variation on the DV (Support for McCain) that may have been caused by the IV (Mailer). 2) Time Order: Given that it is a controlled experiment, we can be certain that the IV, or cause was experienced before the post-test on the DV was administered 3) Non-Spuriousness: Given that it is a controlled experiment, and we can limit the impact of any outside variables as well as reasonably assume that the randomly selected experimental and control grp were essentially the same prior to experiment. Ohio Union Members IV: (Cause): Mailer Sample: Units of Analysis (randomly assigned) Union Mailer No Mailer Support McCain (38%) Post-test: DV: (Effect): Opinion (Poll) Support McCain (37.6%) Compare Results Experimental Group Control Group Time (Controlled Environment)

Obama Ad Strategy AFL-CIO Example: Experimental Research Design: Agenda Mailer 1) Correlation: a post-test of both the experimental grp and the control grp would capture any variation on the DV (Support for McCain) that may have been caused by the IV (Mailer). 2) Time Order: Given that it is a controlled experiment, we could be certain that the IV, or cause was experienced before the post-test on the DV was administered 3) Non-Spuriousness: Given that it is a controlled experiment, and we can limit the impact of any outside variables as well as reasonably assume that the randomly selected experimental and control grp were essentially the same prior to experiment. Ohio Union Members IV: (Cause): Mailer Sample: Units of Analysis (randomly assigned) Union Mailer No Mailer Support McCain (38%) Post-test: DV: (Effect): Opinion (Poll) Support McCain (37.6%) Compare Results Experimental Group Control Group Time (Controlled Environment)

Examples of Experimental Design (33) Introduction to American Government Example: Does it Increase Political interest? Hypothesis: taking course increases political interest in college students. …

Problems With Experimental Design Hard to get representative samples (hard to get accurate sample of an entire population, one solution is to reduce size of population: college students for example.) Artificial Setting (does it test real behavior?) Outside influences (you can never fully isolate subjects from other variables.) Ethical considerations (cannot mistreat or expose humans to harmful stimuli.) …

The Quasi Experimental (Natural Experiment) (37) Quasi-Experimental It is also called the before and after test: you compare the DV (a Pretest and Posttest) before and after the IV has been applied. Differs from experimental design in several ways: 1. Groups are not assigned (we observe some happen, and then go back and sort into experimental and control groups.) 2. Requires a Pretest of DV so amount of change can be measured. …

Quasi-Experimental Research Quasi-Experimental Research Design: State of the Union and Obama Correlation: change between pretest and post-test has to be significant (indicating IV had an effect), time factor. Time Order: includes measure of DV before and after IV. Non–spuriousness: effect of all outside forces is theoretically equal on all subjects. (they are all exposed to same amount of TV ads, thus any changes comes from the IV). Ohio Union Members DV: (Effect): Pre-test Units of Analysis (NOT randomly assigned) Opinion of Obama DV: (Effect): Post-test Opinion of Obama Compare Results Experimental Group Control Group Watched SOU Did NOT Watched SOU Did NOT Watched SOU IV: (Cause): Pre-test

Quasi-Experimental Research Quasi-Experimental Research Design: State of the Union and Obama Correlation: change between pretest and post-test has to be significant (indicating IV had an effect), time factor. Time Order: includes measure of DV before and after IV. Non–spuriousness: effect of all outside forces is theoretically equal on all subjects. (they are all exposed to same amount of TV ads, thus any changes comes from the IV). Ohio Union Members DV: (Effect): Pre-test Units of Analysis (NOT randomly assigned) Opinion of Obama DV: (Effect): Post-test Opinion of Obama Compare Results Experimental Group Control Group Watched SOU Did NOT Watched SOU Did NOT Watched SOU IV: (Cause): Pre-test Time

Quasi-Experimental Research Quasi-Experimental Research Design: State of the Union and Obama Correlation: change between pretest and post-test has to be significant (indicating IV had an effect) Time Order: includes measure of DV before and after IV. Non–spuriousness: effect of all outside forces is theoretically equal on all subjects. (they are all exposed to same amount of TV ads, thus any changes comes from the IV). Ohio Union Members DV: (Effect): Pre-test Units of Analysis (NOT randomly assigned) Opinion of Obama DV: (Effect): Post-test Opinion of Obama Compare Results Experimental Group Control Group Watched SOU Did NOT Watched SOU Did NOT Watched SOU IV: (Cause): Pre-test Time

Quasi-Experimental Research Quasi-Experimental Research Design: State of the Union and Obama Correlation: change between pretest and post-test has to be significant (indicating IV had an effect) Time Order: includes measure of DV before and after IV. Non–spuriousness: effect of all outside forces is theoretically equal on all subjects. (they are all exposed to same amount of TV ads, thus any changes comes from the IV). Ohio Union Members DV: (Effect): Pre-test Units of Analysis (NOT randomly assigned) Opinion of Obama DV: (Effect): Post-test Opinion of Obama Compare Results Experimental Group Control Group Watched SOU Did NOT Watched SOU Did NOT Watched SOU IV: (Cause): Pre-test Time OUTSIDEWORLDOUTSIDEWORLD

Quasi Experimental Design

Experimental: 2014 State of the Union Example Hypothesis: watching the State of the Union address will improve the public’s opinion of how well Obama has handled the economy. Subjects: students in a class Pretest: before SOU give them a survey measuring their attitudes about the candidates Post-test: did they watch the debate, and what is the strength of their preference.

Quasi-Experimental: 2011 State of the Union Example Hypothesis: watching the State of the Union address will improve the public’s opinion of how well Obama has handled the economy. Subjects: students in a class Pretest: before SOU give subjects a survey measuring their attitudes about the president’s handling of the economy. Post-test: measure the strength of their preference after SOU. Note: no control group. Why?

Quasi Experimental: Presidential Debate Example (38) Hypothesis: watching the State of the Union address will improve the public’s opinion of how well Obama has handled the economy. Subjects: students in a class Pretest: before SOU give them a survey measuring their attitudes about the candidates Post-test: did they watch the debate, and what is the strength of their preference.

Quasi Experimental: Presidential Debate Example (38) Hypothesis: watching a presidential debate increases intensity of support for the candidate. Subjects: students in a class Pretest: before debate give them a survey measuring their attitudes about the candidates Post-test: did they watch the debate, and what is the strength of their preference.

Meeting Conditions of Causality: Quasi Experimental (38) Correlation: change between pretest and post-test has to be significant (indicating IV had an effect) Time Order: includes measure of DV before and after IV. Non–spuriousness: effect of all outside forces is theoretically equal on all subjects. (they are all exposed to same amount of TV ads, thus any changes comes from the IV). …

Correlational Design (40) It is very simple: collecting data on the IV and DV in order to see if there is a pattern or relationship. It is the most common design in political science. Examples: Turnout in Urban Areas IV: urbanization DV: voter turnout Operational definitions: Urbanization: percentage of pop. Living in “urban places,” according to US Census. Turnout: votes cast divided by voting-age population.

Correlational Design

Correlational Design: Examples What impact does race, region or class have on voter turnout? Race: M-C, C-ED, BW, City Voter Turnout M-C, C-ED, WW, City Voter Turnout Region: M-C, C-ED, BW, City Voter Turnout M-C, C-ED, BW, Suburbs Voter Turnout Class: M-C, C-ED, BW, City Voter Turnout W-C, C-ED, BW, City Voter Turnout IV: CauseDV: Effect?

Correlational Design: Examples What impact does race rate of low-birth weight babies born in the US? Education and Class: M-C, C-ED, BW, Suburbs RLBW 1/100 M-C, NC-ED, BW, Suburbs RLBW 1/100 W-C, N-ED, BW, Suburbs RLBW 1/100 Race and Class: M-C, C-ED, BW, Suburbs RLBW 1/100 M-C, C-ED, WW, Suburbs RLBW 1/1000 W-C, C-ED, WW, Suburbs RLBW 1/500 Region: M-C, C-ED, BW, B-USA RLBW 1/100 M-C, C-ED, BW, NB-USA RLBW 1/1000 M-C, C-ED, BW, NB-USA, US 6 months RLBW 1/100 IV: CauseDV: Effect

Meeting Conditions of Causality: Correlational Design (38) Correlation: is directly tested between the IV and DV. Time Order: it is weakest here: there is no consideration for the point in time when the IV and DV occurred. Have to reliable on IV that are known to exist before DV, like race, gender. Non-spuriousness: considers control variables. …