Experiments The Beauty of Random Assignment How Experiments Work Strengths and Weaknesses

The Beauty of Random Assignment Problem: In non-experimental studies, what determines the values that an independent variable takes on? Often, a confounding variable determines these values, and affects the DV. For instance, the confound of “Are you a serious student” may determine where you will sit in a class.

An Example of a Confound from Recent Research For example: Does contacting registered voters through phone calls or visits make them more likely to turn out on election day? Potential Confound: Previous participation records. Campaigns with limited resources concentrate their mobilization efforts on voters who have turned out in the past.

The Beauty of Random Assignment Solution: Interrupt the causal path that leads from the confound to the independent variable by “randomly assigning” the values that the IV takes on in each case. Randomly assign seats so that there are just as many serious students and slackers in each part of the lecture hall. Randomly assigned some voters to be contacted, in order to measure the actual effects of mobilization (which are quite weak).

The Beauty of Random Assignment Treatment Group All of the cases in this group have been assigned one value of the IV (sit in front, take medicine, etc.) Control Group All of the cases have been assigned a different value of the IV (in most cases, left alone or given placebo) In all other respects (including the values of confounding variables that they take on), these groups are similar.

How Experiments Work Step #1: Random Assignment Begin by splitting your cases into two or more groups of 30 or more through a process that is truly random. Using something like a random number generator is key, because many seemingly neutral assignment processes may be correlated with a confounding variable. Examples: arrival times, last names, section times, bleeding hearts.

How Experiments Work Step #2: (Optional) Pre-Test To check how the random assignment process worked, measure the value that the DV takes on for each case before any treatment has been applied. Each group should average about the same values on the dependent variable. Even if something went wrong, we can still learn from the “time-series” comparison. Often it is hard to pre-test.

How Experiments Work Step #3: Apply the Treatment Change the value of the independent variable in the treatment group. Administer the medicine or the placebo, put students in their seats, request that subjects administer an electric shock. This is where ethical issues arise.

How Experiments Work Step #4: Post-Test Measure the value that the DV takes on for each case AFTER the treatment has been applied. Comparing values of the DV in treatment group vs. control group tells us the effect of the treatment, if random assignment worked. Comparing shifts from pre-test to post-test is helpful when random assignment failed.

Schematic of an Experiment Treatment Group (pre-test) Treatment(post-test) RandomCompare Assignment Control Group (pre-test)(post-test)

Strength of Experiments: High Internal Validity Internal validity judges how well a research design has tested a causal relationship, in the cases examined. “Among the cases in our study, do we have reason to believe that IV #1 causes DV? Because random assignment takes away our fear of confounds, experiments have high internal validity.

Weakness of Experiments: Low External Validity External validity judges how confident we can be that a causal relationship identified in our cases can be generalized to the outside world. Our cases may be different than the general population, or our cases may react differently to treatments, or our treatments may be very artificial. You can’t assign every treatment: gender, race.

Studying Sensitive Topics - 1 I. Race – The List Experiment A. One half of a general population, randomly chosen, is asked how many of three items – the federal government increasing the tax on gasoline, professional athletes getting million-dollar salaries and large corporations polluting the environment - make them angry – NOT which items, just how many. Not revealing which items makes people more willing to express socially undesirable answers.

Studying Sensitive Topics The second group gets the same three items plus a fourth item – awarding college scholarship based upon race. 3. If the average of the first group was 2.0 and the average of the second group was 2.6, then it is likely that 60% of the second group selected awarding college scholarships based upon race.

Studying Sensitive Topics - 3 II. Race – The Audit Experiment - Housing A. Units being marketed for sale or rent are randomly selected and separate teams of white and African-American auditors posing as home-seekers are sent to inquire about the availability of the advertised units and the terms under which units might be obtained. Auditors are assigned similar personal, social and economic characteristics.

Studying Sensitive Topics - 4 After each encounter, the auditor fills out a form describing the nature and outcome. African-American auditors were more likely than white auditors: (1) to be told that no units were available; (2) to be shown fewer units; and (3) given higher down payments, rents and interest rates.

Studying Sensitive Topics - 5 B. Phone inquiries offer another opportunity to test for more subtle forms of discrimination. Researchers found white real estate agents were able to accurately identify callers who spoke African-American English from the messages they left. Black “sounding” callers were significantly less likely to (1) get a “call back”; and (2) obtain an appointment to view a home – especially in predominately white areas.