Research Methods in Psychology Independent Groups Designs

Why Psychologists Conduct Experiments To test hypotheses derived from theories effectiveness of treatments and programs Third goal of psychological research explanation examine the causes of behavior

Experimental Research An experiment must include independent variable (IV)‏ dependent variable (DV)‏ An independent variable manipulated (controlled) by experimenter at least two conditions (levels)‏ “treatment” and “control”

Experimental Research dependent variables measured by experimenter used to determine effect of IV In most experiments, researchers measure several dependent variables to assess effect of IV

Example: Body Image Among Young Girls Dittmar, Halliwell, and Ives (2006) Research question Does exposure to very thin body images cause young girls to experience negative feelings about their own body? Independent Variable version of picture book with three levels Barbie (very thin body image)‏ Emme (realistic body image)‏ Neutral (no body images)‏

Body Image Among Young Girls, continued Dependent Variables Several measured body image and body dissatisfaction, including: Child Figure Rating Scale rate perceived actual body shape rate ideal body shape obtain difference score: score of zero: no body shape dissatisfaction positive score: a desire to be bigger negative score: desire to be thinner (body dissatisfaction)‏

Body Image Among Young Girls, continued Dittmar et al.’s hypothesis Young girls who are exposed to the very thin body image (Barbie) will experience greater body dissatisfaction than young girls who are exposed to realistic body images (Emme) or neutral images.

Experimental Control and Internal Validity An experiment has internal validity when we can state confidently that the independent variable caused differences between groups on the dependent variable a causal inference alternative explanations for a study’s findings are ruled out

Control and Internal Validity, continued Example: Suppose young girls who view the Barbie images are more overweight or own more Barbie dolls than girls in the other conditions How do we know viewing the Barbie images in the experiment caused them to experience greater body dissatisfaction? What are some alternative explanations?

Three conditions for causal inference Causal Inferences Three conditions for causal inference Covariation relationship between IV and DV example: young girls’ body dissatisfaction covaried with experimental condition correlation does not imply causation

Causal Inferences, continued Time-order relationship presumed cause precedes the effect example: version of images (cause) was manipulated prior to measuring body dissatisfaction (effect)‏ How can we be sure girls in Barbie condition didn’t have greater body dissatisfaction than the other girls before the manipulation (effect precedes cause)?

Causal Inferences, continued Elimination of plausible alternative causes use control techniques to eliminate other explanations example: if the three groups differ in ways other than the type of images they viewed, these differences are alternative explanations for the study’s findings

Causal Inferences, continued Confoundings when the IV is allowed to covary with a different, potential independent variable confoundings represent alternative explanations for a study’s findings an experiment that is free of confoundings has internal validity

Causal Inferences, continued Example of confounding suppose that after viewing the Barbie images, young girls in this condition are interviewed by a counselor to make sure they’re okay after exposure to the very thin images; as part of this interview, they’re asked specifically about feelings toward their body suppose, too, that young girls in the Emme and Neutral conditions are not interviewed

Causal Inferences, continued What is the confounding? version of images (IV of interest) covaries with interview (present, absent)‏ viewing Barbie images is always paired with interview viewing Emme or neutral images is always paired with no interview alternative explanation for findings cannot be ruled out greater body dissatisfaction in Barbie condition could be explained by interview, not viewing Barbie images Note: This confounding was not present in the Dittmar et al. study

Two control techniques to eliminate alternative explanations holding conditions constant balancing With proper use of control techniques, an experiment has internal validity

Control Techniques, continued Holding conditions constant Independent variable: groups in the different conditions have different experiences example: view Barbie, or Emme, or neutral images Experiences should differ only in terms of the independent variable The only thing we allow to vary across groups are the IV conditions—everything else should be the same for the groups of the experiment

Control Techniques, continued Example of holding conditions constant Dittmar et al. (2006) held constant all the young girls listened to the same story all were given the same instructions all completed the same questions after the story What if only girls in the Barbie condition listened to the story and girls in the other two conditions sat quietly? alternative explanation: listening to a story caused the different outcomes

Control Techniques, continued Balancing some alternative explanations for a study’s findings concern characteristics of participants example What if girls in Barbie condition were more overweight, owned more Barbie dolls, or greater body dissatisfaction even before they viewed the picture books?

Control Techniques, continued Some variables cannot be held constant subjects’ characteristics cannot be held constant participants all have the same body weight same number of Barbie dolls same preexisting levels of body dissatisfaction same everything Balancing controls for alternative explanations due to subject characteristics Goal: make sure that on average, participants (as a group) in each condition are essentially equivalent

Control Techniques, continued How to balance subject characteristics across the levels of the experiment: Participants are assigned to conditions using some random procedure (e.g., two conditions: flip a coin)‏ Random assignment creates, on average, equivalent groups of participants in the experimental conditions Rule out alternative explanations due to subject characteristics

Independent Groups Designs different individuals participate in each condition of the experiment (i.e., no overlap of participants across conditions)‏ three types random groups design matched groups design natural groups design

Individuals are randomly assigned to conditions of the IV Random Groups Designs Individuals are randomly assigned to conditions of the IV Groups of participants are equivalent, on average, before the IV manipulation Any differences between groups on dependent variable are caused by independent variable (if conditions are held constant)‏ Dittmar et al. (2006) study used a random groups design

Random Groups Designs, continued Block randomization A “block” is a random order of all conditions in the experiment Example: a random order of conditions A, B, C could be B C A 1st participant assigned to condition B 2nd participant—condition C 3rd participant—condition A Generate random orders until goal for number of participants in each condition is met (e.g., 10 in each condition)‏

Random Groups Designs, continued Advantages of block randomization creates groups of equal size for each condition controls for time-related events that occur during course of experiment natural changes in experimental conditions, experimenters, participants that occur over time are balanced across the experimental conditions as with all random assignment, block randomization balances subject characteristics across conditions of the experiment

Threats to Internal Validity Ability to make causal inferences is jeopardized when intact groups are used extraneous variables are not controlled selective subject loss occurs demand characteristics and experimenter effects are not controlled

Threats to Internal Validity, continued Intact groups these groups exist before experiment examples children in different classrooms, departments within an organization, sections of Introductory Psychology course individuals are not randomly assigned to intact groups when intact groups (not individuals) are randomly assigned to conditions, subject characteristics are not balanced do not use intact groups

Threats to Internal Validity, continued Extraneous variables practical considerations when conducting an experiment may create confoundings examples of extraneous variables number of participants in each session different experimenters different rooms where experiment is conducted

Threats to Internal Validity, continued Example Suppose two experimenters help to conduct an experiment. One experimenter tests all of the participants in the treatment condition and the second experimenter tests all of the participants in the control condition. This experiment is confounded because any differences on the DV may be due to the IV (treatment, control) or to the two experimenters.

Threats to Internal Validity, continued How to control extraneous variables Balancing randomly assign extraneous variables across the conditions of the experiment example: Each experimenter conducts both treatment and control sessions, and are randomly assigned to administer a condition at any particular time Holding conditions constant hold extraneous variables constant across the conditions of the experiment example: one experimenter conducts both treatment and control sessions

Threats to Internal Validity, continued Subject loss (attrition)‏ occurs when participants fail to complete an experiment equivalent groups formed at beginning of an experiment through random assignment may no longer be equivalent two types of attrition mechanical subject loss selective subject loss

Threats to Internal Validity, continued Mechanical subject loss when equipment failure or experimenter error results in participant’s inability to complete experiment often due to chance factors likely to occur equally across conditions of experiment because mechanical subject loss is due to chance events, it does not threaten internal validity of experiment

Threats to Internal Validity, continued Selective subject loss occurs when participants are lost differentially across conditions some characteristic of participant is responsible for the loss the subject characteristic is related to the dependent variable example: suppose a treatment for depression is compared to a no- treatment control condition selective subject loss might occur if people drop out of the control condition more than the treatment condition

Threats to Internal Validity, continued Placebo control and double-blind experiments demand characteristics are cues participants use to guide their behavior in a study example: in drug treatment research, demand characteristics suggest to participants they will improve as a result of the drug participants may expect to improve expectations may cause improvement, not the drug

Threats to Internal Validity, continued Placebo control group used to assess whether participants’ expectancies contribute to outcome of experiment participants in placebo control group receive a placebo (inert substance), but believe they may be receiving an effective treatment if participants who receive the actual drug improve more than participants who receive the placebo, we gain confidence that the drug produced the beneficial outcome, rather than expectancies

Threats to Internal Validity, continued Experimenter effects potential biases that occur when experimenter’s expectancies regarding the outcome of the experiment influence their behavior toward participants control by keeping experimenters and observers “blind” or unaware of the expected results

Threats to Internal Validity, continued Double-blind experiment procedures in which both participants and experimenters/observers are unaware of the condition being administered controls both demand characteristics experimenter effects allows researchers to rule out participants’ and experimenters’ expectancies as alternative explanations for a study’s outcome

Analysis and Interpretation of Experimental Findings We rely on statistical analysis to claim an independent variable produced an effect on a dependent variable rule out the alternative explanation that chance produced differences among the groups in an experiment Replication best way to determine whether findings are reliable repeat experiment and see if same results are obtained

Analysis of Experimental Designs Three steps Check the data errors? outliers? Describe the results descriptive statistics such as means, standard deviations Confirm what the data reveal inferential statistics

Analysis of Experiments, continued Descriptive Statistics Mean (central tendency)‏ average score on a DV, computed for each group not interested in each individual score, but how people responded on average in a condition Standard deviation (variability)‏ average distance of each score from the mean of a group not everyone responds the same way to an experimental condition

Analysis of Experiments, continued Effect size measure of the strength of the relationship between the IV and DV Cohen’s d difference between treatment and control means average variability for all participants’ scores Guidelines for interpreting Cohen’s d: small effect of IV: d = .20 medium effect of IV: d = .50 large effect of IV: d = .80

Analysis of Experiments, continued Meta-analysis summarize the effect sizes across many experiments that investigate the same IV or DV select experiments to include based on their internal validity and other criteria allows researchers to gain confidence in general psychological principles

Analysis of Experiments, continued Confirm what the data reveal use inferential statistics to determine whether the IV had a reliable effect on the DV rule out whether findings are due to chance (error variation)‏ two types of inferential statistics Null Hypothesis Significance Testing Confidence intervals

Analysis of Experiments, continued Null Hypothesis Significance Testing statistical procedure to determine whether mean difference between conditions is greater than what might be expected due to chance or error variation the effect of an IV on the DV is statistically significant when the probability of the results being due to chance is low

Analysis of Experiments, continued Steps for Null Hypothesis Testing (1) Assume the null hypothesis is true The null hypothesis assumes the population means for groups in the experiment are equal. example: the population mean for body dissatisfaction following Barbie images is equal to the population mean for Emme images or neutral images

Analysis of Experiments, continued (2) Use sample means to estimate population means. example: mean body dissatisfaction for Barbie = -.76 mean body dissatisfaction for Emme = 0.00 mean body dissatisfaction for neutral = 0.00 difference between Barbie and Emme/neutral = -.76 Is the observed mean difference (-.76) greater than what is expected when we assume the null hypothesis is true (zero)?

Analysis of Experiments, continued (3) Compute the appropriate inferential statistic. t-test: test the difference between two sample means F-test (ANOVA): test the difference among three or more sample means (4) Identify the probability associated with the inferential statistic p value is printed in computer output or can be found in statistical tables

Analysis of Experiments, continued (5) Compare the observed probability with the predetermine level of significance (alpha), which is usually p < .05 If the observed p value is greater than .05, do not reject the null hypothesis of no difference conclude IV did not produce a reliable effect If the observed p value is less than .05, reject the null hypothesis of no difference. conclude IV did produce a reliable effect version of picture books (Barbie, Emme, neutral) caused differences in young girls’ body dissatisfaction

Analysis of Experiments, continued Confidence intervals sample means estimate population means Confidence intervals provide the range of values that contains the true population mean with some probability, usually .95

Analysis of Experiments, continued we typically want to conclude that performance in one experimental condition differs from performance in a second condition compute the confidence interval around the sample mean in each condition if the confidence intervals do not overlap, we gain confidence that the population means for the conditions are different —that is, there is a difference among conditions

Analysis of Experiments, continued Example of confidence intervals suppose the confidence interval for mean body dissatisfaction in the Barbie condition is –1.16 -- –.36 This interval contains the true population mean for body dissatisfaction following Barbie images (remember the sample mean is –.76). suppose the confidence interval for mean body dissatisfaction in the neutral image condition is –.25 -- +.25 this interval contains the true population mean for body dissatisfaction following neutral images (the sample mean is 0.00)‏

Analysis of Experiments, continued Barbie: –1.16 -- –.36 Neutral: –.25 -- +.25 because the confidence intervals do not overlap, we can be confidence that the population means for the two groups differ viewing Barbie images, compared to neutral images, produces greater body dissatisfaction in the population of young girls

Analysis of Experiments, continued suppose instead that the confidence intervals overlap: Barbie Neutral –1.56 -- +.04 –.82 -- +.82 even though the sample means differ (–.76 and 0.00), we cannot conclude that the population means differ because the confidence intervals overlap the difference between the sample means could be attributed to chance

External Validity External validity the extent to which findings from an experiment can be generalized to describe individuals, settings, and conditions beyond the scope of a specific experiment any single experiment has limited external validity external validity of findings increase when findings are replicated in a new experiment

External Validity, continued Questions of external validity would the same findings occur in different settings? in different conditions? for different participants? example: research with college students is often criticized because of low external validity sample often doesn’t matter when testing a theory on what dimensions do college students differ?

External Validity, continued Increasing external validity include characteristics of situations, settings, and population to which researchers wish to generalize partial replications field experiments conceptual replications

Additional Independent Groups Designs Matched Groups Design random assignment requires large samples to balance subject characteristics sometimes only small samples are available in matched groups design, researchers select one or two individual differences variables for matching

Matched Groups Design Procedure select matching variable individual differences variables are characteristics of people that differ, or vary choose matching variable related to outcome or dependent variable measure variable and order individuals’ scores match pairs (or triples, quadruples, etc. depending on number of conditions) of identical or similar scores randomly assign participants within each match to the different conditions

Matched Groups Design, continued Important points about matching participants are matched only on the matching variable participants across conditions may differ on other important variables these differences may be alternative explanations for study’s results (confounding)‏ the more characteristics a researcher tries to match, the harder if will be to match

Natural Groups Designs psychologists’ questions often ask about how individuals differ, and how these individual differences are related to important outcomes. examples: Do men and women differ in what they seek in intimate relationships? Are extraverted individuals, compared to introverted individuals, more likely to succeed in business?

Natural Groups Designs, continued Individual differences (subject) variables characteristics or traits that vary across individuals physical characteristics sex, race social (demographic) characteristics ethnicity, religious affiliation, marital status personality characteristics extraversion, emotional stability, intelligence mental health characteristics depression, anxiety, substance abuse

Natural Groups Designs, continued Researchers can’t randomly assign participants to these groups random assignment to male/female groups? When a researcher investigates an independent variable in which the groups (conditions) are formed naturally, we say a “natural groups design” is used

Natural Groups Designs, continued Example: Suppose we want to compare occupational functioning of schizophrenics and normal (nonschizophrenic) controls? Independent variable natural groups variable: schizophrenic vs. normal Dependent variable measure of occupational functioning Result suppose schizophrenics have poorer occupational functioning than normal participants

Natural Groups Designs, continued Causal inferences and natural groups design Researchers can’t make a causal inference when a natural groups design is used example: can we say that schizophrenia causes poorer occupational functioning? No. The two groups likely differ in other ways that may cause poorer occupational functioning among schizophrenics (confoundings)‏ education level, drugs, nutritional status, tardive dyskinesia, etc.

Natural Groups Designs, continued correlational research allow researchers to describe and predict relationships among individual differences variables and outcomes do not allow researchers to make causal inferences about individual differences variables