Between-Subjects Experimental Designs Chapter 10 Between-Subjects Experimental Designs

Chapter Outline Conducting experiments: Between-subjects design Experimental versus control group Manipulation and the independent variable Variability and the independent variable Ethics in Focus: The accountability of manipulation Comparing two independent samples Comparing two or more independent samples Measuring the dependent variable Advantages and disadvantages of the between-subjects design

Conducting Experiments: Between-Subjects Design Between-subjects design – Different participants are observed one time in each group or at each level of a factor Between-subjects experimental design – Levels of a between-subjects factor are manipulated, then different participants are randomly assigned to each group or to each level of that factor, and observed one time Between-subjects factor: Type of factor in which different participants are observed in each group, or at each level of the factor

Conducting Experiments: Between-Subjects Design Control: (a) the manipulation of a variable and (b) holding all other variables constant Experimental or treatment group: Participants are treated or exposed to a manipulation, or level of the IV, that is believed to cause a change in the DV Control group: Participants are treated the same as those in an experimental group, except that the manipulation is omitted Placebo: An inert substance, surgery, or therapy that resembles a real treatment but has no real effect

Conducting Experiments: Between-Subjects Design

Manipulation and the Independent Variable Experimental manipulation – The identification of an IV and the creation of two or more groups that constitute the levels of that variable Natural manipulation: Manipulation of a stimulus that can be naturally changed with little effort Typically involves manipulation of a physical stimulus Ex. Dimmed or brightly lit room, soft or loud sounds Staged manipulation: Manipulation of an IV that requires the participant to be “set up” to experience some stimulus or event Often requires the help of a confederate Confederate: Coresearcher in cahoots with the researcher

Manipulation and the Independent Variable

Manipulation and the Independent Variable Random assignment and control Random assignment: Procedure used to ensure that each participant has the same likelihood of being selected to a given group Can be confident that any differences observed between groups can be attributed to the different levels of the IV and not individual differences Advantages: It makes the individual differences of participants about the same in each group, ensures that participants and individual differences of participants are assigned to groups entirely by chance

Manipulation and the Independent Variable Restricted measures of control Restricted random assignment: Restricting a sample based on known participant characteristics, then using a random procedure to assign participants to each group Control by matching: Assess or measure the characteristic we want to control, group or categorize participants based on scores on that measure, and then use a random procedure to assign participants from each category to a group in the study Control by holding constant: Limit which participants are included in a sample based on characteristics they exhibit that may otherwise differ between groups in a study

Manipulation: Control By Matching

Variability and the Independent Variable As an added measure of control, we can also measure individual differences numerically in terms of error variance Error variance or error: Numeric measure of the variability in scores that can be attributed to or is caused by the individual differences of participants in each group Random variation is measured by determining the extent to which scores in each group overlap The more the scores overlap, the larger the error variance The less the scores overlap, the smaller the error variance Test statistic: Mathematical formula that allows researchers to determine the extent to which differences observed between groups can be attributed to the manipulation used to create the different groups

Overlap and Identifying Error

Ethics in Focus: The Accountability of Manipulation Because the researcher creates or manipulates the levels of an IV in an experiment, he or she bears greater responsibility for how participants are treated in each group Manipulating the levels of an IV can be associated with greater ethical accountability on the part of the researcher

Comparing Two Independent Samples Selecting Two Independent Samples Independent sample: Different participants are independently observed one time in each group

Comparing Two Independent Samples The use of the test statistic Once participants have been assigned to groups, conduct the experiment and measure the same DV in each group To compare differences between groups, compute a test statistic, which is a mathematical formula that allows us to determine whether the manipulation or error variance is likely to explain differences between the groups Two-independent sample t test: Used to test hypotheses concerning the difference in interval or ratio scale data between two group means, in which the variance in the population is unknown t = Mean differences between groups Mean differences attributed to error

Comparing Two or More Independent Samples Selecting multiple independent samples

Comparing Two or More Independent Samples The use of the test statistic One-way between subjects ANOVA: Used to test hypotheses for one factor with two or more levels concerning the variance among group means Used when different participants are observed at each level of a factor and the variance in a given population is unknown F = variability between groups variability attributed to error Post hot test: Computed following a significant ANOVA to determine which pair(s) of group means significantly differ These tests are needed with more than two groups because multiple comparisons must be made Pairwise comparison: Statistical comparison for the difference between two group means

Measuring the Dependent Variable Self report measure – Participants respond to one or more questions or statements to indicate their actual or perceived experiences, attitudes, or opinions Responses can be coded numerically or given on numeric response scales Behaviors can be measured using a single item or many behaviors can be measured using multiple-item surveys Advantages: Easy and cost-effective to administer, allows researchers to measure a lot of data in a little time Disadvantage: Self-report items are often inaccurate

Measuring the Dependent Variable Behavioral measure – Researchers directly observe and record the behavior of subjects or participants How we measure the behavior observed mostly depends on how the behavior is operationalized Ex. To test a hypothesis about eating behavior, we could measure how much people eat, how quickly people eat, or how often people eat Advantage: A more direct measure than self-reported behavior Disadvantages: Can require substantial ethical problems, some aspects of behavior are constructs that do not have obvious behavioral measures

Measuring the Dependent Variable Physiological measure – Researchers record physical responses of the brain and body in a human or an animal The normal and disordered functioning of the physiological responses of participants can be compared between groups in an experiment Ex. Measure stress and anxiety by measuring cortisol levels in a blood sample or by measuring galvanic skin response (GSR) Advantages: When careful collection procedures are used, these measures are unbiased Disadvantages: Expense and training to operate the equipment needed to make measurements

Advantages and Disadvantages of the Between-Subjects Design It is the only design that can meet all three requirements of an experiment (randomization, manipulation, inclusion of a comparison/control group) Places less of a burden on the participant and the researcher Disadvantages Sample size required can be large, particularly with many groups

Within-Subjects Experimental Designs Chapter 11 Within-Subjects Experimental Designs

Chapter Outline Conducting experiments: Within-subjects design Controlling time-related factors Ethics in Focus: Minimizing participant fatigue Individual differences and variability Comparing two related samples Comparing two or more related samples An alternative to pre-post designs: Solomon four-group design Comparing between-subjects and within-subjects designs

Conducting Experiments: Within-Subjects Design Within-subjects design, also called a repeated-measures design – Design in which the same participants are observed one time in each group of a research study Within-subjects experimental design – The levels of a within-subjects factor are manipulated, then the same participants are observed in each group or at each level of the factor To qualify as an experiment, the researcher must (1) manipulate the levels of the factor and include a comparison/control group, and (2) make added efforts to control for order and time-related factors Within-subjects factor: Type of factor in which the same participants are observed in each group, or at each level of the factor

Conducting Experiments: Within-Subjects Design Two common reasons that researchers observe the same participants in each group are as follows: 1. To manage sample size 2. To observe changes in behavior over time, which is often the case for studies on learning or within–participant changes over time The within-subjects experimental design does not meet the randomization requirement for demonstrating cause and effect Because the participants are observed in each group, we cannot use random assignment, therefore do not use randomization

Conducting Experiments: Within-Subjects Design

Controlling Time-Related Factors Time-related factors must be controlled or made the same between groups, such that only the levels of the IV are different between groups Time related factors include those introduced in chapter 6, such as maturation, testing effect, regression toward the mean, and attrition Participant fatigue: State of physical or psychological exhaustion resulting from intense research demands typically due to observing participants too often, or requiring participants to engage in research activities that are too demanding

Controlling Time-Related Factors To control for time-related factors, researchers make efforts to control for order effects Order effects: A threat to internal validity in which the order in which participants receive different treatments or participate in different groups causes changes in a DV Carryover effects: A threat to internal validity in which participation in one group “carries over” or causes changes in performance in a second group Two strategies to control for order effects are to control order and control timing

Controlling Time-Related Factors Counterbalancing – The order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment 1. Complete counterbalancing: A procedure in which all possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment Used when the number of treatments or different groups is small, usually two groups but not more than three To calculate, make the following calculation in which k is the number of treatments or groups: Number of possible order sequences = k!

Controlling Time-Related Factors 2. Partial counterbalancing: A procedure in which some, but not all, possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment Often used when there are three or more groups To ensure that the few counterbalanced order sequences are representative of all order sequences, we must ensure: Each treatment or group appears equally often in each position Each treatment or group precedes and follows each treatment or group one time Latin square: A matrix design in which a limited number of order sequences are constructed such that (1) the number of order sequences equals the number of treatments, (2) each treatment appears equally often in each position, and (3) each treatment precedes and follows each treatment one time

Controlling Time-Related Factors

Controlling Time-Related Factors Interval between treatments or groups We control the interval to minimize possible testing and carryover effects Often need to increase the interval between treatments when a treatment causes physiological arousal, such as exercise Total duration of an experiment We control the total duration to minimize the total demands placed on participants, which, when great, can lead to participant attrition, or fatigue

Ethics in Focus: Minimizing Participant Fatigue Researchers must disclose any potential risks, including those related to participant fatigue To minimize risks: Minimize the duration needed to complete the experiment Allow for a reasonable time interval or rest period between treatment presentations

Individual Differences and Variability The within-subjects design minimizes individual differences between groups because the same participants are observed in each group When the same participants are observed in each group, the individual differences of participants are also the same in each group

Individual Differences and Variability Sources of variability Between-groups variability: Source of variance in a dependent measure that is caused by or associated with the manipulation of the levels (or groups) of an IV This variability is measured by the group means

Individual Differences and Variability Within-groups variability: Source of variance in a dependent measure that is caused by or associated with observing different participants within each group Between-persons variability: Source of variance in a dependent measure that is caused by or associated with individual differences or differences in participant responses across all groups Assume this is equal to zero in a within-subjects design because the same participants are observed in each group These sources of variation are called error Error: Source of variance that cannot be attributed to having different groups or treatments Error = Within-Groups Variability + 0

Individual Differences and Variability

Comparing Two Related Samples The goal in experimentation is to minimize the possibility that individual differences, or something other than a manipulation, caused differences between groups

Comparing Two Related Samples Selecting two related samples Related sample, also called a dependent sample: The same or matched participants are observed in each group There are two ways to select two related samples: 1. The same participants are observed in each group 2. Participants are matched, experimentally or naturally, based on the common characteristics or traits that they share

Comparing Two Related Samples 1. The same participants are observed in each group We can select one sample from one population and observe that one sample of participants in each group This type of sampling, used with a repeated- measures design, can be used in an experiment only if we manipulate the levels of the IV and control for order effects

Comparing Two Related Samples 2. Matched-samples design – A within- subjects research design in which participants are matched, experimentally or naturally, based on preexisting characteristics or traits that they share Cannot be used in an experiment because groups are created based on preexisting characteristics and not on a manipulation made by the researcher

Comparing Two Related Samples The use of the test statistic Test statistic: Mathematical formula that allows us to determine whether the manipulation or error variance is likely to explain differences between the groups Related-samples t test, also called a paired- samples t test: Statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data for two related samples in which the variance in one population is unknown t = Mean differences between groups Mean differences attributed to error

Comparing Two Related Samples

Comparing Two or More Related Samples Selecting multiple related samples Only the repeated-measures design can be used to observe participants in more than two groups

Comparing Two or More Related Samples The use of the test statistic One-way within-subjects analysis of variance (ANOVA): Statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when the same participants are observed at each level of a factor and the variance in a given population is unknown F = Variability between groups Variability attributed to error The one-way within-subjects ANOVA informs us that at least one group is different from another group – it does not tell us which pairs of groups differ We compute post hoc tests when we have more than two groups

Comparing Two or More Related Samples

An Alternative to Pre-Post Designs: Solomon Four-Group Design Solomon four-group design – Experimental research design in which different participants are assigned to each of four groups in such a way that comparisons can be made to: (1) determine if a treatment causes changes in posttest measure (2) control for possible confounds or extraneous factors related to giving a pretest measure and observing participants over time

An Alternative to Pre-Post Designs: Solomon Four-Group Design To apply the Solomon four-group design, participants are randomly assigned to one of four groups:

An Alternative to Pre-Post Designs: Solomon Four-Group Design A strength is that multiple comparisons can be made Comparison 1: Difference in pretest-posttest scores for Group A compared to difference in pretest-posttest scores for Group B Comparison 2 (control): Group A posttest and Group C posttest comparison Comparison 3 (control): Group B pretest and Group D posttest Comparison 4 (control): Group B posttest and Group D posttest Comparison 5 (control): Difference in posttest scores for Groups C and D compared to posttest scores for Groups A and B Another strength is that a within-subjects design (pre-post comparisons) is combined with a between-subjects design (between-group comparisons) to establish control Main limitation is the complexity of the design

Comparing Between-Subjects and Within-Subjects Designs Between-subjects design: Randomization The between-subjects design allows researchers to randomly assign participants to groups or to the levels of an IV Advantage of using random assignment is that it makes individual differences about the same in each group Another advantage is that participants are observed one time in one group

Comparing Between-Subjects and Within-Subjects Designs Within-subjects design: Economizing and power The advantage of using a within-subjects design is that fewer participants are required overall The second advantage is that the test statistic has greater power to detect significant differences We can represent the test statistic for each experimental design as follows: Between-subjects design: differences or variance between groups within-groups + between- persons error Within-subjects design: differences or variance between groups within-groups error