Between-Subjects, within-subjects, and factorial Experimental Designs Chapters 10-12 Between-Subjects, within-subjects, and factorial Experimental Designs

Conducting Experiments: Between-Subjects Design Between-subjects design – Different participants are observed one time in each group or at each level of a factor Between-subjects experimental design – Levels of a between-subjects factor are manipulated, then different participants are randomly assigned to each group or to each level of that factor, and observed one time

Conducting Experiments: Between-Subjects Design Control: (a) the manipulation of a variable and (b) holding all other variables constant Experimental or treatment group: Participants are treated or exposed to a manipulation, or level of the IV, that is believed to cause a change in the DV Control group: Participants are treated the same as those in an experimental group, except that the manipulation is omitted Placebo: An inert substance, surgery, or therapy that resembles a real treatment but has no real effect

Manipulation and the Independent Variable Experimental manipulation Natural manipulation: Manipulation of a stimulus that can be naturally changed with little effort Typically involves manipulation of a physical stimulus Ex. Dimmed or brightly lit room, soft or loud sounds Staged manipulation: Manipulation of an IV that requires the participant to be “set up” to experience some stimulus or event Often requires the help of a confederate Confederate: Coresearcher in cahoots with the researcher

Manipulation and the Independent Variable

Manipulation and the Independent Variable Random assignment and control Random assignment: Procedure used to ensure that each participant has the same likelihood of being selected to a given group Can be confident that any differences observed between groups can be attributed to the different levels of the IV and not individual differences Restricted measures of control Restricted random assignment: Restricting a sample based on known participant characteristics, then using a random procedure to assign participants to each group Control by matching Control by holding constant

Manipulation: Control By Matching

Overlap and Identifying Error

Comparing Samples Selecting multiple independent samples Selecting Two Independent Samples Independent sample: Different participants are independently observed one time in each group Selecting multiple independent samples

Comparing Independent Samples The use of the test statistic Independent-Samples T-test for factor with 2 levels One-way between subjects ANOVA: one factor with two or more levels concerning the variance among group means Post hoc test: Computed following a significant ANOVA to determine which pair(s) of group means significantly differ These tests are needed with more than two groups because multiple comparisons must be made

Advantages and Disadvantages of the Between-Subjects Design It is the only design that can include all three: random assignment, manipulation, inclusion of a comparison/control group Disadvantages Sample size required can be large, particularly with many groups

Conducting Experiments: Within-Subjects Design Within-subjects design, also called a repeated- measures design – Design in which the same participants are observed one time in each group of a research study Within-subjects experimental design – The levels of a within-subjects factor are manipulated, then the same participants are observed in each group or at each level of the factor

Conducting Experiments: Within-Subjects Design Two common reasons that researchers observe the same participants in each group are as follows: 1. To manage sample size 2. To observe changes in behavior over time, which is often the case for studies on learning or within– participant changes over time The within-subjects experimental design does not meet the randomization requirement for demonstrating cause and effect Because the participants are observed in each group, we cannot use random assignment, therefore do not use randomization

Controlling Time-Related Factors Time-related factors must be controlled or made the same between groups, such that only the levels of the IV are different between groups Time related factors include those introduced in chapter 6, such as maturation, testing effect, regression toward the mean, and attrition Participant fatigue: State of physical or psychological exhaustion resulting from intense research demands typically due to observing participants too often, or requiring participants to engage in research activities that are too demanding

Controlling Time-Related Factors To control for time-related factors, researchers make efforts to control for order effects Order effects: A threat to internal validity in which the order in which participants receive different treatments or participate in different groups causes changes in a DV Carryover effects: A threat to internal validity in which participation in one group “carries over” or causes changes in performance in a second group Two strategies to control for order effects are to control order and control timing

Controlling Time-Related Factors Counterbalancing – The order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment 1. Complete counterbalancing (K!) 2. Partial counterbalancing Left Right Center

Controlling Time-Related Factors

Individual Differences and Variability The within-subjects design minimizes individual differences between groups because the same participants are observed in each group When the same participants are observed in each group, the individual differences of participants are also the same in each group

Individual Differences and Variability Sources of variability Between-groups variability: Source of variance in a dependent measure that is caused by or associated with the manipulation of the levels (or groups) of an IV This variability is measured by the group means

Individual Differences and Variability

Comparing Two Related Samples Selecting two related samples Related sample, also called a dependent sample: The same or matched participants are observed in each group There are two ways to select two related samples: 1. The same participants are observed in each group 2. Participants are matched, experimentally or naturally, based on the common characteristics or traits that they share

Comparing Two Related Samples The use of the test statistic Test statistic: Mathematical formula that allows us to determine whether the manipulation or error variance is likely to explain differences between the groups Related-samples t test, also called a paired-samples t test: Statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data for two related samples in which the variance in one population is unknown t = Mean differences between groups Mean differences attributed to error

Comparing Two Related Samples

Comparing Two or More Related Samples Selecting multiple related samples Only the repeated-measures design can be used to observe participants in more than two groups

Comparing Two or More Related Samples The use of the test statistic One-way within-subjects analysis of variance (ANOVA): Statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when the same participants are observed at each level of a factor and the variance in a given population is unknown F = Variability between groups Variability attributed to error

Comparing Two or More Related Samples

Testing Multiple Factors in the Same Experiment Factorial design – Research design in which participants are observed across the combination of levels of two or more factors In stats class, this was referred to as Two-Way ANOVA (or more)

Testing Multiple Factors in the Same Experiment Factorial experimental design – Research design in which groups are created by manipulating the levels of two or more factors (can be between-, within- and mixed-design) Completely crossed design: A factorial design in which each level of one factor is combined or crossed with each level of the other factor, with participants observed in each cell or combination of levels

Selecting Samples for a Factorial Design in Experimentation We select ONE sample from a population, then randomly assign the same or different participants to groups created by combining the levels of two or more factors or IVs Create the groups by combining the levels of each IV Identify a factorial design by the number of levels for each factor Then assign participants to groups

Selecting Samples for a Factorial Design in Experimentation Decaf Reg Coffee Easy Task Difficult Task Decaf Reg Coffee Water Easy Task Difficult Task

Types of Factorial Designs Between-subjects design – Levels of two or more between-subjects factors are combined to create groups, meaning that different participants are observed in each group Ex. Researchers recorded how well participants comprehended a passage that varied by type of highlighting and text difficulty (Gier, Kreiner, & Natz-Gonzalez, 2009)

Types of Factorial Designs Within-subjects design – Levels of two or more within-subjects factors are combined to create groups, meaning that the same participants are observed in each group

Types of Factorial Designs Mixed factorial design – Different participants are observed at each level of a between-subjects factor and also repeatedly observed across the levels of the within- subjects factor

Main Effects and Interactions Two-way analysis of variance (ANOVA) – Statistical procedure used to analyze the variance in a DV between groups created by combining the levels of two factors F = Variability between groups Variability attributed to error The test statistic can also be used in quasi-experiments however, because the quasi-experiment does not methodologically control for individual differences, the design cannot demonstrate cause and effect

Main Effects and Interactions Two-way factorial design – Research design in which participants are observed in groups created by combining or crossing the levels of two factors Using this design we can identify three sources of variation: Main Effect for Factor A Main Effect for Factor B) Interaction Effect (the combination of levels of Factors A and B

Main Effects and Interactions Main effects – Source of variation associated with mean differences across the levels of a single factor A significant main effect indicates that group means significantly vary across the levels of one factor, independent of the second factor Interactions – Source of variation associated with how the effects of one factor are influenced by, or depend on, the levels of a second factor A significant interaction indicates that group means significantly vary across the combined levels of two factors In a table summary, an interaction is a measure of how cell means at each level of one factor change across the levels of a second factor

Identifying Main Effects and Interactions in a Graph Even if a graph shows a possible main effect or interaction, the use of a test statistic is still needed to determine whether it is significant Graphing only main effects We would observe changes at the levels of one factor, independent of the changes in a second factor When significant, look at the row and column means to describe the effect

Including Quasi-Independent Factors in an Experiment The factorial design can be used when we include preexisting or quasi-independent factors Participant variable – A quasi-independent or preexisting variable that is related to or characteristic of the personal attributes of a participant Typically demographic characteristics (ex. age, gender) An effect of a quasi-independent variable shows that the factor is related to changes in a DV It does not demonstrate cause and effect because the factor is preexisting