9 Experimental Design

9.1 Foundations of Experimental Design Experiments have the strongest internal validity due to: Random assignments to a group What is random assignment? Random assignment is the process of assigning your sample into two or more subgroups by chance

9.2 Introduction: The Origins of Experimental Design Sir Ronald Fisher: credited with the invention of the experiment Edward L. Thorndike & Robert S. Woodworth: identified the need for control groups Sir Austin Bradford Hill: conducted the first clinical trial Jonas Salk: used the clinical trial to test polio vaccine

9.2a Distinguishing Features of Experimental Design Program group Treatment group Comparison group Control group Probabilistically equivalent Program group: In a comparative research design, like an experimental or quasi-experimental design, the program or treatment group receives the program of interest and is usually contrasted with a no-treatment comparison or control group or a group receiving another treatment. Treatment group: In a comparative research design, like an experimental or quasi-experimental design, the program or treatment group receives the program of interest and is usually contrasted with a no-treatment comparison or control group or a group receiving another treatment. Comparison group: In a comparative research design, like an experimental or quasi-experimental design, the control or comparison group is a group that is compared or contrasted with a group that receives the program or intervention of interest. Control group: In a comparative research design, like an experimental or quasi-experimental design, the control or comparison group is a group that is compared or contrasted with a group that receives the program or intervention of interest. Probabilistically equivalent: The notion that two groups, if measured infinitely, would on average perform identically. Note that two groups that are probabilistically equivalent would seldom obtain the exact same average score.

9.2b Experimental Design and Threats to Internal Validity 9.5 Threats to internal validity for the posttest-only, randomized experimental design.

9.2c Design Notation for a Two-Group Experimental Design Figure 9.6 Notation for the basic two-group posttest-only randomized experimental design. Analyze differences using a t-test or ANOVA.

9.2d The Difference Between Random Selection and Assignment Random selection is how you draw the sample of people for your study from a population Random assignment is how you assign the sample that you draw to different groups or treatments in your study

9.3 Classifying Experimental Designs Signal-enhancing designs Factorial designs Noise-reducing designs Covariance designs Blocking designs

9.4 Signal Enhancing Designs: Factorial Designs Designs that focus on the program or treatment, its components, and its major dimensions, and enable you to determine whether the program has an effect, whether different subcomponents are effective, and whether there are interactions in the effects caused by subcomponents

9.4a The Basic 2 x 2 Factorial Design Figure 9.9 An example of a basic 2 3 2 factorial design. Factor: A major independent variable. Level: A subdivision of a factor into components or features. Figure 9.10 Design notation for a 2 x 2 factorial design

9.4a The Basic 2 x 2 Factorial Design: Possible Outcome Figure 9.11 The null effects case in a 2 x 2 factorial design.

9.4a A Main Effect of Time in Instruction in a 2 x 2 Factorial design Figure 9.12 A main effect of time in instruction in a 2 x 2 factorial design.

9.4a A Main Effect of Setting in a 2 3 2 Factorial Design Figure 9.13 A main effect of setting in a 2 x 2 factorial design.

9.4a Main Effects of Both Time and Setting in a 2 x 2 Factorial Design Figure 9.14 Main effects of both time and setting in a 2 x 2 factorial design.

9.4a An Interaction in a 2 x 2 Factorial Design Figure 9.15 An interaction in a 2 x 2 factorial design

9.4a A Crossover Interaction in a 2 x 2 Factorial Design Figure 9.16 A crossover interaction in a 2 x 2 factorial design.

9.4b Benefits and Limitations of Factorial Designs Enhances the signal Efficient design Only design that allows you to examine interactions Limitations Complex More participants required

9.4c Factorial Design Variations: 2 x 3 Figure 9.17 Main effect of setting in a 2 x 3 factorial design.

9.4c Factorial Design Variations: 2 x 2 x 3 Figure 9.21 Example of a 2 x 2 x 3 factorial design

9.4c Factorial Design Variations: Incomplete Design Figure 9.23 An incomplete factorial design

9.5 Noise-Reducing Designs: Randomized Block Designs Helps minimize noise through the grouping of units (e.g., participants) into one or more classifications (blocks) that account for some of the variability in the outcome Figure 9.24 The basic randomized block design with four groups or blocks.

9.6 Noise-Reducing Designs Covariance Designs Helps minimize noise through the inclusion of one or more variables (covariates) that account for some of the variability in the outcome measure or dependent variable Covariates are the variables you adjust for in your study Figure 9.25 Notation for the basic analysis of covariance design.

9.7 Hybrid Designs: Switching-Replications Experimental Designs A two-group design in two phases defined by three waves of measurement In the repetition of the treatment, the two groups switch roles The original control group in phase 1 becomes the treatment group in phase 2, whereas the original treatment group acts as the control Figure 9.26 Notation for the switching-replications randomized experimental design.

9.7 Hybrid Designs: Switching-Replications – Short Term Effect Figure 9.27 Switching- Replications design with a short-term persistent treatment effect

9.7 Hybrid Designs: Switching-Replications – Long Term Effect Figure 9.28 Switching-Replications design with a long-term continuing treatment effect. Group 1 received the treatment first, then Group 2.

9.8 Limitations of Experimental Design Differential drop out (mortality threat) Ethical problems Social threats to internal validity Difficult to generalize to the real world

Discuss and Debate What is the difference between random selection and random assignment? What are some strengths and weaknesses of experimental designs? Can you think of some research topics for which a factorial design may be a good approach? Random selection is the process of randomly drawing a sample from the population. Random assignment, on the other hand, involves taking the sample the researcher has drawn, and then randomly assigning each unit (participant) to either the treatment group or the control group.   Experiments are often considered to be “the gold standard” in social science research, but they are also criticized for being artificial. What is observed under tightly controlled conditions may not be what happens in the natural environment. Experiments also bring up a host of ethical concerns, which must be carefully addressed. Other research designs may be appropriate, such as a quasi-experimental design, or a qualitative approach. Any time the researcher wishes to enhance the “signal” in a construct, a factorial design is a good strategy to use. In addition, factorial designs allow researchers to expand the number of independent variables in a study, as well as add levels to those variables. Ask your students to provide some real-world examples (e.g., spending $100 or $200 at the mall versus online).