7 Control Techniques in Experimental Research

Introduction Points to note random selection rarely used – researchers focus on the independent variable purposive and convenience samples typically used external validity increased by replication For the rest of this chapter, assume the researcher already has a sample and our focus will be on control techniques

Introduction

Goal of Experimentation Identify the causal effect of the independent variable (IV) on the dependent variable (DV) must have internal validity to do this requires control of extraneous variables eliminate differential influence – confounds

Random Assignment A control technique to equate groups of participants accomplished by ensuring that every member has an equal chance of being assigned to any group Provides maximum insurance that groups are equal Eliminates systematic differences between groups Doesn’t eliminate extraneous variables, but randomly distributes them across groups The most important and powerful control technique www.randomizer.org

Random Assignment Example

Matching Use of any of a variety of techniques to equate participants in the treatment groups on specific variables should be done with variables thought to be related to the IV or may confound IV e.g., intelligence, age, gender Advantages of matching controls for the variables on which participants are matched increases the sensitivity of the experiment

Matching Techniques Holding variables constant example disadvantages gender and conformity study – using only females disadvantages restricts the population size restricts generalization to the type of participants in the study

Matching Techniques Building the extraneous variable into the research design (blocking) make the extraneous variable another IV in the study should be used only when you are interested in the effect of the extraneous variable

Matching Techniques Yoked control matches participants on the basis of the temporal sequence of administering an event Example research question – a researcher wants to know the effect of students having the freedom to choose snack breaks on their classroom productivity each control participant is “yoked” to an experimental participant experimental participant takes a self-chosen snack break the control “yoked” participant is given a snack break by the teacher

Matching Techniques Matching by equating participants individual matching match case by case then randomly assign to groups advantage groups equated on potential extraneous variable disadvantages identifying the variables on which to match matching participants increases as the number of variables on which to match increases some variables difficult to match decrease in generalizability of results

Individual Matching Example

Individual Matching Example Name IQ Sara 110 Jane 95 Donald 120 Debbie 105 Jake 120 Jody 105 Kathy 95 Brian 110 Random Assignment to conditions: Treatment Control Brian Sara Jane Kathy Debbie Jody Jake Donald Average IQ by group: 107.5 107.5 13

Counterbalancing Used only with repeated measures (within participants) designs All participants receive all treatment conditions

Counterbalancing Used to control sequencing effects two types of sequencing effects order effects a sequencing effect arising from the order in which the treatment conditions are administered to participants e.g., practice, fatigue carryover effects a sequencing effect that occurs when performance in one treatment condition affects performance in another treatment condition e.g., effects of treatment A still affecting participant when engaged in treatment B

Counterbalancing Counterbalancing procedures randomized counterbalancing sequence of conditions is randomly determined for each participant example: IV with three levels six possible sequences (ABC, ACB, BAC, BCA, CAB, CBA) each participant randomly assigned to one sequence can use randomizer.org or a random table of numbers

Counterbalancing Counterbalancing procedures intrasubject counterbalancing participants take treatments in more than one order may not be feasible with long treatment sequences example: IV with three levels six possible sequences (ABC, ACB, BAC, BCA, CAB, CBA) each participant is exposed to all sequences

Counterbalancing Counterbalancing procedures complete counterbalancing all possible sequences of treatment conditions are used participants randomly assigned to sequence N! = N multiplied by each number below it examples: 2! = 2 x 1 = 2 AB, BA 3! = 3 x 2 x 1 = 6 ABC, ACB, BAC, BCA, CAB, CBA 4! = 4 x 3 x 2 x 1 = 24 rarely used with more than 3 conditions because number of possible sequences (N!) is too large

Counterbalancing Counterbalancing procedures incomplete counterbalancing most commonly used technique not all possible sequences are used criteria – for sequences enumerated each treatment condition must appear an equal number of times in each ordinal position and each treatment condition must precede and be followed by every other condition an equal number of times sequences determined by formula 1, 2, n, 3, (n-1), 4, (n- 2), 5

Incomplete Counterbalancing Example

Counterbalancing Differential carryover a treatment condition affects participants’ performance in a later condition in one way and in another way when followed by a different condition example the carryover from treatment A is 4 units when it is immediately followed by treatment B, but the carryover from treatment A is 2 units when it is immediately followed by treatment C if suspected, don’t use repeated measures

Control of Participant Effects Double-blind placebo method neither the experimenter nor the research participant is aware of the treatment condition administered to the participant Deception omission of or altering the truth of information given to the participant during a research study used when there is no other way to gain the knowledge and risk does not outweigh the benefit of the information must keep the false information constant for all participants

Control of Participant Effects Control of participant interpretation techniques used to gain insight into participants’ perceptions of experiment retrospective verbal report an oral report in which the participant retrospectively recalls aspects of the experiment questions to ask what did the participant think the experiment was about? what did he or she think the experimenter expected to find? what type of response did the participant attempt to give, and why? how does the participant think others will respond in this situation?

Control of Participant Effects Control of participant interpretation concurrent verbal reports a participant’s oral report of the experiment, which is obtained as the experiment is being performed sacrifice groups groups of participants who are stopped and interviewed at different stages of the experiment concurrent probing obtaining a participant’s perceptions of the experiment after completion of each trial think-aloud technique a method that requires participants to verbalize their thoughts as they are performing the experiment

Control of Experimenter Effects Control of recording errors make researchers aware of making careful observations may require training of researcher before data collection use multiple data recorders (e.g., video cameras, computers) or observers have participants make responses on a computer Control of attribute errors referring to attributes of the experimenter use the same experimenter in all treatment conditions unless the treatment condition interacts with attributes

Control of Experimenter Expectancy Control of experimenter expectancies blind technique a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter e.g., double-blind procedure partial blind technique a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter through as many stages of the experiment as possible automation the technique of totally automating the experimental procedures, so that no experimenter-participant interaction is required