Repeated Measures Balancing Practice Effects with an Incomplete Design Each participant experiences each condition of the experiment exactly once rather than many times as in the complete design Practice effects are balanced across participants in the incomplete design rather than within subjects as in the complete design Two conditions, Experimental (E) and Control (C) Participant One receives E then C Participant Two receives C then E

Incomplete Design Need to vary the order of the conditions to balance for practice effects. The general rule for balancing practice effects in the incomplete design each condition of the experiment (e.g., A, B, C) must appear in each ordinal position (1st, 2nd, 3rd) equally often. If this rule is followed, practice effects will be balanced and will not confound the experiment.

Incomplete Design Techniques for balancing practice effects in the incomplete design include: all possible orders, and selected orders. All possible orders: The preferred technique for balancing practice effects in an incomplete design that has four or fewer conditions of the independent variable.

Incomplete Design All possible orders: With two conditions (A, B), there are two possible orders: AB and BA. Half of the participants would be randomly assigned to receive condition A first, followed by B. The remaining participants would complete condition B first, followed by A. With three conditions (A, B, C), there are six possible orders: ABC, ACB, BAC, BCA, CAB, CBA Participants would be randomly assigned to one of the six orders.

Incomplete Design All possible orders: With four conditions (ABCD), there are 24 possible orders (ABCD, ABDC, ACBD, ACDB, etc.). (see table 7.2) With five conditions, there are 120 possible orders, and with six conditions there are 720 possible orders. At least one participant must receive each order of the conditions. Because of this, all possible orders are usually used for experiments with four or fewer conditions of the independent variable.

Balancing Practice Effects with an Incomplete Design Figure 7.3

Balancing Practice Effects with an Incomplete Design (Horch & Hodgins, 2008) Using vignettes to study stigma with regard to gambling disorders Four different vignettes or “conditions” comparing gambling disorder (G) cancer (C) schizophrenia (S) mild stress (N) Each participant receives these four conditions once In this case reading four different vignettes Order of the vignettes changes across participants

Participants 1 2 3 4 5 6 7 8 9 10 11 12 Participants 13 14 15 16 17 18 19 20 21 22 23 24

Two methods for selecting orders are: Latin Square method, and Incomplete Design Selected orders: Rather than using all possible orders, another method for balancing practice effects in the incomplete design is to select particular orders of the conditions. When there are four or more conditions finding an adequate number of participants is difficult Number of selected orders should be a multiple of the number of conditions Two methods for selecting orders are: Latin Square method, and Random Starting Order with Rotation.

Participants 1 2 3 4 5 6 7 8 9 10 11 12 Participants 13 14 15 16 17 18 19 20 21 22 23 24

Incomplete Design Selected orders: Latin Square: This method is used to make sure each condition appears in each ordinal position once. Each participant is randomly assigned to complete the experiment using one of the orders.

Procedure for Latin Square Box 7.2 Incomplete Design Procedure for Latin Square Box 7.2 Randomly order the conditions of the experiment (ABCD etc.). Number the conditions in your random order (A = 1, B = 2, C = 3, D = 4 and so on). To generate the first order of conditions, use the rule: 1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc. For an experiment with four conditions the order would be 1 2 4 3. To generate the second order of conditions, add 1 to each number in the first order. Because “N” represents the number of conditions (4 in this example), we can’t use N + 1 (4 + 1 = 5) because this would create a fifth condition. Therefore, the rule is that 1 added to the Nth condition is 1. The second order of conditions is: 2 3 1 4. Six by Six example @ http://rintintin.colorado.edu

Incomplete Design Procedure for Latin Square 2 3 1 4 B C A D To generate the third order of conditions, add one to each number in the previous order (again, N + 1 = 1). The third order of conditions is 3 4 2 1. The same procedure is followed for each order. When the letters of the conditions are matched to their numbers, the Latin Square for this example is: 1 2 4 3 A B D C 2 3 1 4 B C A D 3 4 2 1 C D B A 4 1 3 2 D A C B

Incomplete Design Latin Square Each condition appears in each ordinal position equally often which balances practice effects. For example, condition “A” appears in each ordinal position: 1st 2nd 3rd 4th A B D C B C A D C D B A D A C B Find conditions B, C, and D in each ordinal position.

Incomplete Design Latin Square Another advantage of the Latin Square is that each condition precedes and follows each other condition once (e.g., AB and BA, BC and CB) 1st 2nd 3rd 4th A B D C B C A D C D B A D A C B Find other examples in which the order of conditions is counterbalanced.

Incomplete Design Selected orders: Random Starting Order with Rotation: Generate a random order of the conditions (e.g., ABCD), and then rotate the sequence by moving each condition one position to the left each time. 1st 2nd 3rd 4th A B C D B C D A C D A B D A B C Note that each condition appears in each ordinal position to counterbalance practice effects across the conditions of the experiment. Unlike the Latin Square, however, the order of the conditions is not balanced.

Data Analysis of Repeated Measures Designs A complete repeated measures design requires an additional step for data analysis. Because participants complete each condition many times, a summary score (e.g., a mean) is needed to learn participants’ average performance in each condition.

Analysis of Complete Design Example research exercise I.V. Estimation of Time Intervals Level 1 12 seconds Level 2 24 seconds Level 3 36 seconds Level 4 48 seconds D.V. Time estimates for each interval Each participant completed each “condition” six times For each participant Need a summary score for each interval “condition” If there are extreme scores use a median not the mean Then get an average for each interval “condition”

TABLE 7.3

Data Analysis of Repeated Measures Designs When using a complete balancing design Participants complete two conditions (A, B) of an experiment two times each. Average of A condition and average of B condition is analyzed Participants A B Mean of A Mean of B 1 10 16 12 14 11 15 2 8 20 3 18 4 Etc.    5 6 7 9

Data Analysis of Repeated Measures Designs The next step is to calculate the mean and standard deviation for each condition across all of the participants. Condition A Condition B Participant 1 11 15 Participant 2 12 15 Participant 3 13 14 Participant 4 11 16 And So On Mean 12.56 15.48 Std. Dev. 0.649 0.817

Data Analysis of Repeated Measures Designs with complete balancing design Participants complete THREE conditions (A, B, C) of an experiment two times each. Participants A B C Mean of A Mean of B Mean of C 1 10 16 20 22 12 14 21 2 8 24 9 18 3 26 4   5 6 7

Data Analysis of Repeated Measures Designs The next step is to calculate the mean and standard deviation for each condition across all of the participants. Condition A Condition B Condition C Participant 1 11 15 21 Participant 2 12 15 22 Participant 3 14 14 21 Participant 4 12 16 23 And So On Mean 12.56 15.48 21.78 Std. Dev. 0.649 0.817 1.213

The Problem of Differential Transfer When should repeated measures designs not be used? When differential transfer is possible. This occurs when the effects of one condition persist and affect participants’ experiences during subsequent conditions. When instruction for one condition carry over to the next condition Use a problem solving strategy A and then control strategy B When this is possible, use an independent groups design. Researchers can assess whether differential transfer is a problem in their research by comparing the results for their independent variable when tested in an random groups experiment and when tested in a repeated measures experiment.

Differential Transfer Assess whether differential transfer is a problem by comparing results for repeated measures design and random groups design Comparing two different experiments Not very efficient Or; use the 1st ordinal position to represent a random groups design. 1st 2nd 3rd 4th A B D C B C A D C D B A D A C B

Independent Variable Manipulation: A Comparison of Repeated Measures Designs and Independent Groups Designs Independent Variable Manipulation: Repeated Measures: Each participant experiences every condition of the independent variable. Independent Groups: Each participant experiences one condition of the independent variable. What is balanced (averaged) across conditions? Repeated Measures: Practice effects Independent Groups: Individual differences variables