1. Repeated Measures Balancing Practice Effects with an Incomplete Design
Each participant experiences each condition of the experiment exactly once
rather than many times as in the complete design
Practice effects are balanced across participants in the incomplete design
rather than within subjects as in the complete design
Two conditions, Experimental (E) and Control (C)
Participant One receives E then C
Participant Two receives C then E

2. Incomplete Design
Need to vary the order of the conditions to balance for practice effects.
The general rule for balancing practice effects in the incomplete design
each condition of the experiment (e.g., A, B, C)
must appear in each ordinal position (1st, 2nd, 3rd) equally often.
If this rule is followed, practice effects will be balanced and will not confound the experiment.

3. Incomplete Design
Techniques for balancing practice effects in the incomplete design include:
all possible orders, and
selected orders.
All possible orders: The preferred technique for balancing practice effects in an incomplete design that has four or fewer conditions of the independent variable.

4. Incomplete Design All possible orders:
With two conditions (A, B), there are two possible orders: AB and BA.
Half of the participants would be randomly assigned to receive condition A first, followed by B.
The remaining participants would complete condition B first, followed by A.
With three conditions (A, B, C), there are six possible orders:
ABC, ACB, BAC, BCA, CAB, CBA
Participants would be randomly assigned to one of the six orders.

5. Incomplete Design All possible orders:
With four conditions (ABCD), there are 24 possible orders (ABCD, ABDC, ACBD, ACDB, etc.). (see table 7.2)
With five conditions, there are 120 possible orders, and with six conditions there are 720 possible orders.
At least one participant must receive each order of the conditions.
Because of this, all possible orders are usually used for experiments with four or fewer conditions of the independent variable.

6. Balancing Practice Effects with an Incomplete Design
Figure 7.3

7. Balancing Practice Effects with an Incomplete Design
(Horch & Hodgins, 2008)
Using vignettes to study stigma with regard to gambling disorders
Four different vignettes or “conditions” comparing
gambling disorder (G)
cancer (C)
schizophrenia (S)
mild stress (N)
Each participant receives these four conditions once
In this case reading four different vignettes
Order of the vignettes changes across participants

8. Participants 1 2 3 4 5 6 7 8 9 10 11 12
Participants 13 14 15 16 17 18 19 20 21 22 23 24

9. Two methods for selecting orders are: Latin Square method, and
Incomplete Design
Selected orders: Rather than using all possible orders, another method for balancing practice effects in the incomplete design is to select particular orders of the conditions.
When there are four or more conditions finding an adequate number of participants is difficult
Number of selected orders should be a multiple of the number of conditions
Two methods for selecting orders are:
Latin Square method, and
Random Starting Order with Rotation.

10. Participants 1 2 3 4 5 6 7 8 9 10 11 12
Participants 13 14 15 16 17 18 19 20 21 22 23 24

11. Incomplete Design
Selected orders:
Latin Square: This method is used to make sure each condition appears in each ordinal position once.
Each participant is randomly assigned to complete the experiment using one of the orders.

12. Procedure for Latin Square Box 7.2
Incomplete Design
Procedure for Latin Square Box 7.2
Randomly order the conditions of the experiment (ABCD etc.).
Number the conditions in your random order (A = 1, B = 2, C = 3, D = 4 and so on).
To generate the first order of conditions, use the rule:
1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc.
For an experiment with four conditions the order would be
To generate the second order of conditions, add 1 to each number in the first order. Because “N” represents the number of conditions (4 in this example), we can’t use N + 1 (4 + 1 = 5) because this would create a fifth condition. Therefore, the rule is that 1 added to the Nth condition is 1. The second order of conditions is:
Six by Six

13. Incomplete Design Procedure for Latin Square 2 3 1 4 B C A D
To generate the third order of conditions, add one to each number in the previous order (again, N + 1 = 1). The third order of conditions is
The same procedure is followed for each order.
When the letters of the conditions are matched to their numbers, the Latin Square for this example is:
A B D C
B C A D
C D B A
D A C B

14. Incomplete Design Latin Square
Each condition appears in each ordinal position equally often which balances practice effects.
For example, condition “A” appears in each ordinal position:
1st 2nd 3rd 4th
A B D C
B C A D
C D B A
D A C B
Find conditions B, C, and D in each ordinal position.

15. Incomplete Design Latin Square
Another advantage of the Latin Square is that each condition precedes and follows each other condition once (e.g., AB and BA, BC and CB)
1st 2nd 3rd 4th
A B D C
B C A D
C D B A
D A C B
Find other examples in which the order of conditions is counterbalanced.

16. Incomplete Design Selected orders:
Random Starting Order with Rotation: Generate a random order of the conditions (e.g., ABCD), and then rotate the sequence by moving each condition one position to the left each time.
1st 2nd 3rd 4th
A B C D
B C D A
C D A B
D A B C
Note that each condition appears in each ordinal position to counterbalance practice effects across the conditions of the experiment. Unlike the Latin Square, however, the order of the conditions is not balanced.

17. Data Analysis of Repeated Measures Designs
A complete repeated measures design requires an additional step for data analysis.
Because participants complete each condition many times, a summary score (e.g., a mean) is needed to learn participants’ average performance in each condition.

18. Analysis of Complete Design
Example research exercise
I.V. Estimation of Time Intervals
Level seconds
Level seconds
Level seconds
Level seconds
D.V. Time estimates for each interval
Each participant completed each “condition” six times
For each participant
Need a summary score for each interval “condition”
If there are extreme scores use a median not the mean
Then get an average for each interval “condition”

19. TABLE 7.3

20. Data Analysis of Repeated Measures Designs When using a complete balancing design
Participants complete two conditions (A, B) of an experiment two times each.
Average of A condition and average of B condition is analyzed
Participants A B
Mean of A
Mean of B 1 10 16 12 14 11 15 2 8 20 3 18 4
Etc.  5 6 7 9

21. Data Analysis of Repeated Measures Designs
The next step is to calculate the mean and standard deviation for each condition across all of the participants.
Condition A Condition B
Participant
Participant
Participant
Participant
And So On
Mean
Std. Dev

22. Data Analysis of Repeated Measures Designs with complete balancing design
Participants complete THREE conditions (A, B, C) of an experiment two times each.
Participants A B C
Mean of A
Mean of B
Mean of C 1 10 16 20 22 12 14 21 2 8 24 9 18 3 26 4 5 6 7

23. Data Analysis of Repeated Measures Designs
The next step is to calculate the mean and standard deviation for each condition across all of the participants.
Condition A Condition B Condition C
Participant
Participant
Participant
Participant
And So On
Mean
Std. Dev

24. The Problem of Differential Transfer
When should repeated measures designs not be used?
When differential transfer is possible.
This occurs when the effects of one condition persist and affect participants’ experiences during subsequent conditions.
When instruction for one condition carry over to the next condition
Use a problem solving strategy A and then control strategy B
When this is possible, use an independent groups design.
Researchers can assess whether differential transfer is a problem in their research by comparing the results for their independent variable when tested in an random groups experiment and when tested in a repeated measures experiment.

25. Differential Transfer
Assess whether differential transfer is a problem by comparing results for repeated measures design and random groups design
Comparing two different experiments
Not very efficient
Or; use the 1st ordinal position to represent a random groups design.
1st 2nd 3rd 4th
A B D C
B C A D
C D B A
D A C B

26. Independent Variable Manipulation:
A Comparison of Repeated Measures Designs and Independent Groups Designs
Independent Variable Manipulation:
Repeated Measures: Each participant experiences every condition of the independent variable.
Independent Groups: Each participant experiences one condition of the independent variable.
What is balanced (averaged) across conditions?
Repeated Measures: Practice effects
Independent Groups: Individual differences variables