1. Comparing Decision Rules
Decision accuracy of different decision rules combining multiple tests in a higher educational context
Iris Yocarini, Samantha Bouwmeester, Guus Smeets, & Lidia Arends
18/03/2016

2. Decision accuracy of BSA decision
Comparison decision based on true score vs. observed score
Compensatory vs. conjunctive decision rule
Evaluation argument pro compensation: ‘average is more reliable’  correct decision
Decision based on true score
Negative BSA
Positive BSA
Decision based on observed score
Correct classification
Misclassification
False negative
False positive
Correct
classificatie

3. Results Compensatory vs. conjunctive rule
Compensatory lower proportion misclassifications
Decision
Rule
GPA
Minimum
Mean
Proportion
Error 1
5.5
0.06 2 3
0.10 4
0.17 5
0.26
0.24 6
0.14 7
0.15 8
0.18 9
0.25 10 11
6.5
0.23 12
0.22 13 14 15

4. Results Compensatory vs. conjunctive rule
Compensatory lower proportion misclassifications
Decision
Rule
GPA
Minimum
Mean
Proportion
Error
Sample
Size True
Positive
BSA 1
5.5
0.06
92% 2 3
0.10 4
0.17
88% 5
0.26
61%
0.24
40% 6
0.14
78% 7
0.15 8
0.18
77% 9
0.25
60% 10
21% 11
6.5
0.23
53% 12
0.22 13 14
48% 15 9%

5. Results Compensatory vs. conjunctive rule
Compensatory higher sensitivity rate  lower false negative rate
Compensatory lower specificity rate  higher false positive rate
Decision
Rule
GPA
Minimum
Mean
Proportion
Error
Sensitivity
Specificity 1
5.5
0.06
0.98
0.38 2 3
0.10
0.93
0.51 4
0.17
0.86
0.59 5
0.26
0.74
0.71
0.24
0.69
0.79 6
0.14
0.97
0.45 7
0.15
0.94
0.52 8
0.18
0.88 9
0.25
0.75 10
0.63 11
6.5
0.23
0.95
0.56 12
0.22
0.58 13
0.90
0.62 14
0.80
0.72 15

6. Conclusion & Discussion
Decision accuracy important consideration
Compensatory decision rule: relatively fewer false negatives and more false positives
 depends on specific setting & tests used
Focus on both specific decision rule as well as selected tests

7. Thank you for your attention! Questions?
Contact: