Comparing Decision Rules Decision accuracy of different decision rules combining multiple tests in a higher educational context Iris Yocarini, Samantha Bouwmeester, Guus Smeets, & Lidia Arends 18/03/2016

Decision accuracy of BSA decision Comparison decision based on true score vs. observed score Compensatory vs. conjunctive decision rule Evaluation argument pro compensation: ‘average is more reliable’  correct decision Decision based on true score Negative BSA Positive BSA Decision based on observed score Correct classification Misclassification False negative False positive Correct classificatie

Results Compensatory vs. conjunctive rule Compensatory lower proportion misclassifications Decision Rule GPA Minimum Mean Proportion Error 1 5.5 0.06 2 3 0.10 4 0.17 5 0.26 0.24 6 0.14 7 0.15 8 0.18 9 0.25 10 11 6.5 0.23 12 0.22 13 14 15

Results Compensatory vs. conjunctive rule Compensatory lower proportion misclassifications Decision Rule GPA Minimum Mean Proportion Error Sample Size True Positive BSA 1 5.5 0.06 92% 2 3 0.10 4 0.17 88% 5 0.26 61% 0.24 40% 6 0.14 78% 7 0.15 8 0.18 77% 9 0.25 60% 10 21% 11 6.5 0.23 53% 12 0.22 13 14 48% 15 9%

Results Compensatory vs. conjunctive rule Compensatory higher sensitivity rate  lower false negative rate Compensatory lower specificity rate  higher false positive rate Decision Rule GPA Minimum Mean Proportion Error Sensitivity Specificity 1 5.5 0.06 0.98 0.38 2 3 0.10 0.93 0.51 4 0.17 0.86 0.59 5 0.26 0.74 0.71 0.24 0.69 0.79 6 0.14 0.97 0.45 7 0.15 0.94 0.52 8 0.18 0.88 9 0.25 0.75 10 0.63 11 6.5 0.23 0.95 0.56 12 0.22 0.58 13 0.90 0.62 14 0.80 0.72 15

Conclusion & Discussion Decision accuracy important consideration Compensatory decision rule: relatively fewer false negatives and more false positives  depends on specific setting & tests used Focus on both specific decision rule as well as selected tests

Thank you for your attention! Questions? Contact: yocarini@fsw.eur.nl