1. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 1 Expected Classification Accuracy Lawrence M. Rudner Graduate Management Admission Council® LRudner@gmac.com

2. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 2 I will discuss Classification Accuracy with IRT False Positives – False Negatives Repeat opportunities Lower operational cut scores 2-categories / n-categories Easy estimate

3. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 3 Classification table classified master true master classified non-master true master classified master true non- master classified non-master true non-master

4. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 4 Example

5. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 5 False Positive is the area to the right of

6. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 6 Illustration 1 Dataset –50 items: [a,b,c]=[1.43, 0.00, 0.20] –N(0,1): mean see=.393, r=.92 – θc = -.25; 40 th %ile Table 2: Expected and true examinee classifications Expected Classification MasterNon-Master True Master53.9 5.0 Non-Master6.5 34.6 Accuracy = 88.5%

7. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 7 Application 1 add 10 items with parameters a=2.0, b=-.25 and c=0 : Expected and true examinee classifications adding 10 optimal items Expected Classification MasterNon-Master True Master55.4 3.6 Non-Master4.8 36.3 Accuracy = 91.7%

8. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 8 Application 2 : Expected and true examinee classifications using an adjusted operational cut score Expected Classification Master ( c' >-.40) Non-Master ( c' <-.40) True Master ( c >-.25) 56.2 2.7 Non-Master ( c < -.25) 9.8 31.3 Accuracy = 87.5%

9. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 9 Multiple categories

10. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 10 Simplified estimate

11. Copyright © 2004, Graduate Management Admission Council ®. All Rights Reserved. 11 Simplified Estimate 81.7% calculated correct classification 78.0% estimated correct classification 10-item 2001 Maryland State Performance Assessment Program Grade 8 Reading Test, Form A.