1. 1 Establishing A Passing Standard Paul D. Naylor, Ph.D. Psychometric Consultant

2. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 2 Standard Setting The process used to arrive at a passing score Lowest score permitting entry to the field Recommended Standard

3. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 3 Validity is About Interpretation Evidence to support the interpretation What standard setting system was used? What was the rationale for selecting it? What was the qualification of those who participated?

4. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 4 Standards Mandated Norm-referenced Criterion-referenced

5. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 5 Mandated Standards Often used in licensing Difficult to defend Not related to minimum qualification

6. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 6 Norm-referenced Standards Popular in schools Limits entry Inconsistent results

7. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 7 Criterion-referenced Standards Wide acceptqnce in professional testing Determines minimum qualification Not test population dependant Exam or item centered

8. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 8 Procedures Angoff (modified) Nedelsky Ebell Others

9. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 9 Minimally Competent Performance Minimum acceptable performance Minimal qualification Borderline It’s all relative

10. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 10 Angoff Method Judges Selection Training Probabilities Would vs Should Rater Agreement Tabulation

11. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 11 1. How many goals are on a basketball court? A. 1 B. 2 C. 3 D. 4

12. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 12 2. How many halves are in a college basketball game? A. 1 B. 2 C. 3 D. 4

13. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 13 3. How many quarters are in a high school basketball game? A. 1 B. 2 C. 3 D. 4

14. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 14 4. How many points does a 3- point field goal account for in a basketball game? A. 1 B. 2 C. 3 D. 4

15. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 15 5. How many teams are in the NCAA Men’s Basketball National Championship Tournament? A. 48 B. 64 C. 65 D. 32

16. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 16 Application Adjustment Angoff Values Alternative Forms of Exam Passing Score

17. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 17 Validity Evidence How was the passing score set? Who set the passing score?

18. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 18 Minimum Requirements Description of method Results of method used List of participants and qualifications Demographics Credentials Affiliations

19. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 19 Additional Evidence Data from method used Description of training given to participants Definition of minimum qualification

20. Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 20 References Livingston, S.A. & Zieky, M.J. (1982). Passing Scores. ETS. Cizek, C.J. Standard setting guidelines. Educational Measurement: Issues and Practices, 15 (1), 13-21, 12. CLEAR Exam Review (Winter 2001, Summer 2001, and others)