1 Establishing A Passing Standard Paul D. Naylor, Ph.D. Psychometric Consultant

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 2 Standard Setting The process used to arrive at a passing score Lowest score permitting entry to the field Recommended Standard

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 3 Validity is About Interpretation Evidence to support the interpretation What standard setting system was used? What was the rationale for selecting it? What was the qualification of those who participated?

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 4 Standards Mandated Norm-referenced Criterion-referenced

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 5 Mandated Standards Often used in licensing Difficult to defend Not related to minimum qualification

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 6 Norm-referenced Standards Popular in schools Limits entry Inconsistent results

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 7 Criterion-referenced Standards Wide acceptqnce in professional testing Determines minimum qualification Not test population dependant Exam or item centered

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 8 Procedures Angoff (modified) Nedelsky Ebell Others

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 9 Minimally Competent Performance Minimum acceptable performance Minimal qualification Borderline It’s all relative

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 10 Angoff Method Judges Selection Training Probabilities Would vs Should Rater Agreement Tabulation

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri How many goals are on a basketball court? A. 1 B. 2 C. 3 D. 4

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri How many halves are in a college basketball game? A. 1 B. 2 C. 3 D. 4

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri How many quarters are in a high school basketball game? A. 1 B. 2 C. 3 D. 4

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri How many points does a 3- point field goal account for in a basketball game? A. 1 B. 2 C. 3 D. 4

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri How many teams are in the NCAA Men’s Basketball National Championship Tournament? A. 48 B. 64 C. 65 D. 32

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 16 Application Adjustment Angoff Values Alternative Forms of Exam Passing Score

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 17 Validity Evidence How was the passing score set? Who set the passing score?

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 18 Minimum Requirements Description of method Results of method used List of participants and qualifications Demographics Credentials Affiliations

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 19 Additional Evidence Data from method used Description of training given to participants Definition of minimum qualification

Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri 20 References Livingston, S.A. & Zieky, M.J. (1982). Passing Scores. ETS. Cizek, C.J. Standard setting guidelines. Educational Measurement: Issues and Practices, 15 (1), 13-21, 12. CLEAR Exam Review (Winter 2001, Summer 2001, and others)