1. The All-important Placement Cut Scores
Where to Start?
Luz Bay, Ph.D.
Senior Psychometrician
November 2, 2016

2. What are the feasible options and resources?
Standard Setting: The Most Desirable Option
Imperfect Options
Concordance Tables
Proficiency Statements
Simulated Student Testing
Contrasting Group from a Prerequisite Course
National Sources
Comparable School and Courses

3. Standard Setting
The methodology used to define the level of proficiency necessary for a student to succeed in the course to which he/she is being placed and the cut score corresponding to that proficiency
People
Standard Setting Panel
Policy Makers
Process
Describe knowledge, skills, and abilities required to be successful in the course
Iterative Item Rating Process
Product
Performance Level Descriptions
Placement Cut Scores
Bookmark methodology is recommended and supported by The College Board

4. Bookmark Methodology
Contact you ACCUPLACER® Outreach Representative
Receive instructions to request materials
Document describing step-by-step implementation of the process
Ordered Item Booklet (OIB) used to set cut scores
Send your results to the College Board
Receive your cut scores on the 20 – 120 scale

5. Concordance Table
A valid, proven way to compare scores from different assessments
Used to estimate an examinee’s score on one assessment based on their score from a different assessment
Assumes that scores from the two assessments are interchangeable or have the same meaning

6. What is required? the tests must measure the same thing
a correlation coefficient of at least is needed between scores on the two tests
the population of students used to create the concordance table should not differ in a meaningful way from the population of students to which the concordance table results will be applied

7. Caveat emptor
A concordance table can provide a suggested location for where an examinee may score if given the other assessment; but this is not a perfect predictor.
The alignment of the content, correlation, and population are all potential sources of error.
It is entirely possible, if not likely, that an examinee would get a totally different score if they actually took the assessment from what is predicted by the concordance table.

8. Proficiency Statements
Statements of what students know and are able to do at different ranges of ACCUPLACER scores
Example from Elementary Algebra
If a statement matches your list of what students know and are able to do to succeed in the class to which you would like to use the ACCUPLACER test for placement, you may use the corresponding scaled score as your starting cut score

9. Simulated Student Testing
Make a list or description of what students know and should be able to do to succeed in the class
Use the Demo Site and ask a group of faculty members to take the ACCUPLACER test together
If the “successful student” can answer the question correctly, choose the right answer
If the “successful student” cannot answer the question correctly, choose any of the wrong answers
Do not “swap” responses.
Take the average of the sores from the Individual Student Reports generated after taking the test
If you think you might have gotten an answer wrong when the “successful student” should have gotten it right, do not answer correctly another question that the “successful student” is not supposed to be able to answer correctly.

10. Contrasting Group from a Prerequisite Course
At the end of the semester, test all of the students in class
Group the students by “Successful” vs. “Not Successful”
Determine the “point of indifference”
The score at which there are as many successful students as unsuccessful students
You may need to use a “smoothing” or other statistical approaches to compute the cut score

11. National Survey
Fields, R. & Parsad, B. (2012). Tests and Cut Scores Used for Student Placement in Postsecondary Education: Fall Washington, DC: National Assessment Governing Board.

12. Comparable Schools and Courses
Find colleges that use the placement test that you are thinking of using
The college must match the demographic characteristics of your school
The course description must match the description of the course for which you will be using the ACCUPLACER test for placement
Contact the someone in the college about their experience with the particular test and the placement cut scores

13. Final Remarks
The placement cut scores that you use should withstand legal challenges
Document your procedures
Evaluate your cut scores early and often
Implement predictive placement validity study
Participate ACCUPLACER validity studies by the College Board
Use the College Board’s Admitted Class Evaluation Service™ (ACES™)
Make adjustments to your cut scores based on the results of your evaluation
Maximize vendor support
luz.bay
@luzbay
luzbay
Luz Bay