1. A Placement Validity Study for Freshman Composition and College Algebra Gary Greer University of Houston Downtown (greerg@uhd.edu) Annual Conference on The First-Year Experience February 22, 2004 1:45-2:45 PM Addison TX

2. Purpose of this study To check the validity of our cut scores. To be able to translate our student’s placement score into a probability that the student will succeed. At U. of Houston Downtown we used grades and placement scores to check ACCUPLACER’s accuracy in forecasting student performance.

3. Setting Cut Scores NCME recommends in The Code of Fair Test Practices in Education that test users monitor the behavior of cut scores in placement. (see www.apa.org/science/fairtestcode.html) Documentation should include measurement of the relation between predictor and criterion.

4. Measurement An Accuplacer placement score is a predictor. Course grade is the criterion. Course grade can be predicted by placement score. Success is defined as a course grade B or grade C. A probability is associated with the grade predicted.

5. Samples 844 students (405 Males; 439 Females) took the placement test and Freshman Composition. 1089 students (432 Males; 657 Females) took the placement test and College Algebra. Findings from these samples generalize to future students taking Freshman Composition and College Algebra. Probability of making a B is related to a certain placement score.

6. Grade Distributions and Placement Scores ABCDFTotals Freshman Composition Avg SS Score 184 21.8% 98.5 270 32.0% 93.3 178 21.1% 90.7 39 4.6% 88.4 173 20.5% 92.3 844 100% College Algebra Avg EA Score 89 8.2% 79.4 193 17.7% 67.9 220 20.2% 62.3 161 14.8% 60.6 426 39.1% 54.9 1089 100%

7. Research Questions - Freshman Composition 1. 1. What is the probability that a student with Sentence Skill score “X” will make a B in Freshman Composition? 2. 2. What is the probability that a student with Sentence Skill score “X” will make a C in Freshman Composition? 3. 3. What is the correlation between placement score and course grade?

8. Research Questions – College Algebra 1. 1. What is the probability that a student with Elementary Algebra score “X” will make a B in College Algebra? 2. 2. What is the probability that a student with Elementary Algebra score “X” will make a C in College Algebra? 3. 3. What is the correlation between placement score and course grade?

9. Placement Score and Probability of Success Probability of Success Sentence Skills score for C or better Sentence Skills for B or better Elementary Algebra score for C or better Elementary Algebra score for B or better.80114.7594.7077.6562.6047119104.55339686.50207269.4564951.402533.35115103.3072.2539

10. Decision Probabilities To both maximize correct decisions and minimize incorrect decisions, place students who have probability of success>.50 (1 success out of 2) Or choose.67 to get 2 successes out of 3 or.75 to get 3 successes out of 4

11. True and False Decisions Correct Decisions determined by Placing a student into a course who succeeds – true positive Denying placement to a student who would not succeed – true negative Incorrect Decisions determined by Placing a student into a course who will not succeed – false positive Denying placement to a student who would succeed – false negative

12. Biserial Correlations and % of Correct Placements Predictor Correlation between predictor and criterion % correctly placed for C or higher % correctly placed for B or higher Sentence Skills.0975 Sentence Skills.0554 Elementary Algebra.1461 Elementary Algebra.0974

13. Conclusions We present statistics of Accuplacer’s placement accuracy to faculty every three years. We are satisfied when faculty are satisfied. Data collection and submission for a V study has never been easier.

14. Thank You