Copyright © 2004 Educational Testing Service Listening. Learning. Leading. Understanding and Setting New Scores

Copyright © 2004 Educational Testing Service iBT TOEFL Scores The score report universities receive will contain: Four skill scores –Reading –Listening –Speaking –Writing A total score: 0 – 120

Copyright © 2004 Educational Testing Service Setting New TOEFL Scores An opportunity to think more about English language needs & requirements Are all skills equally important for success in your program? If not, do your score requirements reflect this? Students with the same total score can have different abilities Consider using: –skill scores instead of total scores –skill scores and total scores

Copyright © 2004 Educational Testing Service Understanding & Setting New Scores Rubrics for Writing & Speaking Speaking and Writing samples Score comparisons Percentile data English language competency descriptors Results from standard setting studies Standard setting on your own campus

Copyright © 2004 Educational Testing Service Score Comparisons: iBT, CBT, PBT Based on a field test of 3,000 test takers in 30 countries Approx 80% of TOEFL volume comes from these 30 countries Test takers took TOEFL iBT and TOEFL CBT (computer-based) Score comparisons for CBT and PBT (paper-based test) established in 1998

Copyright © 2004 Educational Testing Service Using Score Comparisons: iBT, CBT, PBT Score-to-score and range-to-range comparisons for: –Reading –Listening –Writing –Reading, Listening & Writing only total –Total score (all four skills)

Copyright © 2004 Educational Testing Service Setting Speaking Scores No Speaking section on TOEFL CBT Standard setting panel of ITA experts Professional judgment, not statistically determined Minimally acceptable level of speaking for lowest level of contact with undergraduates: 23 TSE 50 score equivalent: 26

Copyright © 2004 Educational Testing Service Using Score Comparisons: iBT, CBT, PBT Score comparisons can be useful for understanding relationships, but differences in the tests can make it difficult to establish exact comparisons.

Copyright © 2004 Educational Testing Service Using Score Comparisons: iBT, CBT, PBT Remember: If you’re comparing total scores, CBT and PBT TOEFL do not measure Speaking, but iBT does. Using skill scores or skill scores and a total score will give you more information about your applicant. Use the whole profile. ETS encourages you not to set a rigid cut score

Copyright © 2004 Educational Testing Service Percentile Data Based on field test Should be used with caution –Sample size –Motivation & performance –Unfamiliarity with the test, communicative competence & integrated tasks –Addition of Speaking –Participants had lower ability level than general TOEFL population New data from operational test to be published after first year

Copyright © 2004 Educational Testing Service English Language Competency Descriptors Based on 2,300 students who took the new iBT test and responded to questions about their English language ability in each skill Competency descriptors tell us what a student can do, and has difficulty with, in English

Copyright © 2004 Educational Testing Service Results from ETS Standard Setting Studies Panels at 5 universities recommended scores based on seeing the new test and reviewing examinee performance. Results to be published after decision makers discuss recommendations and set final scores. Final scores to be posted on our Web site. Schools similar to those in the study may wish to adopt similar scores.

Copyright © 2004 Educational Testing Service Panel Recommendations – NOT FINAL SCORES Duke Univ. Lehigh Univ. Texas- Austin Carnegie Mellon U. British Columbia Level:Grad Under- grad Writing Speaking Reading Listening Total

Copyright © 2004 Educational Testing Service Standard Setting at Your Institution The best way to ensure your language requirements match your institution Over 2 days, a panel of college or university faculty and staff work view the test and student responses to it. The panel recommends the level of English just high enough to cope with studies at your school.