SAT and Accountability Evidence and Information Needed and Provided for Using Nationally Recognized High School Assessments for ESSA Kevin Sweeney, College Board National Conference on Student Assessment Austen, TX June 2017

SAT and Accountability

Overview SAT structure and scores Points of Conflict Between Accountability and Admissions Uses

SAT Structure and Scores SAT and Accountability SAT Structure and Scores

SAT Structure and Scores SAT and Accountability SAT Structure and Scores Section Scores are placed on a vertical scale. Test, Cross-Test Scores and Total Score are also on a vertical scale.

SAT and Accountability SAT Structure and Scores Overall, in terms of content and technical characteristics, the SAT is well aligned to meet state accountability needs College and Career Ready Benchmarks Vertical scale supporting various growth models Content alignment studies However, there are some areas that where there are potential points of conflict

SAT and Accountability Points of Conflict Accommodations Benchmarks versus Performance Levels Native Language Assessments Tension between an off-the-shelf test and customized needs

SAT and Accountability Accommodations Tension between fairness and invalidity Historically, CB has focused on invalidity Because of input from states, we have recently expanded our allowable accommodations Ideally, accommodations level an “uneven playing field” Typically, empirical evidence to support accommodations policy is scarce What to use as basis for policy? How to handle conflicting accommodations policies between states?

SAT and Accountability Benchmarks versus Performance Levels CCR benchmarks derived from empirical prediction of student academic performance Performance levels typically derived from judgmental standard setting process, grounded in achievement level descriptors No a priori reason for existing benchmarks to equal Performance Levels coming out of standard setting If a state wants a certain cut score to be the CCR benchmark it must be explicitly incorporated into the achievement level descriptors, and into the standard setting process Hope is not a strategy

SAT and Accountability Benchmarks versus Performance Levels Including benchmarks may create results discrepant with standards in earlier grades. Alternately, if a state does not incorporate the CCR benchmarks, they must be ready to explain why their Proficient Level (for example), is higher or lower than the national benchmark.

SAT and Accountability Native Language Assessments According to ESSA States are obliged to: Identify languages that are present to a significant extent Identify languages for which assessments are needed Identify at least the most populous language other than English that is most spoken in the state Articulate the process by which the languages are identified Articulate the reasons for why assessments in other languages are not developed “despite making every effort” States are NOT obliged to: Develop a specific number of assessments in other languages

SAT and Accountability Native Language Assessments While we do offer glossaries and various other accommodations, the SAT is currently offered in only English.

SAT and Accountability Tension between off the shelf test and Customized test needs The SAT is developed for use in national administrations using data from across the country This provides very good information for college admissions purposes However, this presents a tension for states that have an interest/desire/need to assure that their students are represented in the development process (e.g., included in pre-test item statistics)

SAT and Accountability Tension between off the shelf test and Customized test needs Similarly, because the essay is optional, a score that combines the essay with the Reading and/or Writing sections is not possible at the national level However, some states have a need for an ELA score that includes a student produced written component Again, this produces a tension between the needs of customization and the constraints of off-the-shelf

SAT and Accountability Conclusion The SAT solves many problems for state accountability testing In some cases it is an imperfect solution The points of conflict arising form this imperfection need to be worked on jointly by College Board and their state partners

SAT and Accountability Thank You! Kevin Sweeney ksweeney@collegeboard.org Questions???