State Assessment Systems: Issues and Options David T. Conley, Ph.D. Professor University of Oregon Director Center for Educational Policy Research

2 Purposes of State Assessments A state assessment can have multiple purposes School accountability Student accountability (exit requirement) System monitoring and policy decisions Performance feedback to improve teaching College placement or admission Most state assessments end up addressing multiple purposes Some end up with confused purposes or no one clear purpose

3 The Power Equation of State Assessments State assessments represent an exercise by the state of its power over local school systems State constitutions grant states control over subsidiary governmental units, such as school districts However, long traditions of local control come into conflict with this exercise of state authority in the educational policy arena State assessments ultimately become a compromise between achieving state goals and incurring the political costs of exercising state power to intervene into the functioning of local schools

4 What Is Feasible? State assessments should meet the following criteria: Be manageable by school districts Be consistent with state goals Be taken seriously by educators and students Lead educators and students toward desired behaviors Be reasonable in cost, technical requirements and time Meet technical adequacy standards for assessments

5 The Reliability-Validity Tradeoff Generally, assessment is a tradeoff between reliability and validity The more standardized the assessment, the better the reliability and the more limited the validity The more directly a reflection of actual classroom work, the better the validity and the more challenging the reliability Americans tend to value reliability above all else Other nations favor validity

6 Where Are We in Oregon? Oregon assessment, as originally designed, is admirable in many respects Contained mix of methods Multiple choice, performance tasks, work samples Has long track record Has online component (TESA) Appears to be reasonably well accepted by schools Meets NCLB requirements

7 Where Are We in Oregon? However, significant differences exist from elementary to secondary levels Lower scores, less concern with the assessments Some schools take them seriously, others appear less concerned with flat scores While scores can conceivably be used by higher ed (via PASS), they do not appear to be used in this fashion by many students Legislature seems to need a primer on the system each session

8 Considering Options Note that there are tradeoffs with each option- there is no one best option Development and implementation costs are high for any new system or substantial modification Oregon educational system does not seem particularly ripe for innovation or significant change at the moment

9 Defend Current System This is sensible to do for many reasons The current system aligns well (best?) with state standards It is institutionalized and runs smoothly More educators appear to be more capable of using test data for instructional decisions However, it does leave the state exactly where it is currently, with an assessment that is largely in search of a purpose (beyond meeting NCLB requirements)

10 Revise Current System Augment the test to include items that enable the test to be used for college readiness feedback and perhaps limited college placement purposes Attach stakes to current tests so students and schools take them more seriously E.g., compensatory scale composed of GPA and test scores, with minimum score required to avoid mandatory remediation or some other mild sanction

11 Adopt a National Test Tied to College Admission or Placement ACT system (Explore, PLAN, ACT) or PSAT/SAT Will be taken more seriously by students, teachers Allows national comparisons of Oregon students Is relatively efficient to administer Does not necessarily cover state standards as well as state assessment (this needs to be documented, however) May lead to more students considering college, based on IL and CO experiences

12 Adopt National Tests Advanced Placement incentives or subsidy IB subsidy Work with postsecondary institutions to use data from these tests to recruit more students, award scholarships and credit, and place students

13 Replace or Augment State Test with End-of-Course Exams Key benchmark courses such as Algebra II, Biology, 11th grade English Creates a better balance between reliability and validity Development costs, implementation complexity can be high Can align highly with state standards Can provide diagnostic feedback to students, teachers Eventually requires specifying curriculum for tested courses, something that will be difficult for and alien to the state

14 End-of-Course Exams Can address content knowledge and essential skills Help limit curriculum inflation and drift Can be made a component of the course grade Can be combined in a compensatory fashion with other measures Some efforts are underway nationally to develop consistent end-of-course exams

15 Develop School-based Assessment System Can be collection of evidence or extended application-type project Data collected over time or on a one-time basis, or both Best system for providing diagnostic feedback to students Can be well aligned with state standards Requires local buy-in and a specific mechanism to guarantee rater consistency across schools

16 School-based Assessments Senior year review Student must demonstrate competency in key metacognitive skills such as logical and analytical thinking, problem solving, interpretation, teamwork This can be done through a variety of means to accommodate college-bound and non college-bound students No student is denied a diploma, but the information is used to judge schools and provide students with a better assessment of their current ability levels in these key capabilities for future success in college and career

17 School-based Assessments Collection of evidence Builds upon current work sample requirement Links with PASS Can be incorporated into a senior review Can be used to provide ongoing gauge of college readiness for students and parents Postsecondary ed can align entry-level courses with collection performance levels

18 Overall Criteria to Consider Is system’s goal to gauge basic skills or highest achievement? Is system to inform educators or policy makers? Will system have any effects, consequences, or incentives for test- takers and for educators? Does system achieve or promote key state goals for education?

19 Thinking about a System What are the state’s high school exit standards? What is it students should know and be able to do upon graduation? How close are these to college readiness standards? What does performance look like at 8th and 10 grade for students who will be ready to meet 12th grade exit standards? How can 8th grade assessments also be “exit” measures What assessment system generates this information?

20 Oregon Challenges Balance of state influence-local control What purposes should local control serve? Lack of articulation mechanisms Joint boards notwithstanding, state has few ways in which the three sectors work in a coordinated fashion Funding As always, do more with nothing No 12th grade exit standards CAM had career-related standards, but no content standards Weak connections between assessment system and classroom practices or (state) accountability system