1. Oregon’s Alternate Assessment: Past, Present, and Future Tense Oregon Department of Education Dianna Carrizales, PhD Office of Student Learning and Partnerships Office of Assessment and Instruction In conjunction with University of Oregon: Behavioral Research and Teaching Group

2. In the Beginning Accepting the NCLB challenge Accepting the NCLB challenge –Meaningful assessment of students with significant cognitive disabilities Defining the population Defining the population –Functional life skills –Some academic skills

3. The Evolution of a System 2001-2005 2001-2005 2006-2007 2006-2007 10 CIM 8 7 6 5 4 3 8 7 6 5 4 3 Extended CLRAS Extended Assessment Scaffolded Administration Accommodated Assessment General Assessment Targeted Assessment Juried Assessment Accommodated Assessment Modified Assessment Extended K - 2

4. Change and Refinements Accepting the peer review challenge Accepting the peer review challenge –Grade level content –Comparability of forms –Technical documentation –Sound alternate achievement standards based on research-based and documented procedures

5. General Assessment General Assessment General Assessment with accommodations General Assessment with accommodations Extended Assessment Extended Assessment –Elementary –Middle –High Scaffold Administration option of Extended Assessment Scaffold Administration option of Extended Assessment Current Conclusion (2007-2008) 2007-2008 2007-2008 10 CIM 8 7 6 5 4 3 Extended Assessment Scaffolded Administration Accommodated Assessment

6. Consider General Assessment (with or without accommodations) if Student: Student: –Performs at or around grade level –Has academic difficulties that primarily surround reading but may be average or close to average in other subject areas –Has academic difficulties in areas other than reading that are “mild to moderate” and can typically be addressed by using simplified language –Is reading within two to three grades of his or her enrolled level Instruction: Instruction: –Is primarily general curriculum instruction (but may also use a specialized curriculum in some areas)

7. Consider Extended Assessment if Student: Student: –Performs well below grade level –Is significantly below grade level in reading –Has academic difficulties that are generalized (to all subject areas) and are significant –Benefits from specialized individual supports Instruction: Instruction: –Is primarily a specialized curriculum or –From general curriculum must be significantly reduced in breadth, depth, and complexity

8. Consider Scaffolded Administration of Extended Assessment if: Student: Student: – –Performance is significantly impacted by a disability – –Does not read – –Has academic, mobility, and receptive and expressive language difficulties that are generalized and significant – –Relies on individual and significant supports to access reduced content materials Instruction: Instruction: –Is from a specialized curriculum and has functional components and/or –Includes academic goals that are significantly reduced in depth, breadth, and complexity from grade level content

9. Participation by Subject (2006-2007)

10. Participation by Gender (Reading exemplar)

11. 2006-2007 Extended Assessment Participation by Disability Elementary Elementary –Mental Retardation (25.5%) –Specific Learning Disabilities (20.75%) –Autism Spectrum Disorders (16.5%) –Communication Disorder (12.25%) Middle/High Middle/High –Mental Retardation (42%) –Autism Spectrum Disorder (18%) –Specific Learning Disabilities (12.25%) –Other Health Impairments(9.5%)

12. Extended Assessment Description Performance assessment Performance assessment Individually administered Individually administered Response options or selected response (not multiple choice) Response options or selected response (not multiple choice) Scaffold vs. standard administrations Scaffold vs. standard administrations –Direct instruction principles –“Multiple choice” options –Accommodations research Based on grade level content Based on grade level content

13. Development of Grade-Level Items for Oregon’s Extended Assessment Grade-level content standards Grade-level content standards –Reduction in Complexity (one slide) –Reduction in breadth (one slide) –Reduction in depth (one slide) http://www.ode.state.or.us/teachlearn/real/newspaper/ http://www.ode.state.or.us/teachlearn/real/newspaper/ http://www.ode.state.or.us/teachlearn/real/newspaper/ Field and pilot testing Field and pilot testing –Item level piloting (with individual students) –Population piloting (filming and piloting)

14. Reduction in Complexity Development: Development: –Select words with the lowest number of syllables –Reduce number words –Monitor clause structure –Allow opportunities for modeling –Provide more examples when possible –Test rules rather than exceptions –Monitor sequencing to avoid confusion Administration: Administration: –Appropriate pacing –Performance-neutral praise statements

15. Reduction in Depth Evaluate: Evaluate: –Alignment between depth of knowledge in content standards and depth of knowledge in assessment items –Representation of skill-sets required by each item Recognition and reproduction Recognition and reproduction Skill or concept Skill or concept Strategic thinking Strategic thinking Extended thinking Extended thinking

16. Reduction in Breadth Creation of grade bands by typical developmental divisions Creation of grade bands by typical developmental divisions –Elementary, grades 3 – 5 –Middle, grades 6 – 8 –High, grades 9 – 12 (10) Vertical alignment of grade level content standards Vertical alignment of grade level content standards –Standards that span the breadth of the grades assessed Identical Progressive Grade 2Grade 3Grade 4 Read, write, order, model, and compare whole numbers less than 100 Read, write, order, model, and compare whole numbers less than one thousand. Read, write, order, model, and compare whole numbers to one million, common fractions, and decimals to hundredths. Progressive Progressive Identical Grade 4Grade 5Grade 6 Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.

17. Elementary Reading Example

18. Test Description Content prompts: items reflecting content of the test based on grade-level standards Content prompts: items reflecting content of the test based on grade-level standards Prerequisite skills: items taken as a lead-in to the test to determine level of support Prerequisite skills: items taken as a lead-in to the test to determine level of support Access level: Permission of supports (other than accommodations) that can be provided during the performance assessment Access level: Permission of supports (other than accommodations) that can be provided during the performance assessment Scaffold administration: to administer the version with additional prompts, supportive language, and graphics Scaffold administration: to administer the version with additional prompts, supportive language, and graphics

19. Standard Setting Book-marking selected Book-marking selected 32 teachers, 3 days, board-approved 32 teachers, 3 days, board-approved Standards set by grade Standards set by grade Rigorous outcomes Rigorous outcomes Unanswered Mathematics questions remain Unanswered Mathematics questions remain –Difficulty –Population –Administration/training

20. Impact data Outcomes by grade Outcomes by grade

21. Extended Assessment Reading Outcomes Table 1: Extended Reading Impact by Grade, 2006-2007 Grade Does Not Yet Meet Nearly MeetsMeetsExceeds Meets or Exceeds 319%18%36%26%62% 429%15%32%24%56% 530%18%35%17%52% 630%15%31%24%54% 732%18%38%12%50% 838%19%31%13%43% 1039%24%25%13%37%

22. Extended Assessment Mathematics Outcomes Table 2: Extended Mathematics Impact by Grade, 2006-2007 Grade Does Not Yet Meet Nearly MeetsMeetsExceeds Meets or Exceeds 328%36%24%12%36% 452%12%15%21%36% 554%17%19%10%28% 663%23%12%3%15% 769%15%14%2%16% 877%13%7%2%10% 1078%13%9%0%9%

23. Extended Assessment Writing Outcomes Table 3: Extended Writing Impact by Grade, 2006-2007 Grade Does Not Yet Meet Nearly MeetsMeetsExceeds Meets or Exceeds 433%27%34%6%40% 767%11%16%6%22% 1066%3%23%9%31%

24. Extended Assessment Science Outcomes Table 4: Extended Science Impact by Grade, 2006-2007 Grade Does Not Yet Meet Nearly MeetsMeetsExceeds Meets or Exceeds 532%23%37%8%45% 833%48%11%9%20% 1040% 13%7%21%

25. Training structure 2007-2008 2007-2008 –8 regional trainings to train trainers (Oct-Nov) –Held at host sites across the state –300 (2006-2007) assessors will be re-trained on the assessment, decision-making principles, and web- training system to train others in their home-districts –Self-contained web-training site –Data-base of trainers –Training, self-testing, proficiency component monitored at a state level

26. Challenges Instructional content vs. accountability Instructional content vs. accountability Difficulty level Difficulty level Working with the field toward a new mode of thinking about assessing students with significant cognitive disabilities Working with the field toward a new mode of thinking about assessing students with significant cognitive disabilities Who is/is not being tested with this assessment? Who is/is not being tested with this assessment?

27. Lessons learned Expectations can be increased Expectations can be increased Teachers are open to research-based change that is results oriented Teachers are open to research-based change that is results oriented Biggest changes to the assessment since last year: Biggest changes to the assessment since last year: –Use of Prerequisite Skills Biggest changes to training since last year: Biggest changes to training since last year: –Shorter The four “P’s” The four “P’s” –Process –Population –Parallelism –Peer-review

28. New directions Possibly scaling to general assessment Possibly scaling to general assessment Reducing the number of items while increasing the depth and breadth of standards coverage Reducing the number of items while increasing the depth and breadth of standards coverage Reporting Reporting

29. Contact Information Dianna.Carrizales@state.or.us Dianna.Carrizales@state.or.us