State Assessment Overview and Alternate Assessments December 9, 2015

Today’s Topics The Division of Student Assessment and School Improvement State Assessments and Options for Students with Disabilities Computer Adaptive Testing Federal and State Accountability Questions and Answers

Division of Student Assessment and School Improvement Four individual offices: Assessment Development Program Administration and Accountability School Improvement Test Administration, Scoring and Reporting

State Assessment Options for Students with Disabilities Standards of Learning Tests (SOL) without accommodations SOL Tests with accommodations Virginia Substitute Evaluation Program (VSEP) Virginia Modified Achievement Standards Test (VMAST) Virginia Alternate Assessment Program (VAAP)

Assessment Options for Students with Disabilities Assessment Types Recognized by USDOE Virginia Assessments Assesses Grade-Level State Content Standards? Performance Judged on Grade-Level Expectations? General Assessments Standards of Learning (SOL) Tests Yes Alternate Assessments based on Grade Level Achievement Standards (AA-GLAS) Virginia Substitute Evaluation Program (VSEP) 1 Alternate Assessments based on Modified Achievement Standards (AA-MAS) Virginia Modified Achievement Standards Tests (VMAST) 1 No Alternate Assessments based on Alternate Achievement Standards (AA-AAS) Virginia Alternate Assessment Program (VAAP) 1 Not recognized for Federal Accountability

Assessment Options for Students with Disabilities Virginia Assessments Assessment Format Content Areas SOL Tests Online and Paper Pencil Reading, Writing, Mathematics, Science and History Virginia Substitute Evaluation Program (VSEP) 1 Evidence-Based Virginia Modified Achievement Standards Tests (VMAST) 1 Online Only Mathematics and Reading (only available for students pursing a Standard Diploma with credit accommodations or Modified Standard Diploma) Virginia Alternate Assessment Program (VAAP) 1 Not recognized for Federal Accountability

Assessment Options for Students with Disabilities Virginia Assessments Assessment Participants SOL Tests Student is instructed in grade level content. Disability does not prevent the demonstration of knowledge through multiple-choice and technology enhanced formats with or without accommodations. Virginia Substitute Evaluation Program (VSEP) 1 Student is instructed in grade level content. Disability prevents the demonstration of knowledge through multiple-choice and technology enhanced formats even with accommodations. Student requires an evidence-based assessment to access state assessments. Student has a new disability, a rapid deterioration of sensory functions or a unique disability that prevents the student from accessing the SOL assessments even with allowed testing accommodations. Virginia Modified Achievement Standards Tests (VMAST) 1 Student is instructed in grade level content, but routinely requires significant supports in instruction. Disability does not prevent the demonstration of knowledge through multiple choice and technology enhanced formats, but significant supports are needed for access. Virginia Alternate Assessment Program (VAAP) Student is not instructed in grade level content. Disability requires content that is reduced in depth and complexity. 1 Not recognized for Federal Accountability

Standards of Learning (SOL) Tests with Accommodations Accommodations should: enable students to participate more fully in instruction and assessments to better demonstrate their knowledge and skills, be aligned with and part of daily instruction, foster and facilitate independence for students, not create dependence, and not be introduced for the first time during the administration of a state assessment.

Standards of Learning (SOL) Tests with Accommodations Accommodations must be: based upon individual student need and not upon the category of disability, level of instruction, or program setting, justified and documented in the student’s IEP or 504 Plan, and listed in the Virginia Standards of Learning Assessments Test Implementation Manuals or approved in writing through the Virginia Department of Education Special Assessment Accommodation Request procedure prior to being used on tests that are part of the Virginia Assessment Program.

Virginia Alternate Assessment Program (VAAP) Students with significant cognitive disabilities participate in VAAP. In VAAP students submit work samples that demonstrate their achievement of the Aligned Standards of Learning (ASOL) in each tested content area. The ASOL are based on the SOL but are reduced in depth and complexity. Teachers select one ASOL from each reporting category. In grades 3 through 8 students must submit collections of evidence (COE) in the same subject areas as required of their non-disabled peers. High school students submit COE for reading, writing, mathematics, science, and history.

Virginia Alternate Assessment Program (VAAP) VAAP collections of evidence (COE) are scored locally using a state-provided rubric. Teachers who score the COE are trained using state-provided materials. 10% of COE are also scored by professional scorers to ensure score validity. The number of proficient scores on VAAP assessments that can be included in federal accountability calculations is capped at 1% of the number of students in fall membership for that grade.

Virginia Alternate Assessment Program (VAAP) Virginia Department of Education has been exploring different options for VAAP Participated in a multi-state, grant funded consortium beginning in 2012-2013 under the leadership of the University of Kansas to develop an online alternate assessment for students with significant cognitive disabilities Assessment was called Dynamic Learning Maps, or DLM. Twenty eight school divisions participated in field testing with approximately 165 teachers.

Virginia Alternate Assessment Program (VAAP) In February 2015, based on feedback from school divisions, communications with other states, and DLM staff, the Virginia Department of Education (VDOE) decided not to participate in DLM in 2015-2016. VDOE decided that school divisions would continue to administer and score the VAAP in 2015-2016.

Virginia Substitute Evaluation Program (VSEP) VSEP is intended for a small number of students with disabilities who are unable to participate in the SOL assessments because of their disabilities. Requires the same level of achievement as the SOL tests. Work samples that demonstrate achievement of each SOL are submitted for scoring. Scored by professional scorers to ensure validity. Primarily used as an alternative assessment for the end-of-course tests; although, it is available under very specific circumstances for students in the lower grades.

Virginia Substitute Evaluation Program (VSEP) Available to students with disabilities who are pursuing: verified credits for a Standard Diploma with or without credit accommodations, or verified credits for an Advanced Studies Diploma, or literacy or numeracy certification for a Modified Standard Diploma. Students are not required to attempt the SOL test first. VSEP is NOT accepted for federal accountability; students count as “non-participants.”

Virginia Modified Achievement Standards Test (VMAST) Available only in Algebra I and End-of Course (EOC) Reading to students with disabilities who are eligible to use credit accommodations to earn a Standard Diploma. To be eligible to take VMAST for verified credits toward a Standard Diploma, a student must Meet the VMAST criteria (available online), Pass the high school course, and Score 374 or below on the EOC SOL test after taking the SOL test at least twice

Virginia Modified Achievement Standards Test (VMAST) Available only in Grade 8 Reading and Mathematics to students with disabilities who are pursuing a Modified Standard Diploma. (Note: The Modified Standard Diploma is available only to students who entered the 9th grade for the first time prior to the 2013-2014 school year.)

Virginia Grade Level Alternative (VGLA) Status of the VGLA: Eliminated in 2015-2016 for students with disabilities VGLA reading tests continue to be available for eligible students with limited English proficiency. Superintendent’s Memo #202-14 (August 1, 2014)

Assessment Resources Procedures for the Participation of Students with Disabilities in Virginia Accountability System (www.doe.virginia.gov/testing/participation/index.shtml) VSEP Resources (www.doe.virginia.gov/testing/alternative_assessments/vsep_va_substitute_eval_prog/index.shtml) VMAST Resources (www.doe.virginia.gov/testing/alternative_assessments/vmast_va_mod_achievement_stds_test/index.shtml) VAAP Resources (www.doe.virginia.gov/testing/alternative_assessments/vaap_va_alt_assessment_prog/index.shtml) Standards of Learning Test Implementation Manuals (www.doe.virginia.gov/testing/test_administration/index.shtml#implementation_manuals) Your Division Director of Testing

Leadership Opportunities Be sure to understand the state assessment options, including SOL accommodations, the various participation criteria, and program expectations. You may be: responsible for training IEP/504 teams on assessment topics. consulted on complex cases involving assessment options. responsible for supporting teachers through professional development and technical assistance as they collect evidence for VAAP or VSEP. asked to implement a system for monitoring and pre-scoring collections. responsible for coordinating local scoring events. required to provide guidance to staff working with students in private special education schools, alternative schools, and receiving homebound instruction.

Computer Adaptive Testing: A Brief Overview

Current Testing System All students taking a particular test (grade 5 mathematics) take one of several versions Test forms are constructed by testing contractor staff and reviewed by Department staff and by committees of educators

Computer Adaptive Testing (CAT) In CAT, the computer customizes the selection of items administered to the individual student. The computer scores the student’s responses to a test item and selects the next item based on the student’s response. CAT is iterative in that the process of choosing questions, scoring responses, and selecting new items is repeated throughout the test.

Advantages of CAT Increased precision of measurement, especially for high-performing and struggling students May improve student motivation because content is administered at a more engaging level of difficulty, especially for struggling students Well-suited for measuring growth May require fewer test items

Advantages of CAT May allow for shorter testing times for students Provides improved security since students in the same test session are responding to different items

Test Blueprint for Grade 6 SOL Mathematics Grade 6 Mathematics Test Blueprint Summary Table The Grade 6 Mathematics CAT has 45 operational items and 8 field test items for 53 total items. The Grade 6 Mathematics traditional test has 50 operational items and 10 field test items for 60 total items.

Score Reports Individual score reports will be available within 24 hours after testing is completed. Score reports will include: the student’s score, performance level, and a description of the student’s performance level. item level performance information for the student through the Student Details by Question (SDBQ) report.

CAT Implementation Schedule for Grades 6 – 8 Mathematics, Fall 2015 In Fall 2015, grades 6, 7, and 8 mathematics online tests will be computer adaptive tests, including plain English online versions and audio versions. These tests will be slightly shorter in length than the paper/pencil versions of the grade 6, 7, and 8 mathematics tests. Test blueprints for grades 6, 7, and 8 mathematics have been updated on the VDOE web site.

CAT Resources

CAT Resources A CAT Training Test for use by students taking the CAT test for grades 6, 7, and 8 is available on the DOE Web site. The CAT Training Test provides students with a simulated experience for navigation through a CAT form. Students can practice answering a small number of questions (grade 6 mathematics released items) to familiarize themselves with CAT features that are similar to a traditional test and those that are new or different. The CAT Training Test is shorter than an actual CAT (due to a limited number of grade 6 mathematics released test items) and does not have the same level of customization for students. It serves as a brief simulation for CAT training purposes only.

Accountability

Federal Accountability State Accountability Based on the Elementary and Secondary Education Act (ESEA) Primarily focuses on student performance in reading and mathematics and graduation rates Uses state assessment participation and pass rates as the main indicator of results Considers the performance of all students and subgroup performance Based on the Standards of Accreditation Focuses on student performance in English (reading and writing), mathematics, science and history and graduation rates. Uses state assessment pass rates as the main indicator of results Considers performance of all test-takers in each content area

Accountability: Who Must Be Tested? Under federal law all students are expected to participate in the required reading, mathematics, and science assessments. This expectation includes students with disabilities and English language learners. All students are to be held to the same high academic standards.

Accountability Students with disabilities are required to participate in the state accountability systems (Individual with Disabilities Education Improvement Act of 2004 - IDEA 2004) Students with disabilities are required to participate in all content area assessments that are available to students without disabilities (IDEA 2004) Students with disabilities must be given equal opportunity to participate in and benefit from the policies and procedures customarily granted to all individuals (Code of Virginia)

Students with Disabilities Accountability: Assessment Performance (2014-2015 Results from State School Report Card) State Assessment All Students Pass Rate Students with Disabilities Reading 79% 45% Mathematics 48% Writing 77% 40% History 86% 60% Science 82% 51%

Student Accountability Student accountability is reflected in diploma options for students with disabilities including the: Advanced Studies Diploma Standard Diploma Standard Diploma with Credit Accommodations Modified Standard Diploma (Only available to students who entered 9th grade for the first time prior to 2013-2014 school year) Applied Studies Diploma

Accountability: 1% Cap for VAAP Federal regulations limit the number of proficient and advanced proficient scores on alternate assessments based on alternate achievement standards within the Virginia Alternate Assessment Program (VAAP) that may be counted toward Virginia's Federal Accountability System to one percent of the tested grade levels.

Accountability: 1% Cap for VAAP This regulation is to ensure that students with the most significant cognitive disabilities participate fully in the standards and accountability under the Elementary and Secondary Education Act (ESEA), and that schools receive credit for making progress with these students.

Accountability: 1% Cap for VAAP 1% Cap is applied at the state level. 1% Cap is applied at the division level. 1% Cap is NOT applied at the school, classroom, or teacher level.

Accountability: 1% Cap for VAAP All students exceeding the 1% cap with proficient scores will need to have their performance status reassigned from passing to failing for Virginia's Federal Accountability System.

Accountability: 1% Cap for VAAP The 1% cap is determined: By dividing the number of VAAP proficient/advanced proficient scores by the number of students at each tested grade level. Tested grade levels are 3,4,5,6,7,8, and 11 The number of students at the tested grade level is determined by the September 30th fall membership reported to VDOE by each school division.

Accountability: 1% Cap for VAAP Students in State Operated Programs (SOPs) are not included. Students in Regional Programs are included. Students placed in programs where the school division is responsible for the student’s special education services are included. Such as: Residential facilities Court placements Private schools

Accountability: 1% Cap for VAAP The 1% calculation applies to both Reading and Mathematics 2 separate calculations Pass rates are usually different for each content area Redistribution of scores examined for each content area

Accountability: 1% Cap for VAAP An Example: Judson County 16,400 students in Fall Membership in Grades 3-8 and 11 16,400 X 0.01 = 164 164 is the 1% Cap for Judson County 197 students passed VAAP Mathematics 197 – 164 = 33 scores need to be reassigned to meet the division’s 1% Cap in Mathematics

Leadership Opportunities The number of students participating in VAAP at each school may need to be monitored: Identify the number of VAAP participants as early as possible in the school year. Rule of Thumb: 1% of Fall Membership for Grades 3-8,11 Select regular times to monitor (e.g., Monthly, End of Semester). Share findings with school administrators. Provide VAAP participation criteria information and training to IEP team members. Provide training to IEP coordinators using the guidance document: VAAP Participation Criteria and the Determination of Significant Cognitive Disabilities (PDF)

Leadership Opportunities The performance of students with disabilities on state assessments counts and will be included in the federal and state accountability calculations As a special education leader, you may be asked or expected to provide leadership to improve student performance on state assessments. This may include developing a plan: for the school division, for working with specific schools, to make gains in particular content areas, or to address graduation rates

Leadership Opportunities The performance of students on state assessments, including students with disabilities, is dependent on the quality of instruction being provided on the Standards of Learning and the Aligned Standards of Learning. How can you be an instructional leader to those who are directly responsible for providing instruction to students with disabilities?

Ensuring Quality Instruction Our accountability system is based on the Standards of Learning (SOL). The SOL are our content standards and represent what teachers are expected to teach and what students are expected to know. To ensure quality instruction on the SOL, teachers serving students with disabilities must have a thorough understanding of each standard and be able to effectively incorporate the teaching strategies, supports, and tools that will address individual disabilities.

Ensuring Quality Instruction Instructional Resources: Standards and Curriculum Framework by Content Area www.doe.virginia.gov/testing/sol/standards_docs/english/index.shtml www.doe.virginia.gov/testing/sol/standards_docs/mathematics/index.shtml www.doe.virginia.gov/testing/sol/standards_docs/science/index.shtml www.doe.virginia.gov/testing/sol/standards_docs/history_socialscience/index.shtml SOL Resources and Instructional Strategies ttaconline.org/standards-of-learning Disability Resources ttaconline.org/resources

Putting the Pieces Together Regulations Parents Students Graduation Rates Accreditation and Federal Accountability Professional Development State Assessments VDOE Principals Teachers Superintendents School Boards 50

10 Tips: Keeping Up with State Assessments for Students with Disabilities Develop a good working relationship with the Division Director of Testing in your division. Ask that all testing memos regarding students with disabilities be forwarded to you. Review Superintendent Memos each week at www.doe.virginia.gov. Memos are posted every Friday. Attend VDOE sponsored workshops on alternate assessment programs

10 Tips: Keeping Up with State Assessments for Students with Disabilities Review the Student Assessment and School Improvement webpage at www.doe.virginia.gov/testing/index.shtml. Read and refer to the following documents regularly (current versions) Procedures for Participation of Students with Disabilities in Virginia’s Accountability System Virginia Alternative and Alternate Assessments Administrator Manual VAAP and VSEP Implementation Manuals SOL Test Implementation Manuals (Appendix C)

10 Tips: Keeping Up with State Assessments for Students with Disabilities Analyze your division and school assessment data to find areas of strengths and areas needing improvement among students with disabilities. Identify areas for professional development needs. Review your IEPs to see if accommodations and assessment decisions are clearly documented. Staff may need to be reminded that accommodations are designed to provide equal access to assessments for students with disabilities. Accommodations based solely on the potential to enhance performance are inappropriate.  

10 Tips: Keeping Up with State Assessments for Students with Disabilities Follow-up and monitor assessments for students educated outside of your division in other school divisions, regional programs, private schools, etc. Follow-up and monitor special education students in your division such as home-bound students and students in alternative schools. Ensure that instructional programs for students in your division support their preparation for state assessments.  

Where to Seek Answers or Assistance Special Education Student Assessment VAAP 1% Cap Guidance regarding significant disabilities Standard Diploma with Credit Accommodations Local Alternate Assessments for Students with Disabilities Technical assistance for state assessments including SOL accommodation questions Timelines and due dates VAAP scoring and data entry Timelines and due dates for VSEP Local scoring of VAAP collections Understanding student assessment data

Division of Student Assessment and Questions Division of Student Assessment and School Improvement student_assessment@doe.virginia.gov (804) 225-2102