Slide 1 National Center on Educational Outcomes (NCEO) Test Security Policies and Procedures for Alternate Assessments Sheryl Lazarus Meagan Karvonen NCEO University of Kansas Leila Williams Melissa Gholson Arizona Department of Education West Virginia Department of Education Toni Wheeler Idaho Department of Education National Conference on Student Assessment (NCSA) San Diego, CA June 22-24, 2015

Slide 2 II. Standards and Guidelines that Address Cheating or Test Security Standards for Educational and Psychology Testing (APA/AERA/NCME, 2014) Practices for Statewide Large-Scale Assessment Programs (ATP/CCSSO, 2013) Testing and Data Integrity in the Assessment of Student Achievement (NCME, 2012) National Center on Educational Outcomes (NCEO)

Slide 3 Standards for Educational and Psychology Testing The Standards indicate that: “ Regardless of the purpose of testing, the goal of fairness is to maximize, to the extent possible, the opportunity for test takers to demonstrate their standing on the construct(s) that test is intended to measure. (p. 51) “ Test users must balance test security with the rights of all test takers and test users. (p. 117) National Center on Educational Outcomes (NCEO)

Slide 4 Operational Best Practices for Statewide Large-Scale Assessment Programs National Center on Educational Outcomes (NCEO) The Operational Best Practices indicate that the document addresses: “...best practices for achieving test security throughout the process of all paper-based and technology-based assessments, using methods that support the needs of students with disabilities and English learners. (p. 81)

Slide 5 Testing and Data Integrity in the Administration of Statewide Student Assessment Programs (NCME) National Center on Educational Outcomes (NCEO) “ Policies and procedures must ensure that all students have appropriate, fair, and equal opportunities to show their knowledge, skills, and abilities. Students who need accommodations due to language differences or students with disabilities may require appropriate modifications to materials and administrative procedures to ensure fair access to the assessment of their skills.

III. Analysis. Analysis of States’ Test Security Policies Source: Test security and students with disabilities: An analysis of states’ test security policies (Synthesis Report 95). National Center on Educational Outcomes. Available at:

Slide 7 State policies downloaded for the 50 states + DC Documents analyzed included: Test procedures or administration manuals, general test security documents, accommodations manuals Analyzed documents for both the regular and alternate assessments National Center on Educational Outcomes (NCEO) Procedures

Test Security & Alternate Assessments 27 States have test security policies that specifically addressed alternate assessments.

Slide 9 Reminder that AA-AAS Should be Treated Similarly to Other Assessments National Center on Educational Outcomes (NCEO) “ [The Alternate Assessment] is a state assessment and should be treated as such. This means district and school administrators have the responsibility to develop policies regarding who can see a student's datafolio. “ [The AA-AAS] is a valid and reliable test. It is to be treated as securely as the general assessment in every way. Under no circumstances is this test to be used in any other way than as a summative assessment during the approved time frame. State Examples:

. Of these 27 states, the policies addressed:

Slide 11 Type of Assessment Affected How Security Was Addressed in Policy National Center on Educational Outcomes (NCEO) State Examples Item-based performance tasks “ Test books and student answer documents must be kept secure. Students must not be exposed to test content before the actual testing. If students have prior knowledge of test content, results of testing can give a deceptive picture. Performance tasks “ The performance tasks are considered secure testing materials. “ Tasks are secure materials and should not be sent home with the student.

Slide 12 Type of Assessment Affected How Security was Addressed in Policy, Cont. National Center on Educational Outcomes (NCEO) State Examples Portfolios “ Once the portfolios contain student identifying information, student testing materials, and student work, the portfolios become secure documents and must be treated with the same care as other secure testing materials, keeping them in a locked file cabinet. Checklist “ Checklist is not a secure document. This means that the checklist may be copied.

Slide 13 Duplication and Copying of AA-AAS Test Materials National Center on Educational Outcomes (NCEO) Allowed “ One week prior to the opening of the test administration window, the materials may be printed and then photocopied from a master copy for individual student use. Not Allowed “ The section of the manual that contains tasks may not be duplicated. Unauthorized copying or reuse of any questions within the Examiner’s Manual is illegal. Do not copy any part of the tests or test materials (excluding stimulus cards). State Examples

Slide 14 AA-AAS Administrator Qualifications National Center on Educational Outcomes (NCEO) “ Only authorized district personnel are allowed to download, distribute, and/or enter student data for the Alternate Assessments. Authorized personnel include District Test Coordinators as well as individuals trained in the Alternate Assessment who also have the appropriate permissions (e.g., Qualified Trainers and Qualified Assessors) and security clearance on file at the district office. Most state policies make no distinction between regular assessment and regarding who may administer the AA-AAS... but a few specifically addressed AA-AAS administrator qualifications. State Example

Slide 15 Training Requirements (All Assessments) National Center on Educational Outcomes (NCEO) “ Ensure all personnel involved with testing receive training on appropriate test administration, policies, and procedures including accommodations. State Examples Most state policies did not make training distinctions between administrators of the general assessment and AA-AAS.

Slide 16 Changes to Standard Test Procedures --Wording National Center on Educational Outcomes (NCEO) Allowed “ The instructions to be given to the student are specified for each task. In many cases, the detailed wording that is given in this section will need to be modified to make it appropriate for the particulars of the testing situation. The suggested wording is very simple and direct. The tester should give instructions that are as close as possible to the example, while making the instructions fit the testing situation. Not Allowed “ Test administrators must not change the wording or content of the task. Test administrators must not change the order of the answer options. State Examples

Slide 17 Considerations When Including Students with Disabilities in Test Security Policies Ask 1. Are all test administrators qualified? 2. Does everyone who accesses the assessment sign a test security/ confidentiality agreement? 3. Are adequate logs/records kept? 4. Are accessibility features and accommodations with the fewest security risks identified as preferred features and accommodations?. Available at:

Slide 18 Considerations When Including Students with Disabilities in Test Security Policies, Cont. Ask 5. Do test security policies and procedures address adaptive technology? 6. Are test security procedures in place for situations when a test is accessed prior to administration? 7. Are appropriate procedures used for small group and individual administrations? 8. Are appropriate test security procedures in place for alternate assessments?..

Slide 19 National Center on Educational Outcomes (NCEO) For More Information National Center on Educational Outcomes Sheryl Lazarus

Slide 20 National Center on Educational Outcomes (NCEO) References American Education al Research Association (AERA), American Psychological Association (APA), National Council on measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: Author. Association of Test Publishers (ATP)/ Council of Chief State School Officers (CCSSO) (2013). Operational best practices for statewide large-scale assessment programs. Washington, DC: Author. National Center for Education Statistics (NCES). (2013). Testing integrity symposium: Issues and recommendations for best practice. Washington DC: Institute of Education Sciences (IES). Washington DC: Institute of Education Sciences (IES). Retrieved from National Council on Measurement in Education (NCME) (2012). Testing and data integrity in the administration of statewide student assessment programs. Madison WI: author. Olson, J.F. & Fremer, J. (2013). TILSA test security guidebook. Washington, DC: Council of Chief State School Officers (CCSSO).

Slide 21 National Center on Educational Outcomes (NCEO) The National Center on Educational Outcomes is supported through a Cooperative Agreement (#H326G110002) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. The Center is affiliated with the Institute on Community Integration at the College of Education and Human Development, University of Minnesota. The contents of this document were developed under the Cooperative Agreement from the U.S. Department of Education, but does not necessarily represent the policy or opinions of the U.S. Department of Education or Office within it. Readers should not assume endorsement by the federal government. Project Officer: David Egnor