Maintaining Test Security in a Climate of Innovation Walt Drane, Mississippi Karlynn Laraway, Idaho Scott Mantie, New Hampshire

Agenda Protecting Your Assessments State Experiences Common K-12 Threats Best Practices State Experiences Mississippi Idaho New Hampshire NCSA 2017

K-12 Test Security Threats Coaching during test administration Use of unauthorized technology Pre-knowledge of test content/item theft NCSA 2017

Protection Best Practices Strong policies, procedures, and training Communicating consequences of non-compliance NCSA 2017

Protection Best Practices On-site monitoring during administration Internet and social media monitoring NCSA 2017

Protection Best Practices Data forensics of test results Managing and investigating incidents NCSA 2017

Test Security Challenges Online testing Data privacy Stakeholder attitudes Changing legislation or policies NCSA 2017

Walt Drane, Executive Director Mississippi Walt Drane, Executive Director NCSA 2017

Mississippi Assessments in K-12 High stakes assessments are required in both federal and Mississippi state law: ESEA section 1111(b)(3) (20 U.S.C. § 6311 (b)(3)) MS Code 37-16-1 K-Readiness Assessment English Language Arts – 3rd grade through 8th grade and English II EOC Mathematics – 3rd grade through 8th grade and Algebra I EOC Science – 5th and 8th grade and Biology I EOC U.S. History – 11th grade EOC

High Stakes in Nature High stakes for both the students and the educators: Teacher and Principal Evaluations Gateway Assessments 3rd Grade Reading Summative Assessment Graduation End-of-Course Assessments Algebra I, English II, Biology I, and U.S. History Statewide Accountability Model Rate Districts and Schools A, B, C, D, F Ratings Proficiency and Growth Calculations

Test Security = Exam Integrity Exam Integrity = Validity SECURITY = INTEGRITY Test Security = Exam Integrity Exam Integrity = Validity A Test Security Incident threatens the integrity of the assessment and could undermine or damage the VALIDITY of the assessment results and all decisions based upon those results.

Risks, Threats, Vulnerabilities A vulnerability is a feature of your program that can be exploited. A threat is a potential action by one or more individuals that could compromise your exam. Threats exploit vulnerabilities. A risk is the potential for your program to suffer loss as a result of a test security breach.

Test Security Threats Brain Dumping Chunk and Redirect Teachers participate in Item Writer Workshops to create assessments in Mississippi. Threats exist around teachers memorizing content or concepts Regurgitating assessment questions, content and concepts to students Chunk and Redirect Lawanda Tyler-Jones Principal Clarksdale School District Copying items, topics, and authors Pacing Students

Test Security Threats Drill it and Kill it Send Weaker Students Home Let Stronger Students Take the Assessment First Utilize the same Test Administrator and Proctor Throughout the Testing Window

Strategies for Contending with Test Security Threats Before Testing Train DTCs twice a year Confidentiality Forms Electronic Device Policy District and School Test Security Plans During Testing Forward Assessment Support Team (FAST) 1-800 Hotline Anonymous Tip Web Link State Control of Statewide Testing (MS Code 37-16-3 1(f))

Strategies for Contending with Test Security Threats After Testing Data Forensics (MS Code 37-16-4) Principal Certification Form (MS Code 37-16- 3 3) State Misdemeanor for Test Security Violations (MS Code 37-16-4) Educator Misconduct – Potential Loss of License (MS Code 37-3-2) District May Have to Reimburse the State for Investigations (MS Code 37-16-3 (f))

Karlynn Laraway, Director of Assessment Idaho Karlynn Laraway, Director of Assessment NCSA 2017

Idaho Computer based assessments since 2002 Test Security Awareness and Attitudes Effort to re-engage district and school leadership in the importance of test security Test Security Information NCSA 2017

Incident Prevention: Assessment Integrity Guide Regional Training – Breakout Session Webinar live/recorded Assessment Confidentiality Agreement Training Required for all district test coordinators in 2017/18 NCSA 2017

Incident Detection Vendor Test Administration System Test Incident Log https://apps.sde.idaho.gov/testincidentlog Cheating Analysis NCSA 2017

Incident Management/Escalation Idaho State Department Philosophy Flashlight not a Hammer Student Violations – local decisions Teacher Violations – local decision Referral to Professional Standards Commission NCSA 2017

Monitoring Test Security Desk Review Onsite Visit Recommendations NCSA 2017

Scott Mantie, Director, Division of Program Support New Hampshire Scott Mantie, Director, Division of Program Support NCSA 2017

New Hampshire In addition to the above… “Innovation” may also mean… State PII regulations exacerbating efforts for test security Anonymous student connections with testing vendor(s) Inability to share any PII information outside of state “Transparency” initiatives threaten test item exposure NCSA 2017

Contact Information Walt Drane WDrane@mdek12.org Karlynn Laraway klaraway@sde.idaho.gov Scott Mantie Scott.Mantie@doe.nh.gov Sally Valenzuela Sally.Valenzuela@caveon.com NCSA 2017