Ensuring Test Data Integrity Tracy Cerda Cheryl Alcaya Minnesota Assessment Conference August 5, 2015 “Leading for educational excellence and equity. Every day for every one.”

Ensure the validity of test score interpretation  Report accurate information  Provide resources and interventions where most needed  Provide recognition where merited Protect investment in test content Why Test Security Matters education.state.mn.us 2

Why Now? education.state.mn.us 3 I am writing to urge you to do everything you can to ensure the integrity of the data used to measure student achievement and ensure meaningful educational accountability in your State. As I’m sure you know, even the hint of testing irregularities and misconduct in the test administration process could call into question school reform efforts and undermine the State accountability systems that you have painstakingly built over the past decade. I urge you to make assessment security a high priority by reviewing and, if necessary, strengthening your efforts to protect assessment and accountability data, ensure the quality of those data, and enforce test security.

Why Now? (2) education.state.mn.us 4 Recommendations Prevent test administration irregularities. Detect and analyze test administration irregularities. Respond to and investigate possible test administration irregularities. Ensure the integrity of computer-delivered tests.

Why Now? (3) education.state.mn.us 5 Recommendations Monitor schools identified as high risk for having test administration irregularities and share the results of monitoring with LEAs. Strengthen prevention and handling of test administration irregularities, including formal processes for timely reporting and resolution and documentation of corrective action. Strengthen test security by training test administrators, safeguarding test materials and access to online systems, and requiring timely test administration reports from contractors. Ensure that LEAs and schools put in place procedures that will prevent irregularities from occurring.

Why Now? (4) education.state.mn.us 6

Student Misconduct Educator Misconduct Threats to Test Data Integrity Addressed by Test Security Policies & Procedures education.state.mn.us 7

The Test Policies and Procedures Committee Noted: education.state.mn.us 8 …the majority of Minnesota educators behave ethically and follow the rules for test administration…

The Test Policies and Procedures Committee Noted: education.state.mn.us 9 …however, unethical or illegal conduct by a single educator has the potential to have a widespread negative impact on the integrity of test data.

Included MDE staff and external advisors Met October 2014 – June 2015 Reviewed MDE test security policies and procedures Recommended improvements to test security in Minnesota Issued a final report, soon to be posted to the MDE website The Test Policies and Procedures Committee (TPPC) education.state.mn.us 10

PreventionDetectionInvestigationResolution Test Security Cycle education.state.mn.us 11

Prevention: Include Testing in Your Culture of Academic Integrity education.state.mn.us 12

Train-the-trainerMini module for staff Prevention: Training, Training, Training education.state.mn.us 13

Prevention: District Test Security Procedure Each district is required to have a District Test Security Procedure MDE will begin to conduct desk audits of select districts’ procedures education.state.mn.us 14

Prevention: Assurance of Non-Disclosure and Test Security education.state.mn.us 15

Prevention: Active Monitoring education.state.mn.us 16

Prevention: Site visits by MDE and District Staff education.state.mn.us 17

Prevention: Test site setup, assigned seating, link monitors to students education.state.mn.us 18

Prevention: Cell phone/device policies education.state.mn.us 19

Data forensics to detect cheating in online assessment  Changes to responses: W>R, R>W, W>W  Response latency  Response patterns  And many more in development Forensics generally conducted by testing contractor Detection: Remember Erasure Analysis? education.state.mn.us 20

Security breach is reported in the district Security breach is reported to MDE: Test Security Notification Tip Line MDE requires the district to investigate and document findings Investigation: What Really Happened? education.state.mn.us 21

ConsequencesMedia Requests Investigation: What Happens Next? education.state.mn.us 22

education.state.mn.us 23 Resolution: Completing the Cycle District investigation and documentation submitted to MDE MDE review of all security breach documentation and follow-up Resulting actions by MDE or District: Invalidations Personnel Action MDE and District work group to identify appropriate consequences Learning from the process and enhanced prevention

Key Policy Letters from the Education Secretary or Deputy Secretary, June 24, Available at National Center for Education Statistics (2013). Testing integrity symposium: Issues and recommendations for best practice. Washington, DC: Author. Available at symposium.pdf. symposium.pdf U.S. Department of Education, Office of Inspector General (2014). The U.S. Department of Education’s and Five State Educational Agencies’ Systems of Internal Control over Statewide Test Results. Washington, DC: Author. Available at 1.pdf. 1.pdf principal-sentenced-in-connection-with-mcas-cheating-scheme principal-sentenced-in-connection-with-mcas-cheating-scheme convicted-aps-educators/nktD6/ convicted-aps-educators/nktD6/ References education.state.mn.us 24