TILSA Test Security Guidebook Preventing, Detecting, and Investigating Test Security Irregularities John F. Olson, Ph.D. President and Founder Olson Educational Measurement & Assessment Services National Conference on Student Assessment National Harbor, MD June 20, 2013

Overview of Presentation Purpose of presentation -- share information from the TILSA Test Security Guidebook A new resource from the Council of Chief State School Officers (CCSSO) for states and others on planning and carrying out prevention, detection, and follow ups on testing irregularities (i.e., cheating on assessments) This resource is available at: www.createspace.com/4302978 www.ccsso.org/Resources/Publications/TILSA_Test_Security_Guidebook.html

Overview of Presentation Background/Context of the Technical Issues in Large Scale Assessment (TILSA) Test Security Project Structure/Organization of the Guidebook Brief Summary of Chapters, Appendices, and Resources on Test Security Note: Three Major Presentations on the Project TILSA – Feb. 2013 NCME – April 2013 NCSA – June 2013

Background and Context for TILSA Project Test Security Issues in the Nation and States Lots (and growing)! A “perfect storm” in 2011-2012 Some factors for this storm Increasing stakes, especially for teacher accountability/evaluation Work of interstate assessment consortia Changing technology in states Increased attention by the press Instances discovered in several large school districts

Background and Context for TILSA Project (cont.) TILSA Test Security Project Feb. 2012 – Feb. 2013 TS Workgroup of state assessment leaders formed Workgroup is continuing its work for TILSA Deliverable -- Comprehensive Guidebook on Test Security Guidebook designed for use by states, the Race to the Top Common Assessment Consortia, and others

Structure and Organization of the Guidebook Comprehensive approach used to outline contents of document Focus on three main aspects of test security – prevention, detection, follow-ups Many additional resources and information in appendices Collaborative approach used with lots of state and associate member input, suggestions, advice, and reviews of drafts Expert external review by Greg Cizek

I. INTRODUCTION AND OVERVIEW Chapter I provides information on the background and context for the TILSA Test Security project (this was presented in previous slides)

II. ONGOING/RECENT ACTIVITIES RELATED TO TEST SECURITY FOR NATIONAL AND STATE ASSESSMENTS U.S. Department of Education Request for Information on Testing Integrity National Center for Education Statistics “Special Symposium on Test Integrity” Data Forensics Summit at University of Kansas CCSSO/Association of Test Publishers Operational Best Practices for Statewide Large-Scale Assessment Programs (2010)

II. ONGOING/RECENT ACTIVITIES RELATED TO TEST SECURITY FOR NATIONAL AND STATE ASSESSMENTS (cont.) National Council on Measurement in Education White Paper on Test Integrity Handbook of Test Security published in March 2013 Summary of TILSA Survey of State Members on Test Security Issues Other important ongoing/recent activities also included in this chapter

Three Main Chapters on Test Security III. PREVENTION IV. DETECTION V. FOLLOW-UP INVESTIGATIONS Note: John Fremer will discuss the info in these chapters

VI. SUMMARY AND RECOMMENDATIONS Three main “takehomes” from Guidebook Test security is increasingly important to states, districts, and others involved in assessment States can really use more information and guidance on how to improve test security The Guidebook will help states by providing this information and guidance Note: More examples, sample checklists, resources, and other info in the Appendices

Questions? For More Information Contact Info John F. Olson Olson Educational Measurement & Assessment Services 617-965-1490 (home/office) 210-859-5593 (cell) jmclkolson@yahoo.com