1. Test Security Company Perspective Presented by: John Fremer, President, Caveon Consulting Services NCSA – June 2014

2. Overview of Presentation  Cannot Let Security For “Non-mainstream" Children Be An “Afterthought”  Our Goal is Meaningful Scores – We Have to Do Many Things Right At Every Stage  Prevention  Detection  Investigations  Special Challenges Of On-Line Assessment  Sharing What Is Known Is A Key Step

3. Test Security at Every Stage  Prevention & Planning  Communicate Zero Tolerance  Clear & Consistent Messages  Qualified Personnel – For All Accommodations  Training  Monitoring

4. Prevention & Planning (Cont.)  Test Security Manager  External Test Security Audit  Formal Test Security Standards  Security Vulnerabilities Identified  Recommendations for Correcting

5. DETECTION  Use Multiple Data Forensics Methods  Unusual Gains  Similarity  Erasures (Answer Changing)  Person Fit Analyses

6. DETECTION (cont.)  Adhere to Legal and Regulatory Requirements  Importance of “test user” agreement  Clarify potential consequences of not following testing rules  May want to seek changes to laws or regulations

7. TEST SECURITY INVESTIGATIONS  Areas Needing Attention  Level of evidence warranting an investigation  Roles for state, district, and other staff  Time lines  Model investigations kit

8. TEST SECURITY INVESTIGATIONS (cont.)  Focus Primarily on Improving Security in the Future  Follow “Innocent Until Proven Guilty” approach, but be firm in your questioning  Give most attention to the “Worst of the Worst”  Consider other possible explanations for results

9. SPECIAL CHALLENGES OF ON-LINE ASSESSMENT  Involvement of Readers or Scribes  Selection  Training  Additional Access to Test Materials  May Need to Reproduce Test Materials  Timing  Controls & Monitoring

10. SPECIAL CHALLENGES OF ON-LINE ASSESSMENT (Cont.)  Absence of Formal Policies  Lack of Experience  Availability Of Additional Presentation Features  Facilitates large type  System design may interfere with needed accommodations  Need experienced assistance

11. SHARING WHAT IS KNOWN  Among and Across States  Operational Best Practices  TILSA Guidebook  Possible Additional Material  Learning From Others  ETS, College Board, ACT  Certification Programs  NCEO

12. FOR MORE INFORMATION: John Fremer Caveon Consulting Services Caveon LLC 215-805- 3007 (cell) John.fremer@caveon.com