1. Lessons Learned in Improving Test Security for State Assessments
John Fremer
President, Caveon Consulting Services
National Conference on Student Assessment
Annual Meeting, Philadelphia
June 21, 2016

3. Context for TILSA Test Security Project
Test Security Issues in the Nation and States
Lots (and growing)!
A “perfect storm” rose up in and has continued across the country into recent years
Some factors for this storm
Increasing stakes, especially for teacher accountability and evaluation
Work of interstate common assessment consortia
Increased attention by the press and news stories
Instances discovered in several large school districts

4. Lessons Learned by States
Prevention
Detection
Follow Up Investigations
Summary

5. Good Prevention Practices
Designing for Test Security
Test Administration Monitoring
Sophisticated Use of Data Forensics Analyses
Training Materials for Schools
Commitment to Continuous Improvement

6. Context for Detection Discussion
Increased Use of Data Forensics
Expanded Methods
Focus on Fairness and Validity
Increased Professional Association Attention
Research Programs and Publications
Development of New Detection Tools, e.g., “Trojan Horse” Items, Digital Watermarking

7. Key Detection Lessons Conservative Interpretations
Development of Communications Programs
Development of Data Forensics Use Plans before Conducting Analyses
Need to Adapt to Change in Delivery Approach

8. Good Detection Practices (1 of 2)
Florida’s Communication Program
Regulations and Legislation in Mississippi

9. Good Detection Practices (2 of 2)
Regular Use of Data Forensics
Incorporating Data Forensics into Scoring and Reporting Cycle
Using Multiple Methods
Similarity
Aberrance
Gains
Erasures/Answer Changes

10. Context for Follow Up Investigations Discussion
Highly Publicized Investigations
Admissions Testing Impersonations Cases, Atlanta Public Schools
Growing Willingness to Give Major Weight to Data
Use of External Investigators

11. Key Follow Up Investigation Lessons
Development of Investigation Plans
Recognition of Need for Fundamental Change in Approach to Investigations
Features of Well-Managed Investigations

12. Summary/ Recommendations (1 of 2)
Need for a Comprehensive Security Plan
Address Security in Accommodations at All Stages
Perform Multiple Data Forensics Analyses

13. Summary/ Recommendations (2 of 2)
Address Threats Posed by Increased Technology Use
Monitor the Internet and Social Media
Develop Media Plan for All Levels of Testing Program Management

14. For More Information
Contact Info
Dr. John Fremer
President, Caveon Consulting Services
215– (cell)