1. Migrating your Paper-n-Pencil Test to the Computer with Security in Mind Jamie Mulkey Caveon Test Security Laura Culver-Edgar NCCAOM

2. ® Caveon, 2006 Secure Test Administration RemediationPreventionDetection

3. ® Caveon, 2006 Agenda  Why move to CBT?  Prevention (Before)  Policy and process considerations  Detection (During)  Monitoring candidates  Remediation (After)  Candidate, record, and data management

4. ® Caveon, 2006 Making the decision to move to CBT  Better access and distribution of testing program  More efficient testing  More flexible testing  Ability to develop/deliver innovative item types  Better test security…

5. ® Caveon, 2006 Security Advantages  Ability to turn items on and off as needed.  Bad items, compromised items, overexposed items.  Unlikely that people will get the same items  CBT, especially CAT can help in this regard  Ability to investigate item-level data better  Response time, unexpected change in ability during the test, etc.  Faster reporting mechanisms  Important in cases of irregular behavior

6. ® Caveon, 2006 Test Administration (Before)  Proper identification procedures  Define authorized/unauthorized devices  Process to report any discrepancies  Proctors and other test administration personnel trained  Interventions are used when suspicious activity occurs

7. ® Caveon, 2006 Security Measures - Biometrics

8. ® Caveon, 2006 Testing continuously vs. testing windows  Size of testing population and the availability of testing sites/seats  Length of test and availability of testing sites/seats  Monitoring of item exposure through internet braindumping/forums/chatrooms  Monitoring of possible item exposure in various time zones  Results to candidate are released immediately or after a period of time  Size of your item pool  Test administration: form-based or CAT

9. ® Caveon, 2006 Test Administration (During)  Managing candidate identification  biometrics  Continuous monitoring or examinees  Personnel, video/audio monitoring  Physical control over unauthorized material and devices  Physical space to hold personal items  Taking advantage of computer technology  Managing an intervention when suspicious activity occurs  Halt testing, notification and escalation

10. ® Caveon, 2006 Confirming exam exposure with Data Forensics™ Helps validate level of exposure  Has a test been compromised and to what extent?  When and where was it compromised?  Who are the potential culprits?

11. ® Caveon, 2006 Caveon Case Study: Global Certification Exams - Effect on Test Performance

12. ® Caveon, 2006 Incidence of Test Abuse by Exam Exam 425 has experienced the greatest incidence of test abuse with suspected test abuse occurring in ~180 of every 1000 tests. High instance of collusion and cheating indicate probable sale and/or sharing of test content on the Internet.

13. ® Caveon, 2006 Test Administration (After)  Relevant data immediately forwarded to the test sponsor  Score validation and reporting  Using Data Forensics to Identify and:  Invalidate scores  Manage rogue test centers  Manage item exposure

14. ® Caveon, 2006 Remediation  Execution of policies  Incident response matrix  Sanctions for poor-performing sites  Investigation  Warning  Closure  Cease & Desist / Take down

15. ® Caveon, 2006 Tier 1

16. ® Caveon, 2006 Security Planning  Roles and Responsibilities  Security Budget  Policies and Procedures  Putting an overall plan together

17. ® Caveon, 2006 Questions & Answers

18. ® Caveon, 2006 Thanks for attending! Jamie Mulkey, Ed.D. jamie.mulkey@caveon.com 916 652-4017 phone www.caveon.com Laura Culver-Edgar, MBA, MT(ASCP) 703 299-5891 lcedgar@nccaom.org Please contact us: