1. 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA

2. 2 Exam Security: Introducing the Presenters During Test Development Lawrence J. Fabrey, Ph.D. During Test Administration Dave Moore After the Test Chris Beer Tadas Dabsys

3. 3 Benefits of ARELLO-Accreditation Independent Evaluation Confidence and Protection Defensible Standards Reasonable Consistency in Quality Facilitates License Recognition www.ARELLO.org (ARELLO Guidelines for Accreditation) www.ARELLO.org

4. 4 ARELLO Additional Resources About Exam Development An Accreditation Overview ARELLO ® Matters to Consider in Exam RFP ARELLO ® Top Ten Questions and Complaints on Licensing Exams ARELLO ® Supreme Court Case - Exam Validity ARELLO ® Examination Accreditation Program - White Paper on Determining Minimum Passing Scores

5. 5 Security During Examination Development Physical Environment Electronic Access Staffing Inventory Storage Subject Matter Experts Other Functions Translation, Printing, Artwork, etc.

6. 6 Development – Physical Access Secure Room/Area Vault – Key Access Limited Authorized Access Chain-of-Custody Shredding – including notes/drafts

7. 7 Development – Electronic Access Passwords & Screensavers Selective Access Permission Specific Time Period Read-only, non-print, non-export Webinars

8. 8 Development - Staffing Staff Confidentiality Agreements Conflict of Interest Audits

9. 9 Development - SME’s and Other Functions Subject Matter Experts Translators Artwork Supplies for Practical Exams Printers Couriers

10. 10 Development – Inventory and Distribution Printing (Serial #’s, Shrink wrap) Shipping (Traceable, Armed Courier) Electronic Transmission Web Access

11. 11 Security During Exam Administration Paper & Pencil Computer-based Testing Practical Exams Open-book Exams Special Accommodations Facilities & Staff Candidates

12. 12 Administration – Paper & Pencil Printing Shipping Receipt and Inventory Storage Contingency Plan Return Shipment & Inventory

13. 13 Administration - CBT CBT Centers Restricted Login Access Restricted Time Periods Restricted Printing Cameras, DVR’s

14. 14 Administration – Special Administrations Practical Exams Raters (SME) – Conflict of Interest Supplies

15. 15 Administration – Open-book Open-book Exams Notes Highlight/Underline Post-its/Dividers Screening

16. 16 Administration - Facilities Limited Access Spacing Candidates Monitor Restroom Breaks Secure Storage - Supplements Special Accommodations

17. 17 Administration - Staffing Conflict of Interest Self-regulating Restricted Access (Email, Internet) Audits Training – Active Proctoring Termination

18. 18 Administration - Special Special Accommodations Interpreters

19. 19 Administration - Candidates Candidates Identification Roster Signature Fingerprint Login Clothing/Purses Activity/Mannerisms

20. 20 Available through SkyMall and Web

21. 21 Caught on Camera Smart phone Notes

22. 22 I know I will be Caught

23. 23

24. 24 Show Me the Money!

25. 25 Security After the Test Data Forensics Monitoring Websites Item Updating Facility Screening Audits Incident Reporting

26. 26 After – Data Forensics - Items p-value: Percentage of candidates who answer an item correctly pbis:Correlation between performance on an item and performance on the overall exam option stats: Percentage of candidates who answer each option correctly option pbis: Correlation between performance on the options and the overall exam

27. 27 After – Data Forensics - Items p-value: Too high, too low pbis:Too low options:Distractors with no responses Distractors with more responses than key O pbis:Distractors higher than key OmitsAbove expectations

28. 28 After – Data Forensics - Candidates Suspicious Candidate Activity Test for significant differences in pass rates Better performance on harder items High scores in low test times Suspicious improvement in scores

29. 29 After – Monitoring Websites Cram Schools Sample Exams False Advertising Social Networking

30. 30 After – Item Updating Monitor Item Statistics Replace Items Add New Items

31. 31 After – Facility Screening Look for Notes in Testing Station Screen Restrooms and Hallways

32. 32 After - Audits Test Center Audits “Secret Shopper” Programs Inventory Continuing Education for Proctors

33. 33 After – Incident Reporting Documentation Evidence Reporting to Client

34. 34 Examples Cheating Incidents

35. 35 ARELLO Accredited Providers Applied Measurement Professionals  http://www.goamp.com http://www.goamp.com Pearson VUE  http://www.pearsonvue.com http://www.pearsonvue.com PSI Services LLC  http://psionline.com http://psionline.com

36. 36 QUESTIONS? COMMENTS! Fabulous Prize Giveaway