Computer Based Testing Security & Administration 2007 North American Pesticide Applicator Certification & Safety Education Workshop Portland Maine 2007

CBT Security & Administration 10 Minute Review Metro Institute Background Metro Institute Background Testing Security Basics Testing Security Basics CBT System Overview CBT System Overview Step-by-Step Procedure Step-by-Step Procedure System Reports System Reports Concluding Statements & Questions Concluding Statements & Questions

Metro Institute Background Training Institute Training Institute –Initial Applicator Training & CEU’s Computer Based Testing – 2003 Computer Based Testing – 2003 –3GB USU Research Center –Began Providing CBT for Arizona SPCC Present Goals – 2007 Present Goals – 2007 –CBT & Web Training Worldwide –Implement CBT Nebraska, Alabama, N. Carolina, S. Carolina –National Pesticide Applicator Core Exam – CBT Conversion

Testing Security Basics Server Physical Security Server Physical Security –Hosted by Dedicated ISP Provider –Offsite Backup Performed Every Hour Test Data Security Test Data Security –Dual Server System – Web & Data –ISP Firewalls & Port Blocking & Password –MD5 Login Encryption –SSL Data Transfer Encryption –No ActiveX or Java Applets Employed

Testing Security Basics Agency Data Security Agency Data Security –Test Applications to Metro via FTP –Test Scores Returned to Agency Via FTP Testing Sites Security Testing Sites Security –Proctored & Monitored –Photo ID Required, Photo Taken Testing Computer Security Testing Computer Security –Hardened Kiosk - Stores No Data Ever

CBT System Overview

Step-by-Step Procedure Application Application Server Operations Server Operations Testing Appointment Testing Appointment Arrival at Testing Site Arrival at Testing Site Accessing Test Questions Accessing Test Questions Testing Process Testing Process Test Completion Test Completion Final Data Delivery to Agency Final Data Delivery to Agency

Application Application Complete Application Complete Reviewed & Approved Reviewed & Approved Agency Data Entry Agency Data Entry Letter to Applicator Letter to Applicator Cued for Daily FTP Cued for Daily FTP –Name & ID # –Date Approved –Tests Approved

Server Operations Web Server solicits & receives data from Agency Server Web Server solicits & receives data from Agency Server Web Server submits password to Data Server for transfer Web Server submits password to Data Server for transfer Data transferred from Web Server to Data Server thru firewall Data transferred from Web Server to Data Server thru firewall

Testing Appointment Applicant calls Metro with ID number Applicant calls Metro with ID number Metro queries Web Server for authority to offer test Metro queries Web Server for authority to offer test Approval confirmed & test set for specific date & site thru Web Server Approval confirmed & test set for specific date & site thru Web Server

Arrival at Testing Site Photo ID is requested & identification confirmed Photo ID is requested & identification confirmed Digital photo taken & sent to test-taker file Digital photo taken & sent to test-taker file Test-taker proctored into specific test, computer IP address confirmed Test-taker proctored into specific test, computer IP address confirmed Test-taker gets color scratch paper, enters ID#, agrees to terms, begins testing Test-taker gets color scratch paper, enters ID#, agrees to terms, begins testing

Accessing Test Questions Six events must be completed in sequence to receive test question: Six events must be completed in sequence to receive test question: –Application Received –Test Appointment –Day & Location –Proctor ID’s, Photos & Releases Test –IP Address Confirmed –Proctor Signs in test-taker to workstation

Testing Process Test questions/answers randomly generated & sent in multiple packets Test questions/answers randomly generated & sent in multiple packets Packets may contain one or two words Packets may contain one or two words Packets may or may not take same route Packets may or may not take same route Answers return to Web Server as single number Answers return to Web Server as single number

Test Completion Tester may review questions & answers until timeout Tester may review questions & answers until timeout Once “Finish” button selected (or timeout) test is recorded Once “Finish” button selected (or timeout) test is recorded Completed test can not be altered Completed test can not be altered Score sheet is printed Score sheet is printed 100%

Final Data Delivery to Agency Test-taker scores & pictures sent from Metro Web Server to Agency File Server via FTP daily Test-taker scores & pictures sent from Metro Web Server to Agency File Server via FTP daily Agency issues License to all test-takers with passing scores Agency issues License to all test-takers with passing scores

Reports Histogram – Score Distribution Histogram – Score Distribution Discrimination Index Discrimination Index Test Question – Sort by % Correct Test Question – Sort by % Correct Test Results – Sort by Location Test Results – Sort by Location Average Time to Complete Test Average Time to Complete Test Applicator Test Look-up Applicator Test Look-up Applicator Photo Look-up Applicator Photo Look-up Test Questions & Answers Test Questions & Answers

Histogram

Discrimination Index

Test Question % Correct

In Conclusion Metro Online CBT System Attributes: Metro Online CBT System Attributes: –More Secure - Paper-based & computer transmitted testing systems vulnerable –Allows Flexibility - Timely question changes –Secure & Proven - Procedures & protocols taking advantage of latest technology –Reports – Any analysis provided –Free to Agency - Saves money allowing staff & office space reduction or reassignment