Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Accountability and Effectiveness: Maintaining Security at Test Centers Cynthia Miller Murphy, RN, MSN, CAE Executive Director Oncology Nursing Certification Corp.

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 How do we, as test sponsors, verify that our examinations are administered in a secure manner?

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Continuous Quality Improvement Ongoing Process Before examination During examination After examination

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Before Examination Standardized training of all proctors and supervisors Standardized policies and procedures for test administration

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Before Examination Assure that all candidates meet eligibility criteria

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Before Examination: Assure that candidates are well- informed about what security measures to expect

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Inform Candidates About Security Through: Candidate bulletin/handbook Website Direct communications: Authorization to test letters Admission letters

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Inform Candidates About: Identification requirements Sign in/out of test center Personal belongings Use of study aids Use of scratch paper Breaks

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Inform Candidates About: Direct and indirect observation/monitoring Signing confidentiality/security statements Grounds for dismissal/cancellation of results

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 During the Examination Scheduled and Unscheduled Observation

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Paper/pencil Test Site Observers Certified professionals voluntarily observe the test administration at local test sites Selection of sites – convenience, random, specific Select approximately 20% of local test sites

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Paper/pencil Test Site Observer Evaluation Complete a specific questionnaire Provides immediate feedback Proctor behavior Incidents

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Computer-based Testing Observers Test candidate observers who are Board or Test Development members

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Computer-based Test Observers Register for and take the examination Observe all security measures Report back to test sponsor

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 After Examination Data from candidates surveys complaints Reports from testing company irregularity incidents summaries

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Candidate Surveys Ongoing – every administration Test Center Survey - immediate “After Test” Quality Improvement Survey – administered 2-3 weeks after test

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Test Center Survey Paper/pencil Testing Security and quality of the testing experience from receipt of admission letter to completion of the test Additional page for specific test question comments Results for individual test sites and summary

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Test Center Survey “CBT Exit Survey” Five minutes included in “seat time” Security measures Overall satisfaction Results by individual sites

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Test Center Survey Data on the compliance of supervisors and proctors at local test centers Data on the adequacy of local test site facilities Specific feedback to testing company

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 After Examination Irregularity Reports Incident Reports Candidate Complaints

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 After Test Quality Improvement Survey link to random selection of 20% of test candidates Comprehensive customer service review from first contact with test sponsor through test administration, including enforcement of security measures Web-based survey

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 After Test Quality Improvement Survey Responses are anonymous Obtain quantitative and qualitative data Summarize by test sites and overall

Presented at CLEAR’s 23rd Annual Conference Toronto, Ontario September, 2003 Accountability and Effectiveness Review all data Provide specific feedback to testing company Require specific improvement plans