Pre-Employment Testing Software Using Emergency Response scores from the Data Entry MT and Data Entry MT (Audio) test modules for hiring purposes Pre-Employment Testing Software Click on the screen, use the scroll bar, or press the right arrow key to view the next slide. Press the left arrow key to go back to the previous slide. Note: No audio accompanies this presentation Copyright 2007 © Biddle Consulting Group. Inc.

Please send comments or suggestions to info@criticall911.com This presentation is provided to CritiCall users as part of their CritiCall Elite Service Plan Membership Please send comments or suggestions to info@criticall911.com CritiCall® is a registered trademark of Biddle Consulting Group, Inc. 800-999-0438

Please keep in mind… This training session applies ONLY to the Emergency Response scores associated with the Data Entry MT and the Data Entry MT (Audio) test modules The 60% minimum scores recommended for the Emergency Response portion of these two modules are based on a national survey that collected input from over 150 experienced dispatchers and calltakers from a wide variety of PSAPs from around the country Even so, to insure fairness job experts from your agency should also endorse this 60% minimum score following the procedures described in this presentation before it is used for selection purposes

Will an automated version of this process become available in the future? We will be including an automated process for using the Emergency Response scores from the Data Entry MT and Data Entry MT (Audio) test modules in a future upgrade to CritiCall In the meanwhile, we wanted to offer you the option to use these scores manually

IMPORTANT Testing must be performed on CritiCall Version 4.0.3 or higher if you wish to use the Emergency Response scores from these two test modules Use Emergency Response scores from ONLY the Data Entry MT and Data Entry MT (Audio) test modules for making hiring decisions

This update will not work with CritiCall versions prior to 4.0 If you have not already installed Version 4.0.3 or higher installed, go to the www.criticall911.com/updates.html web page to download the update This update will not work with CritiCall versions prior to 4.0

Before you use the 60% minimum score for the Emergency Response portion of these two test modules Have 7 to 10 of your subject matter experts take the Data Entry MT and/or Data Entry MT (Audio) test module with the Validation Wizard turned “on” Have your Subject Matter Experts (SMEs) read and understand the text on the following page before they take these test modules with the Validation Wizard turned “on” The wording on the next slide is also available in printed form from the CritiCall User’s Group Web pages Tips section

“After you have taken the Data Entry MT and/or Data Entry MT (Audio) test modules you will be asked to indicate a minimum Keystrokes-per-Hour (KPH) score that a minimally qualified applicant should obtain in order to pass and be considered for hiring, promotion, or transfer into the job you are validating the test for (e.g., dispatcher, calltaker, etc.). When recommending a cutoff score please keep in mind that this agency will also require the test taker to correctly respond to a minimum of six out of ten of the emergency response decision-making items while simultaneously entering the data during these two test modules (and only these two modules).”

You should you use the 60% minimum for the Emergency Response portions of the Data Entry MT and Data Entry MT (Audio) only after your job experts have suggested a Keystrokes-per-Hour cutoff score using the process described on the previous two slides The steps to using the Emergency Response Scores from the Data Entry MT and the Data Entry MT (Audio) test modules are as follows…

First, make certain that the cutoff scores for all of the test modules in the AutoTest Code you will be using are appropriately set using the “Set Cutoff Scores” feature in the CritiCall Administrator program Note: You cannot set a cutoff score for the Emergency Response portion of testing using this feature. However, setting the cutoff scores for other portions of the CritiCall testing will make it much easier for you to more quickly determine who passes or fails the test. Next, print out a “pass/fail” Candidate Score Report for the test taker to determine whether that test taker passed or failed the non-Emergency Response portions of the AutoTest Code See the Set Cutoff Scores training presentation in the Training section of the CritiCall User’s Group web pages for details about setting cutoff scores and printing pass/fail reports.

Question: Why should I print out and check the “Final Status” section of the Pass/Fail version of the Candidate Score Report before I manually check the Emergency Response scores for these two modules? Answer: If the “Final Status” section of the Pass/Fail version of the Candidate Score Report indicates that the job candidate has already “failed” the CritiCall test, there is no need for you to manually check the Emergency Response scores for these two modules since passing or failing that portion of the test will not undo the previously-determined failure.

However, if the Final Status section of the Pass/Fail version of the Candidate Score Report indicates the test taker has “passed” the CritiCall test thus far, print a version of the test taker’s Candidate Score Report without the Pass/Fail information on it Here’s how…

Click here To print a Candidate Score Report for test takers without the Pass or Fail status automatically indicated, click on the “Candidate Score Report” link on the CritiCall Administrator’s Screen

  First, select the candidate(s) whose Candidate Score Report you wish to print by clicking on the check box by their names

  Do not check here Click here Then, to create a Candidate Score Report that indicates test scores, but does NOT automatically indicate a test taker’s pass/fail status, do NOT check () the “Pass/Fail Report” option on the Candidate Score Report dialogue box before clicking on the “Print Report…” button

(This is the report version you need for the next steps!) A printed Candidate Score Report with the test taker’s scores, but without the Pass/Fail indictors, looks like this (This is the report version you need for the next steps!) 50 % 80 %

To use the 60% minimum for the Emergency Response scores for the Data Entry MT and/or Data Entry MT (Audio) test modules, first find either (or both) of those test modules listed on the Candidate Score Report 50 % 80 %

50% 80% Next, go down the Emergency Response Score column to view the Emergency Response scores from the Data Entry MT and/or Data Entry MT (Audio) test modules

IMPORTANT Use Emergency Response scores from only these two test modules for hiring purposes 50% 80% Emergency Response scores from other test modules do not have sufficient test reliability to be used for selection purposes

Do not to use Emergency Response scores from any of the other test modules for hiring purposes 50 % 80 % Emergency Response scores from other test modules do not have sufficient test reliability to be used for selection purposes

The next slide demonstrates how to make this comparison Manually compare the score the test taker received on the Emergency Response portion of these test modules against a 60% minimum If the test taker scores 60% or higher on the Emergency Response portion of these test modules, they pass the Emergency Response section of the test f the test taker scores less than 60% on the Emergency Response portion of either of these test modules, they fail the Emergency Response section of the test The next slide demonstrates how to make this comparison

Compare the test taker’s Emergency Response scores for these test modules to a 60% minimum score The test taker “fails” if he or she scores lower than 60% on the Emergency Response portion of either the Data Entry MT and/or Data Entry MT (Audio) test modules 50% - Failed this section 50% 80% 80% - Passed this section In this example, since the test taker scored less than 60% on the Emergency Response portion of the Data Entry MT test module, she “failed” that portion of the CritiCall testing process

Options for using the 60% minimum score FAIL SLOW PASS Ignore the Emergency Response scores for these two modules entirely In other words… don’t pay any attention to the scores on this portion of the test Use the 60% minimum score only for diagnostic purposes Compare the scores for information purposes, but do not pass or fail a test taker based on their scores on the Emergency Response portion Fail everyone who scores lower than the 60% Emergency Response minimum score on both the Data Entry MT and the Data Entry MT (Audio) test modules. Fail everyone who scores lower than the 60% Emergency Response minimum score on the Data Entry MT, or the Data Entry MT (Audio) test modules, or both.

The key to minimizing liability is to treat all test takers who score below 60% on this portion of the test in the same way

Reminder… Do not use Emergency Response scores from test modules other than Data Entry MT and Data Entry MT (Audio)

Do not use the Emergency Response scores from these two test modules if testing was conducted with a CritiCall version prior to 4.0.3

The following are several questions we are frequently asked about Emergency Response scores

How does the Emergency Response portion help employers determine if a job candidate has the ability to multi-task? While responding to Emergency Response items is relatively easy when not performing other tasks, it becomes much more difficult when it must be performed at the same time when entering data. This is the very nature of multi-tasking. Scientific research advises us that having job candidates perform two job-related tasks at the same time appears to be the best way to determine a person’s ability to multi-task. CritiCall is unique in that it measures the ability to perform more than one computerized job-related task at the same time. This is a vital element for testing the ability to multi-task since most of a dispatcher’s work is performed using a computer.

What about the Emergency Response scores from other test modules? The Emergency Response score from the Call Summarization 2 MT test module is combined with that module’s data entry and multiple-choice scores to create a composite primary score The Emergency Response score from that module is used to help create a combined primary score for that module This is why the Emergency Response column for the Call Summarization 2 MT module is blank on the non Pass/Fail version of the Candidate Score Report The Emergency Response scores from the remaining current CritiCall test modules currently do not contain sufficient testing reliability to be used for selection purposes Most other test modules only have two Emergency Response items associated with them, which means a person could only score 0%, 50%, or 100%. This makes it virtually impossible to reliably differentiate between acceptable and not-acceptable performance. Emergency Response items during other CritiCall test modules are designed to act as “distractions” during testing to more accurately simulate the multi-tasking complexity of the job We are working at adding more multi-tasking throughout the test as appropriate

Why is it acceptable to use a nationally developed minimum score for the Emergency Response portion of these two test modules, but not for other areas of testing? The 60% minimum for the Emergency Response portion of the test merely sets a minimum hurdle that a test taker must achieve while performing another task on which they are graded This is similar having a physical ability test that requires a job candidate to run a specified distance within a certain amount of time. The distance is determined in advance (just as the 60% minimum has been determined in advance in this case). Once the distance is determined the employer’s Subject Matter Experts decide on the time limit in which the specified distance is to be run (similar to the SMEs setting a Keystrokes-per-Hour cutoff score for these two test modules after being informed the test taker must also correctly respond to 6 out of 10 Emergency Response items). Thus, the 60% Emergency Response minimum in this instance is fundamentally different from the cutoff scores used throughout the CritiCall test

Finally… Due to challenges outside of our control to the CritiCall program that can occur when it is installed on a wide variety of computer systems, it is always a good idea to manually double-check the pass/fail status of job seekers who have taken the CritiCall test against the cutoff scores you have chosen to use for all of the CritiCall test modules.

We hope you have found this presentation to be helpful Pre-Employment Testing Software Your suggestions and comments welcome at info@criticall911.com Copyright 2007 © Biddle Consulting Group. Inc.