Click on the screen or press the right arrow key to view the next slide. Use the left arrow key  to go back. Welcome to the OPAC Using the Validation Wizard Tutorial Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Office-Skills Testing Software

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. This presentation is provided to OPAC System users as part of their OPAC Plus Service Plan Membership Please send comments or suggestions to OPAC® and Office Proficiency Assessment & Certification® are registered trademarks of Biddle Consulting Group, Inc

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. What is the Validation Wizard? The OPAC Validation Wizard automatically collects information from an employer’s experienced employees (referred to as “Subject Matter Experts” or “Job Experts”) to determine if the tests being used measure specific job-related knowledge, skills, and abilities required for performing the target job It also automatically determines appropriate cutoff scores for those tests based on input from the employer’s own “job experts” The Validation Wizard is based upon the content validity standards set forth in the federal Uniform Guidelines on Employee Selection Procedures (Sections 14C and 15C).

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Index Click on the title below to directly go to that section. Click anywhere else on the slide (or the “continue” button in the lower right corner) to view the entire presentation in sequence Before using the Validation Wizard What is Validation? Who answers the Validation Wizard’s questions? Can we practice using the Validation Wizard before actually using it to validate a test?Can we practice using the Validation Wizard before actually using it to validate a test? Using the Validation Wizard How to turn the Validation Wizard “on.” The Validation Wizard Process (contains actual “screen shots” of the Validation Wizard survey questions)The Validation Wizard Process Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Before using the Validation Wizard Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. What is “Validation?” According to the U.S. Department of Labor (2000), Validation refers to what characteristic the test measures and how well the test measures that characteristic

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Validation of the tests you use makes good business sense and is required under federal guidelines in some circumstances

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Three simple steps to using the Validation Wizard 1.Create an AutoTest Code containing the tests you wish to use for employment purposes (e.g. hiring) 2.Turn the Validation Wizard “ON” 3.Have 7 to 10 of your experienced employees take the tests contained in that AutoTest Code and respond to the Validation Wizard’s questions when they appear

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Based on the information provided by your Job Experts … The Validation Wizard will automatically determine if the tests are valid for the target job as it is performed at your organization so the test can be used for employment selection purposes The Validation Wizard will automatically calculate cutoff scores that can be used for employment purposes by your organization

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Note The Validation Wizard is designed to help users who are not testing experts to address minimum standards of job relatedness for tests which are anticipated, or known to produce little, if any, adverse impact on protected groups. However, even for tests without adverse impact, it makes good business sense to establish their job relatedness in order to be fair to candidates and to obtain employees who have adequate levels of the knowledge, skills, and abilities actually needed on the job. This is why we strongly recommend that OPAC users make use of its built-in Validation Wizard feature to determine whether the tests being used are job-related and fair to all test takers, as well as for setting appropriate cutoff scores. Organizations that find adverse impact against protected groups of test takers when using OPAC testing may wish to conduct their own, full- blown employer-specific job analysis and validation study to more fully address the federal Uniform Guidelines. Please contact Biddle Consulting Group, Inc. at to find out more about this option.

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Using the Validation Wizard Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. You determine which tests are included in the AutoTest Code to be validated. In other words, you choose which OPAC tests you will use for making your employment decisions –Click on xxx to learn more about setting the AutoTest Code, or click anywhere else on the slide to continue with the presentation Who determines which tests we will be validating?

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. You create an AutoTest Code in the AutoTest Setup section of the Administrator’s Program See OPAC System Administrator’s manual for details This is a “Screen Shot” of the AutoTest Setup section of the Administrator’s Program

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. You determine whether you allow your Job Experts to choose the AutoTest Code to be validated or whether you only allow them access the one AutoTest Code you wish to validate –Click on xxx to learn more about how to allow or limit your Job Experts’ choice, or click anywhere else on the slide to continue with the presentation Who determines which AutoTest Code my Job Experts will validate if there is more than one AutoTest Code saved?

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. In the System Customization section of the Administrator’s program you check this box to allow your Job Experts to choose which AutoTest Code they are going to validate Or, you might not wish to allow the Job Experts to choose and instead specify which one AutoTest Code (the “default”) will be available to the Job Experts. You do this by not checking the “Allow AutoTest Code Selection by Candidate” box This is a “Screen Shot” of the System Customization section of the Administrator’s Program

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The remainder of this presentation contains additional information that may help you better understand the Validation Wizard and its use

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Turning the Validation Wizard “ON” Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Select System Customization in the Administrator’s Program This is a “Screen Shot” of the Administrator’s Program opening screen

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Check the “Conduct Validation Survey” box. Then select “OK” and the Validation Wizard is turned “on” This is a “Screen Shot” of the System Customization section of the Administrator’s Program

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. To turn the Validation Wizard “off,” return to the System Customization section of the Administrator Program and uncheck the box that you used to activate the Wizard

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Who Answers the Validation Wizard’s Questions? Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The Validation Wizard is completed by your own Job Experts Job Experts (also called “Subject Matter Experts” or “SMEs”) are employees who currently hold the job classification you are testing for, and are considered successful job performers, and/or supervisors who have held the targeted job classification, and/or instructors or trainers for the targeted job classification

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. How many of my Job Experts should take the Validation Wizard? The courts have ruled that 7 to 10 Job Experts are sufficient –Of course, if the target job classification has fewer than 7 to 10 Job Experts available, a smaller number will have to suffice –For jobs with hundreds of current employees, more than 7 to 10 Job Experts may be beneficial

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. How do we choose our Job Experts? We recommend using employees who have at least one year’s experience with the tasks performed on the job, if possible The Job Experts you choose should be considered successful job performers The Job Experts used during the validation process should be representative of the job classification’s employees in terms of gender and race/ethnicity. –In addition, underrepresented racial/ethnic and gender minorities should be included, if possible

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Can we practice using the Validation Wizard before we record our actual responses? Yes, but all entries made using the Validation Wizard are permanently stored and reported by OPAC. They cannot be deleted or modified. –If you wish to practice using the Validation Wizard you should practice on an AutoTest Code that will never be used for employment purposes Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The Validation Wizard Process Return to Index Continue

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. If the Validation Wizard is turned “on” when a Job Expert first enters the OPAC Candidate Program, they are greeted with this message. This prevents job applicants from accidentally taking the test with the Validation Wizard turned “on.” This is a “Screen Shot” of the Candidate Program

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. When the Validation Wizard is turned on, your Job Experts take each test in the AutoTest Code just as a job applicant would At the end of each test the Job Experts are asked to respond to a series of questions to determine if the test is related to the job as it is performed at your organization What happens during the Validation Wizard Process?

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. What questions are asked by the Validation Wizard? The next series of slides are actual “screen shots” of the six Validation Wizard steps that appear after a Job Expert completes an OPAC test

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The Job Title and Job Expert’s name is automatically filled in by the OPAC program Validation Wizard Page 1 Note: The “Job Title" automatically filled in here is the name of the AutoTest Code being validated

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Validation Wizard Page 1 Job Expert uses pull-down menus to indicate race and gender

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Validation Wizard Page 1 Job Expert enters number of years’ experience related to the target job

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Demographic information is collected so that there will be a record of which Job Experts validated the test for your organization –This information is required if the validity of your testing process should ever be challenged Years of Experience –This number should be the total number of years that this Job Expert has held, supervised, trained, and/or instructed the targeted job classification

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Note: During the actual Validation Wizard process, Job Experts cannot go on the to next step until all of the questions in the previous step have been addressed

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The skill being measured by this test is shown here Validation Wizard Page 2 We must type and enter information using the computer most of the day. Job Expert enters requested information into white-colored field Job Expert answers questions about this skill

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Some survey questions require a “yes” or “no” response, while others require the Job Expert to choose one from a multiple-choice list. The expert merely clicks on a radio button to indicate his or her choice. Fill-in-the-blank information is entered by the Job Expert into the blank white field as instructed during the survey using the keyboard  

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Job Expert enters MOST important task here and then answers questions about that task Validation Wizard Page 3 Writing letters to customers using the computer.

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Validation Wizard Page 4 Updating correspondence using the computer. Job Expert enters SECOND MOST important task and then answers questions about that task

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Job Expert enters suggested cutoff score here Validation Wizard Page 5 85 There is more information about the suggested cutoff scores on the next slide Job Expert answers question about the difficulty of the test here

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. The Job Expert enters a cutoff score in the blank space provided that they believe a minimally qualified applicant should score in order to pass and be considered for hiring This methodology for setting cutoff scores has been endorsed by the U.S. Supreme Court in U.S. v. South Carolina (1978) If the Job Expert puts a suggested cutoff score that is higher than the score that the expert achieved on the test, an explanation must be made why the higher cutoff score is being suggested (see next slide)

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved The fire alarm went off while I was taking this test and I missed several items because of that. Job Expert only needs to fills in this space if the cutoff they suggest is higher than the score they achieved on the test. This field appears ONLY if the suggested cutoff score is higher than the score the Job Expert achieved on the this test

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Information about how cutoff scores suggested by different Job Experts are combined together to create a final cutoff score is explained in the Validation Report presentation

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. Validation Wizard Page 6 Based on the Job Expert’s responses to this question, OPAC automatically determines whether applicants can be justifiably ranked according to their score on the test, instead of just “passed” or “failed” (More information about this can be found in the Validation Report presentation) This is the last of the six Validation Wizard Survey Steps

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. After the Job Experts have completed taking all of the tests in the AutoTest Code and responded to the questions asked by the Validation Wizard, you can automatically generate a Validation Report based on their responses Please see the Validation Report training presentation for more information

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved. We hope this presentation has been helpful! Please contact OPAC at with your comments and/or suggestions about this presentation, or us at Return to Index

Copyright © 2007, Biddle Consulting Group, Inc. All rights reserved Office-Skills Testing Software