1. Online Innovations to the Assessment Interview Impacts on Accuracy, Scoring, and Participant Reactions Tom Janz, Ph D Chief Science Officer Behavior Description Technologies

2. .00 -.05 Graphology, Numerology, Astrology.05 -.15 Resume Ratings.15 -.20 Unstructured 1-on-1 interviews.20 -.24 Traditional reference checks.30 -.35 Panel Interviews.35 -.40 Validated Personality Tests.50 -.55 Cognitive Ability Tests.50 -.55 Structured Behavior-based Interviews.60 -.65 Ability Test plus Structured Interview.65 -.70 Ability Test plus Work Simulation Why the Assessment Interview went Behavioral

3. Why the Behavioral Interview is going Online Increase in Candidate Comfort: Increase in Candidate Comfort: Online Interviews rated as less stressful Online Interviews rated as less stressful Online Interviews leave a better first impression Online Interviews leave a better first impression Interviewers report feeling better prepare Interviewers report feeling better prepare Increase in Decision Power: Increase in Decision Power: Study finds Online Interview.69 vs.48 Campion & Mueller Study finds Online Interview.69 vs.48 Campion & Mueller Reduction in Staffing Spend: Reduction in Staffing Spend: Fewer finalists needed for on-site interviews Fewer finalists needed for on-site interviews Interviews started online take 30-50% less time Interviews started online take 30-50% less time

4. The Road to Online Interviewing No one thought candidates would type out answers to behavioral questions– Not even Lowell Hellervik, Bill Byham, or Paul Green. No one thought candidates would type out answers to behavioral questions– Not even Lowell Hellervik, Bill Byham, or Paul Green. Following the lead of computerized testing, early versions put the questions on screen. Following the lead of computerized testing, early versions put the questions on screen. Candidates typed whole answers into a text box, guided by answer hints or probes. Candidates typed whole answers into a text box, guided by answer hints or probes.

5. Innovations in the Assessment Interview supported Online: Answer Coaching Answer Coaching Performance Auto-Confirm Performance Auto-Confirm Best-Answer Anchoring Best-Answer Anchoring Behavioral Judgment Testing Behavioral Judgment Testing

8. Innovation 1: Answer Coaching

9. Candidate Reactions to the Online Interview: Here are the questions we asked--

10. We graph the answer ratings from over 10,000 Restaurant Manager, 900 Pharmaceutical Sales, and 70 Mine Manager candidates who completed a profile.

12. Innovation 2: Performance Auto-Confirm. Merely collecting a contact name and a means to connect compels more accurate self evaluations (bogus pipeline research). Merely collecting a contact name and a means to connect compels more accurate self evaluations (bogus pipeline research). Users who provide email addresses for their contacts enable a labor-less confirmation. Users who provide email addresses for their contacts enable a labor-less confirmation. Users who give permission to confirm their performance at any time make auto- confirmation possible. Users who give permission to confirm their performance at any time make auto- confirmation possible. Here is how it works. Here is how it works.

15. Innovation 3: Best Answer Anchoring. Most quality approaches to rating behavioral interview answers feature rating anchors. Most quality approaches to rating behavioral interview answers feature rating anchors. Research finds behavioral anchors add about 10 validity points over Lickert answers. Research finds behavioral anchors add about 10 validity points over Lickert answers. However, behavioral anchors often take on the flavor of apple pie and motherhood. However, behavioral anchors often take on the flavor of apple pie and motherhood. Performance component levels that are differentiated primarily by frequency adjectives (still better than no guideline). Performance component levels that are differentiated primarily by frequency adjectives (still better than no guideline). How about providing BOTH performance components and REAL “Best Answers”? How about providing BOTH performance components and REAL “Best Answers”?

18. Innovation 4: Behavioral Judgment Testing Situational Judgment Tests probe how people WOULD ACT in challenging, job-related, situations. Situational Judgment Tests probe how people WOULD ACT in challenging, job-related, situations. How people WOULD ACT cannot be confirmed. How people WOULD ACT cannot be confirmed. How people WOULD ACT overlaps with their mental ability for picking the most effective answers. How people WOULD ACT overlaps with their mental ability for picking the most effective answers. Behavioral Judgment Tests focus on what candidates DID DO, in challenging, job-related, situations, seeking confirmation from third parties. Behavioral Judgment Tests focus on what candidates DID DO, in challenging, job-related, situations, seeking confirmation from third parties.

19. Introductory Screen and Sample Item. Candidates begin by rating HOW OFTEN they faced this scenario.

20. Since this scenario was faced more than 2 times, the next rating anchors WHEN the Best Example took place. Then we show the first Action Option, and collect its similarity to what the person DID.

21. Now we independently collect the candidate’s similarity rating to the second action option. (Note: The candidate does not know how many action options there are for each scenario.)

22. The Ratings Review Page displays ALL the action options (up to 5) and asks candidates whether they would like to adjust their ratings.

23. Finally, candidate are asked to provide the name, position, and contact info for someone who can confirm their actions in the situation they selected for this scenario.