1. ROI: Demonstrating Your Value Warren Bobrow, Ph.D. All About Performance, LLC warren@allaboutperformance.biz

2. What We Know About Selection Identifies people with the KSAPs to do a job well. Identifies people with the KSAPs to do a job well. Presumption that the job and performance metrics are relatively stable. Presumption that the job and performance metrics are relatively stable. Depending on the cut-score, we presume that new hires are more effective than incumbents and rejected applicants. Depending on the cut-score, we presume that new hires are more effective than incumbents and rejected applicants.

3. What We Don’t Often Know About Selection Is there demonstrable value when objective/dollar-based performance metrics are not available? Is there demonstrable value when objective/dollar-based performance metrics are not available? Are there situations where hiring the most talented people works to our detriment? Are there situations where hiring the most talented people works to our detriment? –Seasonal/Temp employees –Quick advancement

4. Selecting Seasonal Employees Telephone contact center Telephone contact center Actually two different seasons, but big ramp ups (700%) in each Actually two different seasons, but big ramp ups (700%) in each Season lasts about 6 months, including training time Season lasts about 6 months, including training time

5. Questions In This Study Are selection systems validated on full-time employees valid for seasonal ones? Are selection systems validated on full-time employees valid for seasonal ones? Do organizations shoot themselves in the feet by hiring highly talented temps? Do organizations shoot themselves in the feet by hiring highly talented temps? What, if any, is the non-performance based utility of using validated selection tools for hiring temps? What, if any, is the non-performance based utility of using validated selection tools for hiring temps?

6. The Validation Studies Jobs Jobs –Customer & Tech Support –Inbound Sales (registrations, upgrades, etc.) Conducted a Job Analysis Conducted a Job Analysis Concurrent Validation on Full-Time Incumbents Concurrent Validation on Full-Time Incumbents Predictive Validation on Seasonal Employees Predictive Validation on Seasonal Employees –Cut-score not really used, so there was plenty of variance

7. Validated Predictors Personality Personality –Agreeableness –Conscientiousness –Customer Service Orientation Aptitude Aptitude –Numeric and Verbal Reasoning Biodata Biodata

8. The Validation Studies--Concurrent Gathered 6 weeks of weekly performance data. Gathered 6 weeks of weekly performance data. –Extremely reliable Developed 4 scoring bands Developed 4 scoring bands –Highly Recommended –Recommended –Possibly Recommended –Not Recommended

9. The Validation Studies--Predictive Gathered weekly performance data for the entire season. Gathered weekly performance data for the entire season. –Extremely reliable –Tracked turnover (voluntary and involuntary)

10. Validation Results Concurrent validity =.52 (p<.001) Concurrent validity =.52 (p<.001) Predictive validity =.31 (p<.001) Predictive validity =.31 (p<.001) Methodological note: Methodological note: –Correlations not corrected for test or criterion unreliability There was some shrinkage (primarily due to the biodata), but tests validated on full-timers were valid for seasonal employees There was some shrinkage (primarily due to the biodata), but tests validated on full-timers were valid for seasonal employees

11. Other Issues--Turnover Many organizations need seasonal/part-time workers. Many organizations need seasonal/part-time workers. Very few workers want non-full time work. Very few workers want non-full time work. –Always looking to go full-time some place else –Not much organizational commitment, so will look for similar work for higher pay. –In a normal economy, as a group, may not be as talented as those seeking full-time employment Are talented seasonal employees more likely to voluntarily turnover than less talented ones? Are talented seasonal employees more likely to voluntarily turnover than less talented ones?

12. Results--Turnover Not Recommended Possibly RecommendedRecommended Highly RecommendedTotal Stayed15 (11.0%) 21 (17.3%) 29 (30.8%) 26 (29.6%) 91 (20.8%) Left121 (89%) 100 (82.7%) 65 (69.1%) 62 (70.4%) 229 (79.2%) Total1361219488439

13. Results--Turnover This data indicates that those who do better on the test are more likely to stay (not get fired or quit) through the season. This data indicates that those who do better on the test are more likely to stay (not get fired or quit) through the season. Additionally, those who scored Not Recommended or Possibly Recommended were more likely to be fired than those who scored Recommended or Highly Recommended (chi- squared=17.3, df=3, p <.001). Additionally, those who scored Not Recommended or Possibly Recommended were more likely to be fired than those who scored Recommended or Highly Recommended (chi- squared=17.3, df=3, p <.001). No differences in voluntary turnover. No differences in voluntary turnover.

14. ROI—Cost of Turnover Recruitment Recruitment Selection Selection Training Training

15. ROI Data Assumptions Turnover Assumptions Turnover Assumptions –70% turnover for the high test scoring group –85% turnover for the low test scoring group –79% turnover if random selection

16. ROI—What Are the Costs of Recruitment? Staff time Staff time Advertising Advertising Job fairs Job fairs Background checks Background checks Drug screens Drug screens Total of $862/position (client estimate) Total of $862/position (client estimate)

17. ROI—Cost of Recruitment 9 more people/100 hired to be recruited 9 more people/100 hired to be recruited –Conservative estimate in that replacements will turnover at the same rate and more will have to be recruited 9 * 862 = $7,758 9 * 862 = $7,758 –$862 = Recruiting Costs/Hires

18. ROI—What are the Costs of Training? Instructors Instructors Materials Materials Facilities/meals, if any Facilities/meals, if any Lower initial productivity Lower initial productivity

19. ROI—Cost of Training In this case, training costs $18/hr per trainee for 80 hours In this case, training costs $18/hr per trainee for 80 hours –Cost per trainee is $1,440 –To train 9 more is $12,960

20. ROI—What Are the Costs of Testing? Staff Time Staff Time Room Rental Room Rental Materials Materials Cost Per Test Cost Per Test

21. ROI—Cost of Testing Testing Testing –$20/person –41% passing rate –Total additional cost of $252

22. ROI—Total Cost of Turnover Recruitment: $7,758 Recruitment: $7,758 Testing: $252 Testing: $252 Training: $12,960 Training: $12,960 Total Per 100 Hires: $20,970 Total Per 100 Hires: $20,970 Cost of testing to hire those 100: $3,618 Cost of testing to hire those 100: $3,618 Net cost savings: $17,352 Net cost savings: $17,352

23. Caveats Every situation will differ in costs Every situation will differ in costs –Testing cost in this example is very low. Some of these costs are hard (materials, instructors) and others are soft (staff time) Some of these costs are hard (materials, instructors) and others are soft (staff time)

24. Conclusions The value of testing can go beyond performance The value of testing can go beyond performance –Think scorecard or other key metrics Sometimes, these values are easier to quantify than performance. Sometimes, these values are easier to quantify than performance. As a profession we need to look for these ancillary benefits when talking about our value. As a profession we need to look for these ancillary benefits when talking about our value.

25. Questions and Comments