1. Training and Experience Dr. Mike Aamodt Radford University maamodt@radford.edu Updated 12/06/2004

2. Personnel Selection Methods Training & Education Experience –Applications/Resumes –Biodata –Interviews Knowledge Ability –Cognitive –Physical –Perceptual Skills –Work Samples –Assessment Centers –References Personality & Character –Personality Tests –Integrity Tests Medical –Medical Exams –Psychological Exams –Drug Testing

3. Ratings of Training Education Work-Related Training Military

4. Education as a Predictor of Job Performance Education can be broken further into 4 categories: – Degree – Years of education – Major – GPA

5. Does Education Predict Performance?

6. Summary of Meta-Analyses Meta-analysisOccupationKNρ Aamodt (2004)Police549,120.28 Vineberg & Joyner (1982)Military35.25 Hunter (1980) Hunter & Hunter (1984) Schmidt & Hunter (1998) USES data base 42532,124.10 Dunnette (1972)Entry level petroleum 15.00

7. Validity of Education for Police Our Updated Meta-Analysis Academy Supervisor Discipline Grades Ratings Problems _________________________ Number of studies 32 54 54 Sample Size 6,153 9,120 21,416 Uncorrected validity.26.17 -.08 Corrected validity.38.28 -.12 Sampling error % 100% 80% 73%

8. Our Meta-Analysis (Cont.) Academy GradesPatrol Performance ____________________________ Education level N Score N Score ___________________ __________ _____ GED/HS Diploma 376 -.23 c 915 -.12 a 12-64 college hours 100 -.05 ac 391.02 b AAS 98.16 ab 139.07 bc 64 college hours 62.01 ac 45.29 bc Bachelor’s 255.32 b 441.18 c

9. Interaction of Education and Experience Ratings of Patrol Performance Experience Educational Level< 2 years2+ YearsTotal High School Diploma M -.09 ab -.16 a -.13 a SD(.92)(.98) Some college or associate’s degree M -.26 a.07 b -.09 a SD(1.02)(.94) B.A. or higher M.04 ab.34 c.19 b SD(1.02)(1.12)

10. Education and Academy Grades Effect of Academy Length KNrρSE % Overall326,153.26.38100 Academy Length 12-17 weeks162,477.31.46100 > 20 weeks153,473.25.36100

11. Does Education Add Incremental Validity Over Cognitive Ability?

12. Education and Incremental Validity Schmidt & Hunter (1998) say no –Cognitive ability (r =.51) –Cognitive ability and education (r =.52)

13. Education and Incremental Validity Our meta-analysis for police says yes –Academy Grades Cognitive ability (r =.33) Cognitive ability and education (r =.34) –Patrol Performance (uncorrected) Cognitive ability (r =.16) Cognitive ability and education (r =.29)

14. Can You Substitute Education for a Cognitive Ability Test? Education has less adverse impact But –Cognitive ability and education are only slightly correlated.29 in our police data.00 assumption by Schmidt & Hunter (1998) of applicants applying for a particular job.55 estimate by Schmidt & Hunter (1998) for the general population –Education adds incremental validity for law enforcement –Our dataset contains plenty of college graduates with low cognitive ability

15. Does Grade Point Average Predict Performance?

16. Validity of GPA GPA is a valid predictor of performance on the job, training performance, starting salary, promotions, and grad school performance GPA is most predictive in the first few years after graduation (Roth et al., 1996) GPA will result in high levels (d=.78) of adverse impact (Roth & Bobko, 2000) People with high GPAs –Are intelligent (r=.50; Jensen, 1980) –Are conscientious (r=.34; Bevier et al., 1998)

17. Validity of GPA Meta-Analysis Results rρ Work-Related Criteria Job performance (Roth et al., 1996).16.36 Training performance (Dye & Reck, 1989).29 Promotions (Cohen, 1984).16 Salary (Roth & Clarke, 1996) Starting salary.13.20 Current salary.18.28 Graduate School Performance (Kuncel et al., 2001) Grades.28.30 Faculty ratings.25.35

18. Predicting Graduate School Performance Graduate GPATeaching Ratings GRE Score.33*.00 GPA – Overall.31*.02 GPA – Psych.32*.03 GPA – Junior/Senior.40*.09 Faculty References Mental agility.25*-.08 Vigor -.26*-.07 Extroversion.03.24* Agreeableness -.11.06 Conscientiousness.16-.04

19. Does College Major Predict Performance?

20. College Major as a Predictor of Job Performance Aamodt (2004) meta-analysis Validity of having a criminal justice major ___________________________________ r k n ____ ___ _____ Patrol performance -.02 9 1,706 Academy grades.01 6 976 Discipline problems.00 10 2,621 ___________________________________

21. Will Education Requirements Result in Adverse Impact?

22. Adverse Impact TechniqueWhite- Black White- Hispanic Meta-analysis Cognitive ability1.10.72Roth et al. (2001) GPA.78Roth & Bobko (2000) Biodata.33Bobko et al. (1999) Reference Ratings.25Aamodt (2002) Structured interview.23Huffcutt & Roth (1998) Recommendations.22Aamodt (2002) HS diploma.20.52Aamodt (2002) Bachelor’s degree.18.26Aamodt (2002) Personality.09Schmitt et al. (1996) Integrity tests.07-.05Ones & Viswesvaran (1998)

23. Lingering Questions Is the validity of education job specific? What is the actual incremental validity of education over cognitive ability? Why would education predict performance? –Knowledge –Liberal arts skills –Mental ability –Motivation

24. Work-Related Training as a Predictor of Job Performance Aamodt (2000) Meta-Analysis Performance in the academy predicts probationary performance (ρ=.44) Performance in the academy predicts patrol performance (ρ =.38) Performance in the probationary period predicts patrol performance (ρ =.44)

25. Military Training as a Predictor of Job Performance It is common for Police departments to give preferential treatment to candidates with Military experience for two reasons: –Reward veterans for serving their country –They believe veterans will be better cops because the training that they’ve gone through and that of police are similar

26. Validity of Military Experience and Police Performance Academy Grades Patrol Performance Commendations K9168 N2,0614,0901,337 r.02-.03-.07 ρ.04-.05-.10 SE %90%23%81%

27. Military Experience in Candidate Selection In selecting applicants, the employer must be careful because: –Minorities tend to have a higher rate of undesirable discharges; therefore any questions about military service dates, type of discharge, and discipline received while in the service can result in adverse impact. - Gatewood & Feild (2001)

28. Experience

29. Past behavior predicts future behavior –Experience is a valid predictor of future performance (r =.27; Quinones et al., 1995) Types of Experience –Work –Life

30. Experience Evaluated through: –Application blanks –Resumes –Interviews –Reference checks –Biodata instruments

31. Experience Considerations –How much experience? –How well did the person perform? –How related is it to the current job?

32. Experience Predicts Best… Credit prior work experience only: –In the same occupational area as that in which performance is to be predicted –In the performance of tasks or functions that have direct application on the job Recency of experience should be used as a decision rule for awarding credit only when justified on a case-by-case basis Credit for duration of work experience should be limited to a few years. High prediction up to about 3 years of experience, declining to low prediction for more than 12 years of experience.

33. Experience for Selection : Some Concerns Sullivan (2000) claims that “experience in solving ‘past problems’ is rapidly losing its applicability to current and future problems.” Organizations will increase their applicant pool if they delete the “ancient history” requirements (i.e. “Ten years experience required”).

34. Sullivan (2000) 1)Reduce or eliminate the number of years required in your ads and replace them with “the demonstrated ability to solve problems with our required level of difficulty. 2)Use simulations and actual problems to assess applicants. 3)Develop “future-oriented” questions for applicants. 4)Train evaluators and compensation professionals to put less weight on experience of candidates. 5)Revise job descriptions to include level of difficulty. 6)Identify the amount and type of experience and competencies that would predict job performance. 7)Check to see if there is a correlation between the number of years of experience an employee has and their success in your firm.

35. Experience: Some More Concerns Performance matters “Haven’t done” doesn’t mean “can’t do” Experience has a shelf life Listing something on a resume is not experience Where you get your experience matters Experience does not guarantee success Experience is expensive More experience might be bad (old ways and ideas)