ROI in HRM: Validity of Selection Procedures Nicole Blake Edgar Nolasco Deena Desai

Employee Selection: Will Intelligence and Conscientiousness Do the Job? Orlando Behling

Key Terms General Intelligence “g” Big Five Meta-Analysis – combining research results from existing studies that meet a specific criteria for to generate a general conclusion.

Summary Measuring general intelligence “g” in addition to the Big Five provides a stronger predictor of job performance in past jobs. Group Hypothesis: What have we learned since Behling?

Article A Meta-Analytic Investigation of Conscientiousness in the Prediction of Job Performance: Examining the Intercorrelations and the Incremental Validity of Narrow Traits By: Nicole M. Dudley, Karin A. Orvis, Justin E. Lebiecki and Jose M. Cotina George Mason University Nicole Blake

Key Terms Global Conscientiousness: A broad characteristic of being conscientious (careful, thorough). Four Narrow traits of conscientiousness: ▫Achievement ▫Order ▫Dependability ▫Cautiousness Nicole Blake

Introduction: The Big Five Conscientiousness: the most important & the most valid personality predictor of job performance. Multiple meta-analyses from previous research examined and used in the current meta analyses. Goal of this study Nicole Blake

Hypothesis 1) The narrow traits of conscientiousness are a better predictor than global conscientiousness alone. 2) Narrow traits can be a useful predictor for specific occupations but not all occupations. Nicole Blake

Method Validity coefficients and intercorrelations among the 5 predictors were obtained from previous studies. Criteria for inclusion 2 moderators: type of performance criterion & occupational type. Hierarchical regression analysis Nicole Blake

Results 1: Incremental Validity for Job Performance Criteria Nicole Blake

Results 2: Overall Job Performance by Occupational Type Nicole Blake

Resume Screening & Personality Testing Recruiters Inferences of Applicant Personality Based on Resume Screening: Do Paper People have a Personality? By: Michael S. Cole, Hubert S. Feild, William F. Giles, Stanley G. Harris. Excludes potential candidates. Considerable error present in recruiters inferences. Nicole Blake

Take Home Message Focusing on narrow traits is a better predictor of job performance than focusing on broad traits. Managers should consider narrow traits when the goal is to predict contextual performance or to predict overall performance for specific occupations ▫Saves money in the long run by hiring the best fit applicants. Nicole Blake

Article Situational Judgment Tests, Response Instructions, and Validity: A Meta-Analysis By: Michael A. McDaniel, Nathan S. Hartman, Deborah L. Whetzel and W. Lee Grubb III Edgar Nolasco

Key Terms SJT- Situational Judgment Test ▫Behavioral tendency ▫Knowledge Cognitive ability or g – It is a set of abilities, skills or processes that are part of nearly every human action. Big Five– Extraversion, Emotional Stability, Agreeableness, Conscientiousness and Openness. Edgar Nolasco

Introduction/Study Two things being tested through this study Using a Meta-Analysis ▫The psychometric artifact method was used in the study Data set from the McDaniel (2001) study Previous research shows that SJTs predict performance because they measure job knowledge Edgar Nolasco

Example of an SJT You have just attended a training course delivered by your manager. While talking to the others privately after the course, several of them were critical of how your manager delivered the course. Your manager is unaware of this feedback and you felt the course delivery was adequate. Rate the effectiveness of each of the actions below on the following scale: 1 = Very Ineffective, 2 = Ineffective, 3 = Effective, 4 = Very Effective. a. Tell your manager that you personally think the course went well and don’t mention the feedback from the other delegates. b. Tell the other delegates that they should communicate their feedback to your manager if they feel strongly about it. c. Tell your manager about the feedback from the other delegates, without mentioning them by name. d. Don’t mention any feedback about the course to your manager. Edgar Nolasco

Method Study uses: ▫Data from McDaniel (2001) ▫The psychometric artifact distribution meta-analysis method (Hunter and Schmidt, 1990, 2004) Criteria: ▫Employees or applicants only ▫Paper-Pencil format ▫Priority to supervisor ratings over job performance ▫Hierarchical level, # of employees supervised, years of management experience, and salary omitted Edgar Nolasco

Results : Criterion-Related and Incremental validity Edgar Nolasco

Take Home Message When looking for the candidate both behavior and knowledge based SJTs should be asked. ▫SJTs that use behavioral instruction are easier to manipulate because respondents know what to say. SJTs are unique: either serve assessment of typical or maximal performance SJTs offer job relevance, ease of use, flexible, are validated and are fair Edgar Nolasco

Further Evidence for the Validity of Assessment Center Dimensions: A Meta-Analysis of the incremental Criterion- Related Validity of Dimension Ratings John P. Meriac, Brian J. Hoffman David J. Woehr and Matthew S. Fleisher Deena Desai

Key Terms Assessment Centers (AC) – objective method of measurement of skills, competencies and characteristics for a specific job Dimensions – Characteristics beyond the Big Five and cognitive ability Hierarchical Regression – adds factors at each stage Forward selection – testing individual variables Deena Desai

Introduction Job Performance PersonalityAC’s Cognitive ability Assessment Centers (AC’s) Weak construct validity Independent influence on job performance Deena Desai

Hypothesis How much if any, incremental variance in job performance do AC dimension ratings account for over and above that accounted for by cognitive ability and personality? Deena Desai

Method Perform a Meta-Analysis ▫Retrieve studies that meet criteria ▫Extract AC dimensions ▫Input in one of the seven dimension categories identified by Arthur et. al. ▫Forward selection Deena Desai

Sample Breakdown 48 Samples 23 Published 17 Unpublished dissertations/ manuscripts 4 Conference presentations 4 Unpublished tech reports Deena Desai

Sample Breakdown 135 AC Dimension Labels Organizing & Planning Problem Solving Drive Influencing Others Consideration/ Awareness of others Stress Tolerance Communication Deena Desai

Results Deena Desai

Results AC dimensions have an incremental variance in job performance above that of personality and cognitive ability ▫The individual characteristics in addition to personality and cognitive ability have a significant relationship with job performance Cognitive ability + the big five + organization and planning = most significant impact on job performance Deena Desai

Take Home Message AC’s are costly therefore should not be used for all positions AC’s must utilize up to date measures to be effective Further research is required to obtain alternative methods that effectively measure specific characteristics Deena Desai

Group Take Home The research demonstrates that to date: ▫Specific measurements of narrow traits beyond personality and cognitive ability are the strongest predictors of job performance. ▫Assessing general characteristics for a variety of jobs increases the risk of litigation.