1. Human Resource Selection, 8e
Gatewood
Human Resource Selection, 8e
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

2. Validity of Selection Procedures
Chapter 8
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

3. An Overview of Validity
In this chapter, we focus on validity, its relation to reliability, and the principal analytic strategies available for determining the validity of selection procedure data
Like reliability, the importance of validity applies to selection procedures and criteria
Validity shows what is assessed and determines the kinds of conclusions we can draw from data
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

4. An Overview of Validity Validity: A Definition
Validity concerns the accuracy of judgments or inferences made from scores on selection measures, including predictors and criteria
Most think of only selection procedure validity
But validity applies to criteria as well
We want to know how accurate our hypothesized predictions are re job success
What is the validity of inferences that can be made from the scores on selection measures?
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

5. An Overview of Validity The Relation between Reliability & Validity
It is possible to have a measure that is reliable, but does not assess what we want for selection
The quantitative relationship between validity and reliability is:
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

6. An Overview of Validity Types of Validation Strategies
A validation study helps to determine the legitimate inferences that can be made from scores on a selection measure
Study to determine the accuracy of judgments made from scores on a predictor about a job behavior; 3 classical approaches:
(a) content validation
(b) criterion validation
(c) construct validation
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

7. Content Validation Strategy
A selection procedure or criterion has content validity when its content representatively samples the content of the job associated with successful performance
“Content of the job” is a collection of job behaviors and associated knowledge, skills, abilities necessary for effective work performance
Relies on expert judgment, not statistics
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

8. Content Validation Strategy (2)
Face validity may be confused with content validity
Content validity deals with representative sampling of the content domain of a job
Face validity concerns the appearance of whether a measure is measuring what is intended
Perceived face validity of selection procedures strongest correlation to perception of fairness
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

9. Content Validation Strategy Major Aspects of Content Validation
1. Conducting a Comprehensive Job Analysis
A. Describing the tasks performed on the job
B. Measuring importance of the tasks
C. Specifying WRCs required to perform
D. Measuring the importance of WRCs, including:
(1) an operational definition of each WRC
(2) a description of the relationship between each WRC and each job task
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

10. Content Validation Strategy Major Aspects of Content Validation
D. Measuring the importance of WRCs, including: (2)
(3) a description of the complexity of obtaining WRC
(4) a specification re whether an employee is expected to possess each WRC before training or placement on the job
E. Linking important job tasks to important WRCs
2. Selecting Experts Participating in a Content Validation Study
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

11. Content Validation Strategy Major Aspects of Content Validation
3. Specifying Selection Procedure Content
A. Selection Procedure as a Whole
B. Item-by-Item Analysis
C. Supplementary Indications of Content Validity
Predictor Fidelity
4. Assessing Selection Procedure and Job Content Relevance
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

12. Content Validation Strategy Some Examples of Content Validation
A reading skills test based on actual safety signs and operating procedures employees needed to be able to read
A job-related employment interview with interview questions designed to assess required work performance dimensions
For content validity, derive test content from what people do on the job
See Figure 8.1
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

13. Figure 8.1 – Example Tasks, WRCs, Selection Procedures…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

14. Content Validation Strategy Appropriateness of Content Validation?
Uniform Guidelines (1978)
A content strategy is not appropriate [for] demonstrating the validity of selection procedures which purport to measure traits or constructs such as intelligence, aptitude, personality, commonsense, judgment, leadership and spatial ability
Now, “content validity is appropriate scientifically and professionally for use with tests of specific cognitive skills used in work performance”
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

15. Content Validation Strategy Appropriateness of Content Validation? (2)
Job Analysis and Content Validation
Figure 8.2 summarizes major inference points when using job analysis to establish content validity of a selection procedure
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

16. Figure 8.2 – Inference Points from Analysis of Job Content to Selection…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

17. Content Validation Strategy Appropriateness of Content Validation? (3)
To make inference supporting content validity, three important issues:
1. Does successful performance on selection procedure require the same WRCs needed for successful work performance?
2. Is the mode used to assessing performance on the WRCs the same as required for the job?
3. Are WRCs not required for the job present in the predictor?
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

18. Content Validation Strategy Appropriateness of Content Validation? (4)
Uniform Guidelines specify when content validation alone is not appropriate:
1. When mental processes, psychological constructs or personality traits are not directly observable but inferred from selection method
2. When the selection procedure involves WRCs an employee is to learn on the job
3. When content of the selection device does not resemble a work behavior, or the work setting
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

19. Content Validation Strategy Appropriateness of Content Validation? (5)
How Content Validation Differs from Criterion- Related Validation
(a) in content validation, focus on selection procedure job content domain, but with others, focus is on relation with an external criterion
(b) criterion-related validity narrowly based on specific set of data, but content validity based on broader base of data and inference
(c) criterion-related validity quantitative; content validity uses judgmental descriptors
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

20. Criterion-Related Validation Strategies
Concurrent Validation
Information based on predictor and a criterion for a current group of employees
Validity of the inference signified by a statistically significant (p< .05) relationship
An Example
Figure 8.3 demonstrates
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

21. Figure 8.3 – Selection of Experimental Selection Procedures to Predict...WRCs
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

22. Criterion-Related Validation Strategies
Concurrent Validation (2)
Strengths & Weaknesses
Several factors mitigate use of concurrent validation study:
(a) availability of large sample
(b) differences in job tenure or length of employment
(c) representativeness of employees to applicants
(d) certain employees failing to participate
(e) motivation of employees to participate or manipulate answers
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

23. Figure 8.4 – Representation of Relating Predictor Scores with Criterion Data..
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

24. Criterion-Related Validation Strategies (2)
Predictive Validation
Involves collection of data over time
Job applicants, not employees are data source
Basic steps summarized in Table 1
An Example
Job analysis, choice of criteria and selection of predictors same as Concurrent Validation
Significant difference between concurrent and predictor methods is at predictor administration step
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

25. Table 8.1 – Summary of Major Steps…Concurrent & Predictive…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

26. Criterion-Related Validation Strategies (3)
Predictive Validation (2)
Strengths and Weaknesses
Because of the inference tested by predictive validation, this method is appropriate for use in HR selection
Predicts how well job applicants will be able to perform on the job
One big weakness is time interval required to determine the validity of the measure being examined
Difficult to explain to managers the importance of filing selection measure information before using data
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

27. Figure 8.5 – …Examples of Different Predictive Validation Designs
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

28. Criterion-Related Validation… Concurrent vs. Predictive…
Generally assumed that predictive validation superior to concurrent
Studies:
Minimal differences found between 2 types of designs
For ability tests, either approach viable
Different results for personality & integrity measures
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

29. Criterion-Related Validation… Requirements for a Criterion-Related…
1. Job should be reasonably stable, not in period of change
2. A relevant, reliable criterion free from contamination must be available or feasible
3. Must be possible to base validation study on sample of people and jobs that is representative
4. A large enough, representative sample of people must be available
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

30. Criterion-Related Validation… Criterion-Related Validation over Time
One review: Predictive validity of some measures rapidly decayed over time
But critics of review noted that only one study reviewed incorporated an ability test to predict actual performance on the job
Another study found predictive validity of mental ability tests actually increased over time; that if dexterity tests remained about the same
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

31. Criterion-Related Validation… The Courts and Criterion-Related…
Valid study does not mean victory in court:
1. Some courts preferred to judge validity on format or content of selection instrument
2. Some courts swayed by a test’s legal history
3. Some judges had preferences for use of a predictive strategy vs. concurrent
4. A statistically significant validity coefficient alone did not guarantee decision for firm
5. Judges differed on willingness to accept statistical corrections
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

32. Criterion-Related Validation… …Some Requirements
Drawing from Uniform Guidelines, Principles for the Validation and Use of Employee Selection Procedures and other sources, summary of requirements for each method set out in Table 8.2
Not exhaustive
Only major considerations
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

33. Table 8.2 – …Summary of Major Methods for Estimating Reliability…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

34. Construct Validation Strategy
Psychologists use term construct to refer to a theoretical psychological concept, attribute, characteristic or quality
Intelligence, sociability, etc. are called constructs
Specific measures (like clerical ability test) are measures hypothesized to represent a specific construct
Construct validation helps us determine whether a measure does indeed reflect a specific construct
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

35. Construct Validation Strategy (2)
Figure 8.6 shows hypothesized links between constructs and their measures
This example highlights major steps for implementing a construct validation study:
1. The construct is carefully defined and theoretically developed…
2. A measure hypothesized to assess the construct is developed
3. Studies testing the hypothesized relations are conducted
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

36. Figure 8.6 –…Construct Validation of Link Between… Construct & Predictor
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

37. Empirical Considerations…
Even when we have conducted a content validation study on a selection procedure, we still want to ask:
1. Is there a relationship between applicants’ or employees’ responses to the selection procedure and their performance on the job?
2. If so, is the relationship strong enough to warrant the measure’s use in employment decision making?
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

38. Empirical Considerations… Correlation
Computing Validity Coefficients
A validity coefficient is an index that summarizes the degree of a relationship between a predictor and criterion
Table 8.3 represents a hypothetical collection of sales ability score (predictor) and job performance rating (criterion) on 20 salespeople
Figure 8.7 shows a scatterplot of the data
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

39. Table 8.3 – …Sales Ability Inventory Score & Job Performance Data…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

40. Figure 8.7 – …Scattergram of Sales Ability & Job Performance Scores…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

41. Figure 8.8 – Description of Possible Predictor/Criterion Relationships…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

42. Empirical Considerations… Correlation (2)
Importance of Large Sample Sizes
1. A computation on a small sample must be higher in value to be considered statistically significant
2. It is less reliable, and shows greater variability
3. the chances of detecting that a predictor is valid I lower for small sample sizes than large ones
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

43. Empirical Considerations… Correlation (3)
Interpreting Validity Coefficients
By squaring the validity coefficient, we obtain an index that indicates our tests’ ability to account for individual differences
this coefficient of determination represents the percentage of variance that is explained by variance associated with the predictor
The computation of utility analysis is complex
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

44. Empirical Considerations… Prediction
A statistically significant validity coefficient is helpful for showing the related job success of a group of people, but not individuals
Linear regression and expectancy charts can be used to calculate likelihood of success for individuals
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

45. Empirical Considerations… Prediction (2)
Linear Regression
Involves the determination of how changes in criterion scores are related to changes in predictor scores
An equation mathematically describes the functional relationship between the predictor and criterion
Two common types of regression are simple and multiple
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

46. Figure 8.9 – …Plot of Regression Line for Sales Ability Scores & Performance…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

47. Empirical Considerations… Prediction (2)
Cross-Validation
1. A large group of people on whom predictor & criterion data are available is randomly divided into two groups
2. A regression equation is developed on one group (the “weighting group”)
3. This equation is used to predict the criterion for the other group (holdout)
4. predicted criterion scores are obtained for each
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

48. Empirical Considerations… Prediction (2)
Cross-Validation (2)
4. predicted criterion scores are obtained for each person in the holdout group
5. For the holdout group, predicted criterion scores are then correlated with actual criterion scores. A statistically significant correlation coefficient indicates that the equation is useful for other individuals
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

49. Figure 8. 10 – …Scattergram Scores…on the Employment Interview
Figure 8.10 – …Scattergram Scores…on the Employment Interview..Performance
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

50. Table 8.4 – Percentage of Financial Advisers Rated as Superior…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

51. Figure 8.11 – Individual Expectancy Chart Showing Chances in 100…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

52. Figure 8.12 – Institutional Expectancy Chart Showing Chances in 100…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

53. Empirical Considerations… Factors Affecting the Magnitude of Validity..
Reliability of Criterion and Predictor
Restriction of Range
Criterion Contamination
Violation of Statistical Assumptions
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

54. Figure 8.13 – Appropriateness of Pearson r for Various … Relationships
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

55. Empirical Considerations… Utility Analysis
A Definition of Utility Analysis
The goal is to translate the results of a validation study into terms that are important to and understandable by managers, using dollars-and -cents terms
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

56. Empirical Considerations… Utility Analysis
A Definition of Utility Analysis (2)
Some Preliminary Work on Utility Analysis
1. Validity coefficient: the magnitude of the correlation of the selection procedure with a criterion
2. Selection Ratio: ratio of number of people to be hired to number of applicants available
3. Base Rate: Percentage of employees successful on the job without use of the selection procedure
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

57. Figure 8.14 – Examples of Utility Analysis…Under Different Selection…
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

58. Figure 8.15 – Paretian vs. Normal Distribution of Job Performance
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

59. Broader Perspectives of Validity Validity Generalization
An Overview
Validity coefficients for the same selection methods and criteria measures varied greatly for validation studies performed in different organizational settings
Test validity does generalize across situations
Differences found due to methodological deficiencies
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

60. Broader Perspectives of Validity Validity Generalization (2)
Validity Generalization Methods
1. Obtain a large number of published and unpublished studies for a selection procedure
2. Compute the average validity coefficient
3. Calculate the variance of differences reported in the studies
4. Subtract, from the amount of the differences, variance due to small sample size
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

61. Broader Perspectives of Validity Validity Generalization (3)
Validity Generalization Methods (2)
5. Correct the average validity coefficient and the variance for errors due to other methodological deficiencies
6. Compare the corrected variance to the average validity coefficient to determine the variation
7. If differences are small, then differences are concluded to be due to methodology
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

62. Broader Perspectives of Validity Validity Generalization (4)
Conclusions from Validity Generalization Studies
Schmidt & Hunter concluded that it is not necessary to conduct validity studies within each organization for every job
Also, mental ability tests can be expected to predict work performance in most situations
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

63. Broader Perspectives of Validity Validity Generalization (5)
Criticisms of Validity Generalization
The corrective formula used to generalize results usually not based on sufficient data, but on hypothetical values derived from other research work assumed appropriate
Correction formulas may be inappropriate, and may overestimate the amount of variance attributable to study deficiencies
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

64. Broader Perspectives of Validity Validity Generalization (6)
Validity Generalization Requirements
1. The user must be able to show that the proposed selection procedure assess the same WRCs or that it is a representative example of the measure used in the validity generalization study database
2. The user must be able to show that the job in the new employment setting is similar to the jobs or groups of jobs included in the validity generalization study database
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

65. Broader Perspectives of Validity Job Component Validity & Synthetic Validity
An Overview
A number of different approaches
Involves demonstrating that a correlation exists between a selection procedure and at least one specific aspect or component of a job
Once established, assumed that the selection procedure is valid for predicting performance on that job component if it exists on other jobs
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

66. Broader Perspectives of Validity Job Component Validity & Synthetic Validity (2)
Conducting a Job Component Validity Study
1. Conduct an Analysis of the Job Using the Position Analysis Questionnaire
2. Identify the Major Components of Work Required on the Job
3. Identify the Attributes Required to Perform the Major Components of the Job
4. Choose Tests that Measure the Most Important Attributes Identified from the PAQ
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

67. Broader Perspectives of Validity Job Component Validity & Synthetic Validity (3)
Accuracy of Job Component Validity Studies
Generally, job component validity estimates similar to validity coefficients obtained in actual validation studies
The Department of Labor’s O*NET Database consists of job analysis information on a large array of occupations and 42 generalized work activities that occupations may involve
Use of the O*NET for validation is in its infancy
More research needed
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

68. Broader Perspectives of Validity Job Component Validity & Synthetic Validity (4)
Criticisms of Job Component Validity Strategy
Mossholder & Avery note that method has been less successful in predicting actual validity coefficients
The strategy has been relatively less useful in predicting psychomotor test data
The strategy has reported test results from the General Aptitude Test Battery, available only to public employers
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

69. Validation Options for Small Sample Sizes
Content validity
Validity generalization
Job component validity or some other form of synthetic validity – the logical process of inferring test validity for components of jobs
See Figure 8.16
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

70. Figure 8.16 – Illustration of Test Validation Using Synthetic Validity
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.

71. The Future of Validation Research
Changes occurring in the workplace:
1. Increasing numbers of small organizations without the resources, requirements or skills to undertake traditional validation strategies
2. Increasing use of teams of workers
3. Changes in definitions of job success to include commitment, teamwork, quality, etc.
4. The changing nature of work – jobs and requirements more fluid, requiring analytic methods focusing on broad work capacities
© 2017 Cengage Learning® May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website or school-approved learning management system for classroom use.