Assessment

Review Job analysis identifies criteria to aid selection e.g., task-oriented e.g., person-oriented Knowledge of “criterion problem helps us choose “best” criteria Contamination Deficiency So we know what to measure… we need to understand issues surrounding measurement of human traits

Measurement Measurement – the process of assigning numbers to objects or events according to rules (Linn & Gronlund, 1995). Psychological Measurement – concerned with evaluating individual differences in psychological traits. Trait – descriptive label applied to a group of behaviors (e.g., friendly; intelligent)

Utilizing Individual Differences Psychologists assume that most traits are normally distributed in the population. e.g., height, intelligence, KSAs Psychologists study: measuring these differences using these differences to predict performance

Predictor (X) - any variable used to forecast a criterion Criterion (Y) – variable we are trying to predict, typically performance SAT TestCollege GPA IQJob Performance Key Terms

Types of Assessment Biographical Information ask about person’s past activities, interests, & behaviors. Interviews structured vs. unstructured get info., but also behavior sample Work Samples: ask them to do a representative task & rate their performance Assessment Centers multiple representative tasks Letters of Recommendation Psychological Tests

Hiring the Best Job: College Professor Tasks: lecturing, leading discussion What KSAs are associated with each task? What are the best ways to measure KSAs? Can only collect 2 examples of each type of data to assess key competencies Biographical Data Interview questions Work samples Provide brief rationale for each

Types of Assessment Biographical Information Interviews Work Samples Letters of Recommendation Psychological Tests

Biographical Data Good questions are about events that are: historical external discrete controllable (by the individual) verifiable equal access job relevant non-invasive (Mael, 1991) Rationale vs. empirical method

Biographical Data Strong criterion validity drug use, criminal history predicts dysfunctional police behavior (Sarchione et al., 1998) not redundant with personality (McManus & Kelly, 1999) Measurement issues Generalizability Faking Fairness Privacy concerns

Interviews Structured vs. Unstructured Info. gathering vs. interpersonal behavior sample Situational interview “How would you handle a circumstance in which you needed the help of a person you did not like?” Measurement issues structured has more criterion related validity value of unstructured? Illusion of validity Guidelines for structured interviews

Work Samples perform a task under standardized conditions historically were for blue collar jobs e.g. use of tools, demonstrate driving skills white collar examples speech interview for foreign worker, test of basic chemistry knowledge, Measurement issues high criterion validity if skills are similar to job costly to administer work best with mechanical, rather than people-oriented tasks

Assessment Centers Realistic tasks done in groups Assessed by multiple of raters rating multiple domains Multiple methods in basket group exercise leaderless group exercise Strong criterion validity (e.g., teachers, police) overall scores predict job performance Measurement issues costly to administer different ratings on a task too highly correlated dimension ratings not correlated strongly across tasks fix? focus on behavior checklists and rater training

Drug Testing opinion? People are more accepting of it if job involves risks to others (Paronto, et al., 2002) Measurement issues reliability is very high, but not perfect Validity? Normands, Salyards, & Mahoney (1990)  over 5000 postal service applicants  those who tested positive had 59% higher absenteeism, 47% more likely to be fired  no differences in injury or accidents

Letters of Recommendation ever written a letter of recommendation for someone? worst criterion validity of all commonly used assessment tools some use for screening extremely bad candidates Measurement issues restriction of range writer bias/investment

Psychological Test Characteristics Group vs. individual Objective vs. open-ended Paper & pencil vs. performance Power vs. speed

Psychological Test Types Ability Tests Cognitive ability Psychomotor ability Knowledge and skill or achievement Integrity Personality Emotional Intelligence Vocational interest

Integrity Tests Designed to predict whether employee will engage in counterproductive work behavior (CWB) overt vs. personality (covert) Better at predicting general CWB and performance than theft (r = ) Measurement issues difficult to measure criteria! proprietary issues legal and privacy issues faking

Personality Tests measures predispositions toward particular feelings and behaviors not all tests are based on past research many have shown incremental validity e.g., predict when controlling for IQ Measurement issues job relevance not easily/often faked or a problem if faked (e.g., job faking too)

The Big 5 Short version of Big 5

The Big Five Inventory Openness Highs: imaginative, creative, and to seek out cultural and educational experiences. Lows: more down-to-earth, less interest in art & more practical. Conscientiousness Highs: methodical, well organized and dutiful. Lows: less careful, less focused & more likely to be distracted Extraversion Highs: energetic and seek out the company of others. Lows (introverts): tend to be more quiet and reserved. Agreeableness Highs: tend to be trusting, friendly and cooperative. Lows: tend to be more aggressive and less cooperative Neuroticism Highs: prone to insecurity and emotional distress. Lows: more relaxed, less emotional and less prone to distress.

Big 5 Conscientiousness show consistent correlation with job performance across jobs, across cultures Emotional stability and agreeableness predict performance in customer service jobs

Intelligence Tests Have greatest validity Often very easy and inexpensive to use Wonderlic Personnel Test 50 items 12 minute time limit Sample questions Interpreting scores? Scores vary as a function of race and ethnicity Ethical issues? Face validity?

Determining “Test” Utility Goal of testing is to make decisions about individuals on the basis of the amount of a given trait they possess. A test should give us a “true” picture of a person’s traits Test Score = True Score + Error

Reliability and Validity Reliability Test-retest Parallel (Alternate) forms Internal Consistency Validity Face Content Criterion-related Construct-related

Selecting a Predictor Validity Fairness Applicability Cost