Neil H. Schwartz, Ph.D. Psych 560

Slides:



Advertisements
Similar presentations
Neil H. Schwartz, Ph.D. Psych 560
Advertisements

Reliability for Teachers Kansas State Department of Education ASSESSMENT LITERACY PROJECT1 Reliability = Consistency.
Developing a Hiring System OK, Enough Assessing: Who Do We Hire??!!
Chapter 10 Decision Making © 2013 by Nelson Education.
COURSE: JUST 3900 INTRODUCTORY STATISTICS FOR CRIMINAL JUSTICE Instructor: Dr. John J. Kerbs, Associate Professor Joint Ph.D. in Social Work and Sociology.
Cutting scores Using tests to improve decisions: Cutting scores & base rates.
EPIDEMIOLOGY AND BIOSTATISTICS DEPT Esimating Population Value with Hypothesis Testing.
Evaluating Hypotheses Chapter 9. Descriptive vs. Inferential Statistics n Descriptive l quantitative descriptions of characteristics.
Evaluating Hypotheses Chapter 9 Homework: 1-9. Descriptive vs. Inferential Statistics n Descriptive l quantitative descriptions of characteristics ~
Effect of Selection Ratio on Predictor Utility Reject Accept Predictor Score Criterion Performance r =.40 Selection Cutoff Score sr =.50sr =.10sr =.95.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc. Chap 9-1 Chapter 9 Fundamentals of Hypothesis Testing: One-Sample Tests Basic Business Statistics.
Part 5 Staffing Activities: Employment
PSY 307 – Statistics for the Behavioral Sciences
Chapter 7 Evaluating What a Test Really Measures
Classroom Assessment A Practical Guide for Educators by Craig A
Chapter 9: Introduction to the t statistic
Introduction to psychological testing
Chapter 5For Explaining Psychological Statistics, 4th ed. by B. Cohen 1 Suppose we wish to know whether children who grow up in homes without access to.
AM Recitation 2/10/11.
Business Statistics, A First Course (4e) © 2006 Prentice-Hall, Inc. Chap 9-1 Chapter 9 Fundamentals of Hypothesis Testing: One-Sample Tests Business Statistics,
Determining Sample Size
From last lecture (Sampling Distribution): –The first important bit we need to know about sampling distribution is…? –What is the mean of the sampling.
MGTO 324 Recruitment and Selections Validity II (Criterion Validity) Kin Fai Ellick Wong Ph.D. Department of Management of Organizations Hong Kong University.
Chapter 8 Introduction to Hypothesis Testing
Hires 5 Offers 10 Interviews 40 Invites 60 Applicants 240 Adapted from R.H. Hawk, The Recruitment Function (New York: American Management Association,
1 Lecture 19: Hypothesis Tests Devore, Ch Topics I.Statistical Hypotheses (pl!) –Null and Alternative Hypotheses –Testing statistics and rejection.
Part 5 Staffing Activities: Employment
Hires 5 Offers 10 Interviews 40 Invites 60 Applicants 240 Adapted from R.H. Hawk, The Recruitment Function (New York: American Management Association,
Chapter 9 Probability. 2 More Statistical Notation  Chance is expressed as a percentage  Probability is expressed as a decimal  The symbol for probability.
Testing in Industrial & Business Settings. How are people hired for a particular job or position? Employment interview Employment interview Can be influenced.
Stats Lunch: Day 3 The Basis of Hypothesis Testing w/ Parametric Statistics.
Measurement MANA 4328 Dr. Jeanne Michalski
Topic #5: Selection Theory
By: Sara Wiatrek & Colton Ploch Add a title for the presentation1.
HED 362 – L. Good Employee Selection - Review Why do we spend time to validate selection procedures? If your Base Rate is high (85%), what does that mean?
Recruiting and Retaining People Lecture 6: Evaluating Recruitment and Selection.
Chapter 5 (Part 2): Recruitment & Selection MGT 3513 It's not what you look at that matters, it's what you see. Henry David Thoreau.
Chapter 13 Linear Regression and Correlation. Our Objectives  Draw a scatter diagram.  Understand and interpret the terms dependent and independent.
ESTABLISHING RELIABILITY AND VALIDITY OF RESEARCH TOOLS Prof. HCL Rawat Principal UCON,BFUHS Faridkot.
© 2013 by Nelson Education1 Decision Making. Chapter Learning Outcomes  After reading this chapter you should:  Appreciate the complexity of decision.
Chapter 7: Interviewing Candidates Lecturer:
Logic of Hypothesis Testing
Section Testing a Proportion
Copyright ©2016 Cengage Learning. All Rights Reserved
Quality Assurance in the clinical laboratory
Psychology Unit Research Methods - Statistics
Chapter 5: Introduction to Statistical Inference
Unit 5: Hypothesis Testing
Review Ordering company jackets, different men’s and women’s styles, but HR only has database of employee heights. How to divide people so only 5% of.
Yield Pyramid Hires 5 Offers 10 Interviews 40 Invites 60
Hypothesis Testing and Confidence Intervals (Part 2): Cohen’s d, Logic of Testing, and Confidence Intervals Lecture 9 Justin Kern October 17 and 19, 2017.
Understanding Results
Chapter 5 (Part 2): Recruitment & Selection
MANA 4328 Dr. Jeanne Michalski
Human Resource Management By Dr. Debashish Sengupta
MANA 4328 Dennis C. Veit Measurement MANA 4328 Dennis C. Veit 1.
Week 10 Slides.
Chapter 9 Hypothesis Testing.
William P. Wattles, Ph.D. Psychology 302
MANA 4328 Dennis C. Veit Measurement MANA 4328 Dennis C. Veit 1.
Significance Tests: The Basics
Practice The Neuroticism Measure = S = 6.24 n = 54
Statistics II: An Overview of Statistics
Inferential Statistics
Personnel decisions Study Unit 4.
Chapter Nine: Using Statistics to Answer Questions
Copyright ©2016 Cengage Learning. All Rights Reserved
Cal State Northridge Psy 427 Andrew Ainsworth PhD
Chapter 5 (Part 2): Recruitment & Selection
Introduction To Hypothesis Testing
Presentation transcript:

Neil H. Schwartz, Ph.D. Psych 560 Personnel Selection Neil H. Schwartz, Ph.D. Psych 560

Introduction Personnel psychology: the study and practice of: Job analysis Job recruitment Employee selection Evaluation of employee performance Organizational psychology: the study of: Leadership Job satisfaction Employee motivation General functioning of organizations.

Personnel Psychology: Employee Selection: EMPLOYMENT INTERVIEWS Research supports structured employment interviews in reaching agreement on employment decisions. Structured interviews produce mean validity coefficients twice that of unstructured interviews. Employment interviews search for negative rather than positive evidence of a person. A single negative impression is followed by rejection 90% of the time, except when an early impression is favorable (then the rejection rate drops to 25%).

Making impressions in employment interviews Negative factors include: Poor communicaiton skills Lack of confidence or poise Low enthusiasm Nervousness Failure to make eye contact Positive factors include: Ability to express oneself, Self-confidence and poise Enthusiasm Ability to sell oneself Aggressiveness A good first impression is one of the most important factors Wear professional attire with good grooming Project an aura of competence and expertise Give an impression of friendliness or personal warmth Be natural without coming on strong– too much is perceived as manipulative.

Personnel Psychology: Employee Selection: BASE RATES & HIT RATES The notion of base rates and hit rates come into play based on the use of tests to provide information about a person beyond what would be known about the person if the test was not used, and how accurate is the decision to accept or reject. This is important because a person must be placed into one of two categories: selected or rejected. Accept High Score Low Score Applicant Cutting Score Decision Reject

Personnel Psychology: Employee Selection: HIT RATES Hit rate is the percentage of cases in which a test accurately predicts success or failure on those people selected and rejected. Accept Selection Decision Good job Performance Hit Bad job Miss Decision Good job Reject Miss High Score Low Score Cutting Score Applicant Bad job Hit The applicant was accepted but did a bad job. This person is a MISS. The goal in personnel selection is to maximize hits and minimize misses.

Personnel Psychology: Employee Selection: BASE RATES Base rate is the percentage of cases in a population in which a particular characteristic occurs without the use of a test. If the base rate is higher than the hit rate, then the use of a test for selection is unnecessary– unless the test is intended to make selections better than the base rate. The key to effective testing for selection is to add information (or features) beyond what is known from the base rate. If a test can add to the base rate, it is worth using. 40% of applicants not taking the MCAT would fail in medical school. Base Rate: 60% [60% of applicants not taking the MCAT would succeed in medical school.] MCAT Hit Rate: 80% [80% of applicants exceeding a score of 30 succeed in medical school] 20% with a score of 30 or above fail in medical school.

Hits and Misses: Types of Misses One type of miss is a FALSE NEGATIVE A false negative is a miss where a test taker is rejected but would have done well. Another type of miss is a FALSE POSITIVE A false positive is a miss where a test taker is accepted but ended up doing poorly. Both types of misses are considered errors in selection. Both types of misses must be evaluated in terms of the costs of being wrong. For example, what is the cost of being wrong in selecting: Medical school applicants for admission? Psychology graduate school applicants for admission? Suicidal adolescents for treatment? Prisoners convicted of murder for parole? A patient for a delicate neurological procedure to remove a tumor?

Understanding Hits and Misses in terms of ACCURACY and DETECTION Consider an expensive radiological procedure as a test. The Procedure can be used to determine the presence of a brain tumor. The example is the same as personnel selection: That is, who should have the procedure done on them to determine if there is a tumor there? And, what if the test misses the brain tumor? Or, what if the test says that there is a brain tumor and the patient actually does not have one? Hmm….

Understanding Hits and Misses in terms of ACCURACY and DETECTION Example: 83% accuracy; 80% detection Test Result Brain Damage Normal Total 8 (A) 2 (B) 10 (A+B) Actual 15 (C) 75 (D) 90 (C+D) 23 77 100 A = Hit B = False negative C = False positive D = Hit A/(A+B) = detection rate (sensitivity) D/(C+D) = specificity base rate (A+D)/A+B+C+D) = accuracy rate Example: 40% accuracy; 44% detection Test Result Brain Damage Normal Total 40 50 90 Actual 10 100

Adding Test Validity to Selection The decision to use a test for selection must depend on what the test offers. A worthwhile test must provide more information than the base rate alone. A test with high criterion-related validity (concurrent or predictive) provides more value in a selection decision than no test or a test with poor validity. In order to determine a test’s contribution to the selection process, Taylor-Russell Tables are essential.

Using Taylor-Russell Tables: Part 1 Taylor-Russell tables allow you to evaluate the validity of a test relative to the amount of information it contributes beyond the base rate. In order to use the Taylor-Russell table you must have the following information: Definition of success The way success is defined: e.g. success on a performance measure of the job; success on a performance measure in medical school; success must be defined dichotomously– either good or bad. Good = above cutoff; Bad = below cutoff. Determination of base rate The percentage of people who would succeed if there was not testing. Definition of selection ratio The percentage of applicants who are selected or admitted. Determination of validity coefficient The correlation between the test and the criterion of success.

Using Taylor-Russell Tables: Part 2 Proportion of Employees Considered Satisfactory = .20 (Base rate.) Using Taylor-Russell Tables: Part 2 There is a Taylor-Russell Table for each base rate. Consider the tables below– one for BR = .20; the other for BR = .50: The entries in the table are the proportion expected to be successful if you use the test.  Base Rate = .20 Selection Ratio Validity coefficient .10 .30 .50 .90 r = .00 .20 r = .25 .34 .29 .26 .21 r = .50 .52 .38 .31 .22 r = .95 .97 .64 .40  Base Rate = .50 Selection Ratio Validity coefficient .10 .30 .50 .90 r = .00 r = .25 .67 .62 .58 .52 r = .50 .84 .74 .54 r = .95 1.00 .99 .56

Understanding Decisions Based on Validity Very Low Correlation Low Correlation High Correlation Very High Correlation

Understanding Decisions Based on Base Rate

Understanding Decisions Based on Selection Ratio Selection ratio. The selection ratio is the number hired divided by the number who applied. If 100 people apply and 50 are hired, the selection ratio is .5. If 100 people apply and 10 are hired, the selection ratio is .1. Suppose we assume that the top people (i.e., those who score highest on the test) will be selected. That is, if we are selecting 10 of 100, we will take the top 10 scorers. In general, other things being equal, the smaller the selection ratio, the more useful the test.

Considering Base Rate, Selection Ratio, and Test Validity Together Proportion of Employees Considered Satisfactory = .20 (Base rate.) Considering Base Rate, Selection Ratio, and Test Validity Together When r = .00, using the test results in a success rate equal to the base rate, which is the same thing as not using the test. If there is no correlation between the test and success on the job, then using the test will not improve selection. As the correlation gets larger, the success rates go up. For example, in the first column of entries in the first table, the base rate is .20, and the selection ratio is .10. When the correlation is .25, the proportion successful is .34, which is up .14 from .20. When the correlation is .50, the success rate is .52, which is up .32 from .20. When the selection ratio is small, changes in the size of the correlation make a lot of difference in the success rate. When the selection ratio is large, however, changes in the size of the correlation make little difference. For example, in the first table when the selection ratio is .9 and the correlation is .25, the expected success rate is .21, which is up .01 from .20. When we move from a correlation of .25 to a correlation of .95, the success rate goes from .21 to .22, which is not much. This happens because when the selection ratio is large, we basically have to hire anyone who applies; we cannot be selective.  Base Rate = .20 Selection Ratio Validity coefficient .10 .30 .50 .90 r = .00 .20 r = .25 .34 .29 .26 .21 r = .50 .52 .38 .31 .22 r = .95 .97 .64 .40  Base Rate = .50 Selection Ratio Validity coefficient .10 .30 .50 .90 r = .00 r = .25 .67 .62 .58 .52 r = .50 .84 .74 .54 r = .95 1.00 .99 .56