Do Structured Interviews Eliminate Bias? A Meta-Analytic Comparison of Structured and Unstructured Interviews Michael G. Aamodt, Ellyn G. Brecher, Eugene J. Kutcher, & Jennifer D. Bragger Radford University ● College of New Jersey ● Virginia Tech ● Montclair State University Abstract Research Question Results We conducted a meta-analysis of studies investigating the extent to which structured and unstructured interviews are affected by such sources of potential bias as applicant attractiveness, pregnancy, weight, sex, race, and use of non-verbal cues. To be included in the meta-analysis a study had to use an experimental design and directly compare interviews scores of structured and unstructured interviews. On the basis of 24 effect sizes, we found that both unstructured (d = .59) and structured interviews (d = .23) were affected by sources of bias. Though both interviews were affected, unstructured interviews were significantly more susceptible to bias than were structured interviews. Because structured interviews ask all applicants the same questions and use a structured scoring system, do they eliminate the effect of extraneous variables such as age, race, pregnancy, & disability? ► Only 12 studies met the criteria for inclusion ● Disability (2) ● Weight (2) ● Pregnancy (2) ● Sex (3) ● Race (1) ● Priming (1) ● Nonverbal cues (1) ► As expected, extraneous factors such as pregnancy and weight affected scores on unstructured interviews ►Unexpectedly, extraneous factors also affected scores on structured interviews, although to a lesser degree . Background Meta-Analysis Method ► Interviews Vary in Structure ► Highly Structured Interviews ● Are based on a job analysis ● Ask all applicants the same questions ● Have a structured scoring system ● Are more valid (Huffcutt & Arthur, 1994) - Highly structured (ρ = .57) - Unstructured (ρ = .20) ● Reduce racial differences (Huffcutt et al., 2001) - Highly structured (d = .13) - Unstructured (d = .51) ● Reduce gender differences (Huffcutt et al., 2001) - Highly structured (d = .00) - Unstructured (d = .23) ► Find Studies ● Must use experimental design ● Directly compare structured and unstructured ● IV could be any source of bias (e.g., weight, pregnancy) ► Convert findings to d scores for each study ► Cumulate d scores ● Used Meta-Analyzer 5.2 ● Statistics generated - Mean d - 95% confidence interval - % variance expected by sampling error ► Remove outliers ► Search for moderators if necessary Table 1: Meta-analysis results Interview type K N d 95% Confidence Interval SE% Qw Lower Upper Overall 24 1,359 .47 - .19 1.13 40% 60.4* Structured 12 663 .23 100% 3.2 Unstructured 696 .70 - .08 1.50 32% 37.6* Outlier removed 11 648 .59 10.7 K=number of studies, N=sample size, d = mean effect size, SE% = percentage of variance explained by sampling error * Effect sizes are not homogeneous