1. 2012 Industry Liaison Group National Conference EEO Analysis Bootcamp August 28, 2012  9:00am - 12:00 p.m. Drill Instructors: Dan A. Biddle, Ph.D. Patrick M. Nooren, Ph.D. Biddle Consulting Group, Inc.

2. 193 Blue Ravine, Suite 270 Folsom, CA 95630 916.294.4250 www.biddle.com | www.bcginstitute.org Dan A. Biddle, Ph.D. (dbiddle@biddle.com) Patrick M. Nooren Ph.D. (pnooren@biddle.com) Copyright © 2012 BCG, Inc.2 Contact Information

3. Overview of Biddle Consulting Group, Inc. Affirmative Action Plan (AAP) Consulting and Fulfillment Thousands of AAPs developed each year Audit and compliance assistance AutoAAP™ Enterprise software HR Assessments AutoGOJA™ online job analysis system TVAP™ test validation & analysis program CritiCall™ pre-employment testing for 911 operators OPAC™ pre-employment testing for admin professionals Video Situational Assessments (General and Nursing) EEO Litigation Consulting /Expert Witness Services Over 200+ cases in EEO/AA (both plaintiff and defense) Focus on disparate impact/validation cases Compensation Analysis Proactive and litigation/enforcement pay equity studies COMPare™ compensation analysis software Publications/Books EEO Insight™: Leading EEO Compliance Journal Adverse Impact (3 rd ed.) / Compensation (1 st ed.) BCG Institute for Workforce Development 4,000+ members Free webinars, EEO resources/tools Nation-Wide Speaking and Training Regular speakers on the national speaking circuit

4. BCGi Standard Membership (free) Online community Monthly webinars on EEO compliance topics EEO Insight Journal (e-copy) BCGi Platinum Membership (paid) Fully interactive online community Includes validation/compensation analysis books EEO Tools including validation surveys and AI calculator EEO Insight Journal (e-copy and hardcopy) Members only webinars, training and much more… www.BCGinstitute.org BCG Institute for Workforce Development

5. We have four (4) primary topics to cover in 90 minutes, each capable of being it’s own full- day of training... you did sign-up for a bootcamp!! 5Copyright © 2012 BCG, Inc.

6. Comparison of Incumbency to Availability: What We Do Look Like Compared to What We “Should” Look Like Comparison of Selection Rates: How We Got to Where We Are Today Logistic Regression and Hiring: I Understand There is a Significant Difference in Hiring Rates... But It’s Based on Job- Related Practices, Procedures and/or Tests (PPTs) Multiple Regression and Compensation: I Understand There is a Significant Difference in Pay, But It’s Based on Job-Related Criteria Copyright © 2012 BCG, Inc.6 Agenda

7. Copyright © 2012 BCG, Inc.7 Comparison of Incumbency to Availability (i.e., What We Do Look Like v What We “Should” Look Like)

8. Availability analyses are conducted for each location (or FAAP) by “job group” Job groups are aggregations of jobs that are similar in “content, wage rate, and opportunity” Job groups are used to: ― Increase sample size to yield meaningful results ― Reduce the number of analyses conducted Job groups should never cross EEO categories Important Note: Be thoughtful when creating job groups. You could be artificially creating problems! Copyright © 2012 BCG, Inc.8 What We Do Look Like...

9. Copyright © 2012 BCG, Inc.9 Job Group Analysis

10. What we “should” look like is referred to as the “final availability.” –It is an estimate of the number of qualified minorities or women available for employment in a given job group –It’s a combination of both internal and external comparison data (i.e., factors) used to identify what the composition of those qualified to work in the job group is “supposed” to look like –In the “comparison of incumbency to availability” analysis it will be compared to the job group headcounts to determine the existence of underutilization What We “Should” Look Like... Copyright © 2012 BCG, Inc.10

11. External Factor: –Step 1: Define local labor area –Step 2: Identify/select census occupation codes (472) –Step 3: Mathematically weight census codes based upon representation within each job group –Step 4: Identify relevant data other than local (e.g., state/national – if any) Important Note: Results are only as good as the amount of effort put into this process! What We “Should” Look Like... Copyright © 2012 BCG, Inc.11

12. What We “Should” Look Like... Copyright © 2012 BCG, Inc.12

13. Internal Factor: –Positions are not always filled via external sources –Necessary to identify internal sources of availability information –Step 1: Identify “Feeders” for all jobs/job groups –Step 2: Weight feeders based on historical promotions data (for starters... with a heavy dose of personal review and approval) Target Job GroupWeightFeeder Job Group 1A – Management75.01B – Middle Management (Directors) 25.01C – Managers/Supervisors What We “Should” Look Like... Copyright © 2012 BCG, Inc.13

14. What We “Should” Look Like... Copyright © 2012 BCG, Inc.14

15. Factor Weights: –The weight given to the internal and external availability data (i.e., factors) for each job group –Identifies the relative “importance” of each set of data Assigning factor weights requires the user ask the following question: –“Out of 100 hypothetical movements into this job group, what number do I expect to come from a local recruitment area, reasonable recruitment area, or an internal pool?” What We “Should” Look Like... Copyright © 2012 BCG, Inc.15

16. What We “Should” Look Like... Copyright © 2012 BCG, Inc.16

17. External Census Data Actual Representation (Incumbency/Headcount) Data Internal Availability Data Assign Factor Weights Compare Representation (Incumbency) to Availability Create Goal/Action-Oriented Program If Potential Problem Area Exists Copyright © 2012 BCG, Inc.17 Final Availability Data Comparison of Incumbency to Availability (Overview)

18. Regulations require contractors to compare the percentage of minorities and women in each job group with the availability for those job groups determined in the availability analysis When the percentage of minorities or women employed in a particular job group is less than would reasonably be expected... the contractor must establish a placement goal and create action-oriented programs associated with that goal Copyright © 2012 BCG, Inc.18 Comparison of Incumbency to Availability

19. What is “less than would reasonably be expected”? ― Any Difference: Is there any difference between incumbency and availability? ― Whole Person Rule: Is the difference between incumbency and availability at least one whole person? ― 80% Rule: Is incumbency at least 80% of availability? ― Statistical Significance: Is the difference between incumbency and availability statistically significant? Bonus Question: What is the name of the statistical test used to evaluate incumbency v availability? Fisher Exact? Chi Square? Impact Ratio Analysis? Copyright © 2012 BCG, Inc.19 Exact Binomial! Comparison of Incumbency to Availability

20. Statistical Significance Least proactive Legally-oriented Least goals Any Difference Proactive Diversity-Oriented Most goals/misleading? Whole Person Focus on tangible issues Good with small orgs/job groups Balanced 80% Test Has historical value Misleading? Important Note: Identifying underutilization is NOT a declaration of discrimination. Choose a rule that best represents your organizational size/structure and how it views/perceives affirmative action. Copyright © 2012 BCG, Inc.20 Comparison of Incumbency to Availability

21. Copyright © 2012 BCG, Inc.21 Comparison of Selection Rates: How We Got to Where We Are Today

22. Current Representation (+) Hires (+) Promotions Into (-) Transfers From (-) Voluntary Terms (-) Involuntary Terms (+) Transfers Into (-) Promotions From Copyright © 2012 BCG, Inc.22

23. or the complaining party makes the demonstration described above with respect to an alternate employment practice, and the respondent refuses to adopt such alternative employment practice. Copyright © 2012 BCG, Inc.23 DISPARATE IMPACT An unlawful employment practice based on disparate impact is established only if: and the respondent fails to demonstrate that the challenged practice is job-related for the position in question and consistent with business necessity A complaining party demonstrates that a respondent uses a particular employment practice that causes an adverse impact Adverse/Disparate Impact: Legal Overview

24. Title VII Disparate Impact Discrimination Flowchart “or” Diff. in Rates? YESNO Is the PPT Valid? YESNO Alternative Employment Practice? NO Defendant Prevails YES Plaintiff Prevails END Plaintiff Prevails Practice, Procedure, Test (PPT) Plaintiff Burden Defense Burden Plaintiff Burden How selection processes are challenged... Copyright © 2012 BCG, Inc.24

25. Copyright © 2012 BCG, Inc.25 Males Females Does a practice, procedure or test (PPT) results in disproportionate selection rates by gender, race/ethnicity, or age group? Selection Rate Comparison

26. 26 Men Pass (50) Men Fail (50) Men Passing Rate (50%) Women Pass (25) Women Fail (75) Women Passing Rate (25%) 2 X 2 Table Comparison Impact Ratio Analysis (IRA) Fisher Exact / Chi-Square / 80% Test Results in a value indicating if the observed difference in rates is due to chance (i.e., statistically significant). Note: The chi-square estimates the probability and becomes more accurate as the sample size increases. This is why Fisher’s Exact (adj) is required with small sample sizes. However, BCG recommends Fisher’s Exact (adj) at all times. Copyright © 2012 BCG, Inc. Selection Rate Comparison

27. Statistical Significance (Thresholds): ― 5% ― 0.05 ― 1 chance in 20 ― 2.0 Standard Deviations (actually 1.96) Statistical Significance (Outputs) ―Lower p-values=higher SD (or “Z”) values ―For example: P-value:.05 = 1.96 SDs P-value:.01 = 2.58 SDs 27Copyright © 2012 BCG, Inc. Statistical Significance

28. Adverse Impact: The Typical Approach 28Copyright © 2012 BCG, Inc.

29. Statistical significance: The point at which differences become large enough that one can claim a trend exists. Statistical power: The ability to see those trends if, in fact, they do exist. Statistical power is directly related to effect size and sample size: ― Effect size: The size of the difference in selection rates between two groups... the larger the difference the less number of transactions necessary to detect statistical significance ― Sample size: With larger numbers of transactions it becomes much easier to detect statistical significance 29Copyright © 2012 BCG, Inc. Statistical Significance and Power

30. Enforcement agencies have no control over effect size (i.e., the difference in selection rates), but they do have some control over sample size... which is why they often request two (2) years worth of data to analyze. However, simply aggregating all applicants and all hires across strata (as is typically done), can sometimes result in incorrect/misleading findings. 30Copyright © 2012 BCG, Inc. Statistical Significance and Power

31. 31 CompanyA CompanyC CompanyB CompanyD 0.343 0.168 0.088 0.048 Statistical Significance and Power

32. Analyses by AAP job group regardless of different: ― Job titles ― Selection processes ― Hiring managers ― Basic qualifications ― Locations (perhaps) ― Applicant pools for separate requisitions (perhaps) Typically an aggregation of 12 months (sometimes 18/24 months) worth of transactions into a single 2x2 table Considers everyone who applied throughout the year as available for every hire throughout the year 32Copyright © 2012 BCG, Inc. Adverse Impact: The Typical Approach

33. 33 Men Pass Men Fail Women Pass Women Fail ALL applicants and ALL hires for a 12-month period There is nothing wrong with this approach... as an initial inquiry only. Sometimes this approach is used as the basis for a Notice of Violation (NOV) or plaintiff class action litigation; however, it is up to the employer to provide rebuttal analyses that may reflect reality more accurately (e.g., accounting for recordkeeping issues, etc.) Copyright © 2012 BCG, Inc. Adverse Impact: The Typical Approach

34. Testing YearGroup Applicants (#) Selected (#) Selection Rate (%) 2010 Test Men40020050.0% Women1005050.0% 2011 Test Men1002020.0% Women1002020.0% Fisher Exact Test: SD = 2.16 (Significant) Mantel-Haenszel: SD =.024 (NOT Significant) Simpson’s Paradox Copyright © 2012 BCG, Inc.34 2010 + 2011 Tests Combined Men50022044.0% Women2007035.0%

35. Adverse Impact: The Right Way 35Copyright © 2012 BCG, Inc.

36. 36 Men Pass Men Fail Women Pass Women Fail ++ Req. 1Req. 2Req. 3 ALL applicants and ALL hires throughout the time period = Chi-Square or Fisher’s Exact (adj) Copyright © 2012 BCG, Inc. Single Event v. Multiple Events Men Pass Men Fail Women Pass Women Fail Men Pass Men Fail Women Pass Women Fail Men Pass Men Fail Women Pass Women Fail

37. Mantel-Haenszel (MH) Defined In the context of selection rate analyses, the MH: –is a statistical tool that allows researchers to appropriately combine separate and distinct selection processes (e.g., requisitions) into a single analysis –appropriately allows for the statistical benefits of increased sample size while controlling for Simpson’s Paradox –is a useful tool for evaluating whether the employer has a “pattern and practice” that is possibly discriminatory –may (if appropriate) be used to rebut allegations based on single aggregated analyses over time 37Copyright © 2012 BCG, Inc.

38. The Mantel-Haenszel is not for every situation. It requires separate and distinct pools of applicants/test takers, etc. For example, combining: o Requisitions o Locations o Different jobs in same job group o Different hiring seasons and/or groups Not applicable for “pooled requisitions” where the requisition stays “open” and applicants are just regularly selected from the pool. 38Copyright © 2012 BCG, Inc. Mantel-Haenszel (MH) Defined

39. Component “Step” Analyses 39Copyright © 2012 BCG, Inc.

40. Important Note: Enforcement agencies have every right to investigate the practices, procedures, and tests contractors use to screen applicants. However, in the past, due to resource constraints they wouldn’t typically do so unless there was adverse impact in the overall hiring process. Times have changed! Title VII of 1964/1991 Civil Rights Act An unlawful employment practice based on disparate impact is established under this title only if a complaining party demonstrates that a respondent uses a particular employment practice that causes a disparate impact... 40Copyright © 2012 BCG, Inc. Component “Step” Analyses

41. Male v. Female StepsStartingCompletingResult Overall (App vs. Hired) Male - 100 Female - 100 Male - 50 Female - 30 2.81 SD 1. Basic Qualifications Male - 100 Female -100 Male – 79 Female - 77 0.25 SD 2. TestMale - 79 Female - 77 Male – 65 Female - 35 4.80 SD 3. InterviewMale - 65 Female - 35 Male – 60 Female - 32 0.18 SD 4. Final SelectionMale - 60 Female - 32 Male – 50 Female - 30 0.00 SD Component “Step” Analyses

42. Before you start freaking out, there is a tool. Just join BCGi... 42Copyright © 2012 BCG, Inc. Adverse Impact Toolkit™

43. Logistic Regression and Hiring: I Understand There is a Significant Difference in Hiring Rates... But It’s Based on Job-Related Practices, Procedures, or Tests 43Copyright © 2012 BCG, Inc.

44. The goal is not to instruct how to perform proper logistic regression analyses. The goal is to inform you that there is a way to analyze whether applicant differences in job-related criteria are the “real” reason why there is a disparity. Particularly useful when/if the OFCCP claims discrimination but you know that there is a legitimate reason for the disparity. Copyright © 2012 BCG, Inc.44 Logistic Regression

45. Classic adverse impact analyses can only determine if the numerical difference in passing rates between two groups is significantly different. Logistic Regression (LR) can identify if that numerical difference in passing rates is due to applicant differences in job-related criteria (e.g., experience or education). Copyright © 2012 BCG, Inc.45 Logistic Regression

46. Copyright © 2012 BCG, Inc.46 Hiring Decision(s) Hiring Decision(s) Education Relevant Experience Relevant Experience Gender Logistic Regression

47. LR needs to be applied to job-related factors that were actually used or considered in the selection process LR is useful for weighing the practical importance of job- related factors in the hiring or promotion process LR can potentially “pin” the impact on specific job-related criteria LR can also be useful for determining “shortfall” calculations ― For example, how many women would have been hired “but for” the possible discrimination? ― What is the total shortfall for women, given what the model can explain? Copyright © 2012 BCG, Inc.47 Logistic Regression

48. Gender and Qualification Factors BS.E.WalddfSig.Exp(B) GENDER0.360.450.6910.4141.24 QUALIFICATION 10.470.361.6810.1941.59 QUALIFICATION 20.9150.681.7810.182.49 QUALIFICATION 31.870.934.0410.046.51 QUALIFICATION 40.210.340.3710.531.23 Constant-5.070.7643.7110.000.006 Gender Alone BS.E.WalddfSig.Exp(B) GENDER1.39.4111.6110.00064.03 Constant-3.230.3870.2610.000.039 Gender is significant in the absence of any job-related explanatory variables. After controlling for gender differences in the job-related variables, gender is no longer significant.

49. Multiple Regression and Compensation: I Understand There is a Significant Difference in Pay... But It’s Based on Job- Related Criteria Copyright © 2012 BCG, Inc.49

50. Differences in Compensation Experience Performance Tenure Education Job Market Factors Gender Copyright © 2012 BCG, Inc.50 Multiple Regression Used to create a “model” to determine whether differences in compensation are due to “legitimate job- related factors” or (perhaps) an employee’s gender or ethnicity. Multiple Regression Used to create a “model” to determine whether differences in compensation are due to “legitimate job- related factors” or (perhaps) an employee’s gender or ethnicity. Multiple Linear Regression

51. Copyright © 2012 BCG, Inc.51 Compensation Time with Company Correlation Coefficient (r =.35) Customer Service Representative Correlation

52. Copyright © 2012 BCG, Inc.52 Compensation Employee Gender Percent of compensation explained by gender (r 2 =.35 x.35 = 12.3%) r =.35 Correlation

53. Always between -1.00 and +1.00 Range Close to + or – 1.00: stronger the relationship Close to 0.00: weaker the relationship 0.00: no relationship Size Negative: variables move in the opposite direction Positive: variables move in the same direction Direction Square the correlation coefficient to get the percent of one variable that is accounted for by the other variable Coefficient of Determination Copyright © 2012 BCG, Inc.53 The Correlation Coefficient

54. Copyright © 2012 BCG, Inc.54 Compensation Time with Company Regression/Prediction Line 8 years $55,000 r =.35 Customer Service Representative Correlation and Multiple Regression

55. CompensationCompensation Time In Company Time In Company Perf. Score Experience All variables together become the basis for a prediction “model” known as a regression model. The regression model predicts a certain percentage of what makes up an employee’s compensation. R =.67 R 2 = 45% Copyright © 2012 BCG, Inc.55 The Regression “Model”

56. Q: So how does regression help to identify discrimination in pay? R 2 = 45% without gender R 2 = 51% with gender A: If the prediction model becomes significantly better after including the protected variable. Copyright © 2012 BCG, Inc.56 CompensationCompensation Time In Company Time In Company Perf. Score Experience Gender The Regression “Model”

57. Before you start freaking out (again), there is another tool. Just join BCGi... 57Copyright © 2012 BCG, Inc. COMPare™

58. Copyright © 2012 BCG, Inc. 58 It is imperative that practitioners (at least conceptually) “know” these four statistics: 1.Comparison of Incumbency to Availability ― What is availability and how is it calculated? 2.Adverse Impact Analyses ― Typical v. how to do it right ― Component “step” analyses 3.Logistic Regression – Can job-related factors justify significant differences in selection? 4.Multiple Regression – Can job-related factors justify significant differences in compensation? Summary and Conclusion

59. Questions Answers Copyright © 2012 BCG, Inc.59