1 Chapter 2 Studying & Interpreting Worker Behavior Copyright © The McGraw-Hill Companies, Inc. Royalty-Free/CORBIS

2 Module 1: Science What is science? Science has common methods Science is a logical approach to investigation –Based on a theory, hypothesis or basic interest Science depends on data –Gathered in a laboratory or the field Copyright © The McGraw-Hill Companies, Inc.

3 Common Methods (cont'd) Research must be communicable, open, & public –Research published in journals, reports, or books 1) Methods of data collection described 2) Data reported 3) Analyses displayed for examination 4) Conclusions presented

4 Common Methods (cont'd) Scientists set out to disprove theories or hypotheses –Goal: Eliminate all plausible explanations except one Scientists are objective –Expectation that researchers will be objective & not influenced by biases or prejudices

5 Role of Science in Society Expert witnesses in a lawsuit –Permitted to voice opinions about practices –Often a role assumed by I-O psychologists Copyright © The McGraw-Hill Companies, Inc.

6 Daubert Challenge Challenging testimony of an expert on the grounds it is not scientifically credible Daubert v. Merrill-Dow, 1993 –Resulted in introduction of a method for distinguishing between “legitimate science” & “junk science”

7 Scientific Testimony in Court Theories presented in court must: –Be recognized by particular scientific area as worthy of attention –Be peer reviewed or subjected to scientific scrutiny –Have a known “error rate” –Be replicable or testable by other scientists

8 Module 1 (cont'd) Why do I-O psychologists engage in research? –Better equip HR professionals in making decisions in organizations –Provide an aspect of predictability to HR decisions

9 Module 2: Research Research design –Experimental Random assignment of participants to conditions Conducted in a laboratory or the field –Quasi-experimental Non-random assignment of participants to conditions Copyright © The McGraw-Hill Companies, Inc.

10 Research Design (cont'd) Non-Experimental –Doesn’t include “treatment” or assignment to different conditions –2 common designs: Observational design Survey design Copyright © The McGraw-Hill Companies, Inc.

11 Methods of Data Collection Quantitative methods –Rely on tests, rating scales, questionnaires, & physiological measures –Yield results in terms of numbers Copyright © The McGraw-Hill Companies, Inc. C. Borland/PhotoLink/Getty Images

12 Methods of Data Collection Qualitative methods –Include procedures like observation, interview, case study, & analysis of written documents –Generally produce flow diagrams & narrative descriptions of events/processes Copyright © The McGraw-Hill Companies, Inc.

13 Quantitative & Qualitative Research Not mutually exclusive Triangulation –Examining converging information from different sources (qualitative and quantitative research). Copyright © The McGraw-Hill Companies, Inc.

14 Generalizability in Research Application of results from one study or sample to other participants or situations Every time a compromise is made, the generalizability of results is reduced Copyright © The McGraw-Hill Companies, Inc.

15 Sampling Domains for Research Copyright © The McGraw-Hill Companies, Inc. Figure 2.1 Sampling Domains for I-O Research

16 Control in Research Experimental control –Influences that make results less reliable or harder to interpret are eliminated Statistical control –Statistical techniques used to control for influences of certain variables Copyright © The McGraw-Hill Companies, Inc.

17 Ethics Ethical standards of the APA Collection of 61 cases endorsed by SIOP –Illustrates ethical issues likely to arise in I-O psychology Copyright © The McGraw-Hill Companies, Inc.

18 Module 3: Data Analysis Descriptive statistics –Summarize, organize, describe sample of data Frequency Distribution: –Horizontal axis = Scores running low to high –Vertical axis = Indicates frequency of occurrence Copyright © The McGraw-Hill Companies, Inc.

19 Describing a Score Distribution Measures of central tendency Mean Mode Median Copyright © The McGraw-Hill Companies, Inc. Ryan McVay/Getty Images

20 Describing Score Distribution (cont'd) Variability –Standard deviation Lopsidedness or skew Copyright © The McGraw-Hill Companies, Inc. Ryan McVay/Getty Images

21 Descriptive Statistics: Two Score Distributions Copyright © The McGraw-Hill Companies, Inc. Figure 2.2 Two Score Distribution (N=30)

22 Inferential Statistics Aid in testing hypotheses & making inferences from sample data to a larger sample/population Include t-test, F-test, chi-square test Copyright © The McGraw-Hill Companies, Inc.

23 Statistical Significance Defined in terms of a probability statement Threshold for significance is often set at.05 or lower Copyright © The McGraw-Hill Companies, Inc.

24 Statistical Power Likelihood of finding statistically significant difference when true difference exists Smaller the sample size, lower the power to detect a true or real difference Copyright © The McGraw-Hill Companies, Inc.

25 Concept of Correlation Positive Linear Correlation Copyright © The McGraw-Hill Companies, Inc. Figure 2.4 Correlation between Test Scores and Training Grades

26 Concept of Correlation (cont'd) Scatterplot –Displays correlational relationship between 2 variables Regression –Straight line that best fits the scatterplot Copyright © The McGraw-Hill Companies, Inc.

27 Correlation Coefficient Statistic or measure of association Reflects magnitude (numerical value) & direction (+ or –) of relationship between 2 variables Copyright © The McGraw-Hill Companies, Inc.

28 Correlation Coefficient Positive correlation → As one variable increases, other variable also increases & vice versa Negative correlation → As one variable increases, other variable decreases & vice versa Copyright © The McGraw-Hill Companies, Inc.

29 Various Degrees of Correlation Copyright © The McGraw-Hill Companies, Inc. Figure 2.6 Scatterplots Representing Various Degrees of Correlation

30 Curvilinear Relationship Although correlation coefficient might be.00, it can’t be concluded that there is no association between variables A curvilinear relationship might better describe the association Copyright © The McGraw-Hill Companies, Inc.

31 Curvilinear Correlation Copyright © The McGraw-Hill Companies, Inc. Figure 2.7 An Example of a Curvilinear Relationship

32 Multiple Correlation Multiple correlation coefficient –Overall linear association between several variables & a single outcome variable Copyright © The McGraw-Hill Companies, Inc.

33 Meta-Analysis Statistical method for combining results from many studies to draw a general conclusion Statistical artifacts –Characteristics of a particular study that distort the results –Sample size is most influential Copyright © The McGraw-Hill Companies, Inc.

34 Module 4: Interpretation Reliability –Consistency or stability of a measure –Test-retest reliability Calculated by correlating measurements taken at Time 1 with measurements taken at Time 2 Copyright © The McGraw-Hill Companies, Inc.

35 High Test-Retest Reliability Copyright © The McGraw-Hill Companies, Inc. Figure 2.8 Examples of High and Low Test-Retest Reliability: Score Distributions of Individuals Tested on Two Different Occasions

36 Low Test-Retest Reliability Copyright © The McGraw-Hill Companies, Inc. Figure 2.8 (cont’d) Examples of High and Low Test-Retest Reliability: Score Distributions of Individuals Tested on Two Different Occasions

37 Reliability (cont'd) Equivalent forms reliability –Calculated by correlating measurements from a sample of individuals who complete 2 different forms of same test Internal consistency –Assesses how consistently items of a test measure a single construct Copyright © The McGraw-Hill Companies, Inc.

38 Reliability (cont'd) Inter-rater reliability –Can calculate various statistical indices to show level of agreement among raters –Generalizability theory Simultaneously considers all types of error in reliability estimates Copyright © The McGraw-Hill Companies, Inc.

39 Validity Whether measurements taken accurately & completely represent what is to be measured Predictor –Test chosen or developed to assess identified abilities Criterion –Outcome variable describing important aspects or demands of the job Copyright © The McGraw-Hill Companies, Inc.

40 Criterion-Related Validity Correlate a test score with a performance measure (validity coefficient) Predictive validity design –Time lag between collection of test data & criterion data –Test often administered to job applicants Copyright © The McGraw-Hill Companies, Inc.

41 Criterion-Related Validity (cont'd) Concurrent validity design –No time lag between collection of test data & criterion data –Test administered to current employees, performance measures collected at same time –Disadvantage: No data about those not employed by the organization Copyright © The McGraw-Hill Companies, Inc.

42 Content-Related Validity Demonstrates that content of selection procedure represents adequate sample of important work behaviors & activities or worker KSAOs defined by job analysis Copyright © The McGraw-Hill Companies, Inc.

43 Construct-Related Validity Investigators gather evidence to support decisions or inferences about psychological constructs Construct - concept or characteristic that a predictor is intended to measure; examples include intelligence and extraversion Copyright © The McGraw-Hill Companies, Inc.

44 Construct Validity Model Copyright © The McGraw-Hill Companies, Inc. Figure 2.10 A Model for Construct Validity

45 Permissions Slide 1: McGraw-Hill Education Digital Image Library, Royalty-Free/CORBIS, Source Image ID: CB003901, Filename: EIS0049.JPG Slide 11: McGraw-Hill Education Digital Image Library, C. Borland/PhotoLink/Getty Images, Source Image ID: ST000074, Filename: JPG Slide 19: McGraw-Hill Education Digital Image Library, Ryan McVay/Getty Images, Source Image ID: AA008833, Filename: BS28016.JPG Slide 20: McGraw-Hill Education Digital Image Library, Ryan McVay/Getty Images, Source Image ID: AA027309, Filename: BS32073.JPG Slide 45: McGraw-Hill Education Digital Image Library, C. Sherburne/PhotoLink/Getty Images, Source Image ID: SO000477, Filename: JPG