Work in the 21st Century Chapter 2 Methods and Statistics in I-O Psychology

Module 2.1: Science What is science? Approach that involves the understanding, prediction, and control of some phenomenon of interest Science has common methods Science is a logical approach to investigation Based on a theory, hypothesis, or basic interest Science depends on data Gathered in a laboratory or the field

Common Methods (cont'd) Research must be communicable, open, & public Research published in journals, reports, or books 1) Methods of data collection described 2) Data reported 3) Analyses displayed for examination 4) Conclusions presented

Common Methods (cont'd) Scientists set out to disprove theories or hypotheses Goal: Eliminate all plausible explanations except one Scientists are objective Expectation that researchers will be objective & not influenced by biases or prejudices

Role of Science in Society Expert witnesses in a lawsuit Permitted to voice opinions about organizational practices Often a role assumed by I-O psychologists

Module 2.1 (cont'd) Why do I-O psychologists engage in research? Better equip HR professionals in making decisions in organizations Provide an aspect of predictability to HR decisions

Module 2.2: Common Research Designs in I-O Psychology Table 2.1 Common Research Designs in I-O Psychology

Research Designs (cont'd) Experimental Random assignment of participants to conditions Conducted in a laboratory or the workplace Non-experimental Does not include manipulation or assignment to different conditions 2 common designs: Observational design: Observes and records behavior Survey/Questionnaire design (most common) Quasi-experimental Non-random assignment of participants to conditions

Methods of Data Collection Quantitative methods Rely on tests, rating scales, questionnaires, & physiological measures Yield results in terms of numbers C. Borland/PhotoLink/Getty Images

Methods of Data Collection: Qualitative & Quantitative Research Qualitative methods Include procedures like observation, interview, case study, & analysis of written documents Generally produce flow diagrams & narrative descriptions of events/processes Quantitative methods Rely on tests, rating scales, and physiological measures Yield numerical results

Quantitative & Qualitative Research (cont’d) Not mutually exclusive Triangulation Examining converging information from different sources (qualitative and quantitative research).

Generalizability in Research Application of results from one study or sample to other participants or situations The more areas a study includes, the greater its generalizability Every time a compromise is made, the generalizability of results is reduced

Sampling Domains for I-O Research Figure 2.1: Sampling Domains for I-O Research

Control in Research Experimental control Statistical control Eliminates influences that could make results less reliable or harder to interpret Statistical control Statistical techniques used to control for the influence of certain variables

Ethical Behavior in I-O Psychology Ethical standards of the APA SIOP book of 61 cases (Lowman, 1998) Cases illustrate ethical issues that are likely to arise in I-O psychology Joel Lefkowitz (2003) published a recent book on values and ethics in I-O psychology

Table 2.2 Potential Roles Available to the I-O Psychologist and Other HR Managers with Respect to Ethical Problems

Module 3: Data Analysis Descriptive statistics Summarize, organize, describe sample of data Frequency Distribution: Horizontal axis = Scores running low to high Vertical axis = Indicates frequency of occurrence

Describing a Score Distribution Measures of central tendency Mean Mode Median Ryan McVay/Getty Images

Describing Score Distribution (cont'd) Variability Standard deviation Lopsidedness or skew Mean is affected by high or low scores, median is not Mean pulls in direction of skew Ryan McVay/Getty Images

Descriptive Statistics: Two Score Distributions (N = 30) Figure 2.2 Two Score Distribution (N = 30)

Two Score Distributions (N = 10) Figure 2.3. Two Score Distributions (N = 10)

Inferential Statistics Aid in testing hypotheses & making inferences from sample data to a larger sample/population Include t-test, F-test, chi-square test

Statistical Significance Defined in terms of a probability statement Threshold for significance is often set at .05 or lower Significance refers only to confidence that result is NOT due to chance, not strength of an association or importance of results.

Statistical Power Likelihood of finding statistically significant difference when true difference exists The smaller the sample size, the lower the power to detect a true or real difference

Concept of Correlation Positive Linear Correlation Figure 2.4 Correlation between Test Scores and Training Grades

Concept of Correlation (cont'd) Scatterplot Displays correlational relationship between 2 variables Regression Straight line that best “fits” the scatterplot and describes the relationship between the variables in the graph

Correlation Coefficient Statistic or measure of association Reflects magnitude (numerical value) & direction (+ or –) of relationship between 2 variables Ranges from 0.00 and 1.00

Correlation Coefficient Positive correlation → As one variable increases, other variable also increases & vice versa Negative correlation → As one variable increases, other variable decreases & vice versa

Scatterplots of Various Degrees of Correlation Figure 2.6. Scatterplots of Various Degrees of Correlation

Curvilinear Relationship If correlation coefficient is .00, one cannot conclude that there is no association between variables A curvilinear relationship might better describe the association

Curvilinear Correlation Figure 2.7 An Example of a Curvilinear Relationship

Multiple correlation coefficient Overall linear association between several variables & a single outcome variable

Meta-Analysis Statistical method for combining results from many studies to draw a general conclusion Statistical artifacts Characteristics of a particular study that distort the results Sample size is typically the most influential statistical artifact

Module 4: Interpretation Reliability Consistency or stability of a measure Test-retest reliability Calculated by correlating measurements taken at Time 1 with measurements taken at Time 2

High and Low Test-Retest Reliability Figure 2.8 Examples of High and Low Test-Retest Reliability: Score Distributions of Individuals Tested on Two Different Occasions

Equivalent forms reliability Reliability (cont'd) Equivalent forms reliability Calculated by correlating measurements from a sample of individuals who complete 2 different forms of same test Internal consistency Assesses how consistently items of a test measure a single construct

Inter-rater reliability Reliability (cont'd) Inter-rater reliability Can calculate various statistical indices to show level of agreement among raters Values in the range of .70 to .80 represent reasonable reliability Generalizability theory Simultaneously considers all types of error in reliability estimates

Validity Whether measurements taken accurately & completely represent what is to be measured Predictor Test chosen or developed to assess identified abilities or other characteristics (KSAOs) Criterion Outcome variable describing important performance domain

Figure 2.9: Validation Process from Conceptual and Operational Levels

Criterion-Related Validity Correlate a test score (predictor) with a performance measure; resulting correlation often called a validity coefficient Predictive validity design Time lag between collection of test data & criterion data Test often administered to job applicants

Criterion-Related Validity (cont'd) Concurrent validity design No time lag between collection of test data & criterion data Test administered to current employees, performance measures collected at same time Disadvantage: No data about those not employed by the organization

Content-Related Validity Demonstrates that content of selection procedure represents adequate sample of important work behaviors & activities or worker KSAOs defined by job analysis I-O Psychologists can use incumbents/SMEs to gather content validity evidence

Construct-Related Validity Investigators gather evidence to support decisions or inferences about psychological constructs Construct - concept or characteristic that a predictor is intended to measure; examples include intelligence, extraversion, and integrity

A Model for Construct Validity Figure 2.10. A Model for Construct Validity

Construct Validity Model of Strength and Endurance Physical Factors Figure 2.11