Work in the 21st Century Chapter 2 Methods and Statistics in I-O Psychology
Module 2.1: Science What is science? Approach that involves the understanding, prediction, and control of some phenomenon of interest Science has common methods Science is a logical approach to investigation Based on a theory, hypothesis, or basic interest Science depends on data Gathered in a laboratory or the field
Common Methods (cont'd) Research must be communicable, open, & public Research published in journals, reports, or books 1) Methods of data collection described 2) Data reported 3) Analyses displayed for examination 4) Conclusions presented
Common Methods (cont'd) Scientists set out to disprove theories or hypotheses Goal: Eliminate all plausible explanations except one Scientists are objective Expectation that researchers will be objective & not influenced by biases or prejudices
Role of Science in Society Expert witnesses in a lawsuit Permitted to voice opinions about organizational practices Often a role assumed by I-O psychologists
Module 2.1 (cont'd) Why do I-O psychologists engage in research? Better equip HR professionals in making decisions in organizations Provide an aspect of predictability to HR decisions
Module 2.2: Common Research Designs in I-O Psychology Table 2.1 Common Research Designs in I-O Psychology
Research Designs (cont'd) Experimental Random assignment of participants to conditions Conducted in a laboratory or the workplace Non-experimental Does not include manipulation or assignment to different conditions 2 common designs: Observational design: Observes and records behavior Survey/Questionnaire design (most common) Quasi-experimental Non-random assignment of participants to conditions
Methods of Data Collection Quantitative methods Rely on tests, rating scales, questionnaires, & physiological measures Yield results in terms of numbers C. Borland/PhotoLink/Getty Images
Methods of Data Collection: Qualitative & Quantitative Research Qualitative methods Include procedures like observation, interview, case study, & analysis of written documents Generally produce flow diagrams & narrative descriptions of events/processes Quantitative methods Rely on tests, rating scales, and physiological measures Yield numerical results
Quantitative & Qualitative Research (cont’d) Not mutually exclusive Triangulation Examining converging information from different sources (qualitative and quantitative research).
Generalizability in Research Application of results from one study or sample to other participants or situations The more areas a study includes, the greater its generalizability Every time a compromise is made, the generalizability of results is reduced
Sampling Domains for I-O Research Figure 2.1: Sampling Domains for I-O Research
Control in Research Experimental control Statistical control Eliminates influences that could make results less reliable or harder to interpret Statistical control Statistical techniques used to control for the influence of certain variables
Ethical Behavior in I-O Psychology Ethical standards of the APA SIOP book of 61 cases (Lowman, 1998) Cases illustrate ethical issues that are likely to arise in I-O psychology Joel Lefkowitz (2003) published a recent book on values and ethics in I-O psychology
Table 2.2 Potential Roles Available to the I-O Psychologist and Other HR Managers with Respect to Ethical Problems
Module 3: Data Analysis Descriptive statistics Summarize, organize, describe sample of data Frequency Distribution: Horizontal axis = Scores running low to high Vertical axis = Indicates frequency of occurrence
Describing a Score Distribution Measures of central tendency Mean Mode Median Ryan McVay/Getty Images
Describing Score Distribution (cont'd) Variability Standard deviation Lopsidedness or skew Mean is affected by high or low scores, median is not Mean pulls in direction of skew Ryan McVay/Getty Images
Descriptive Statistics: Two Score Distributions (N = 30) Figure 2.2 Two Score Distribution (N = 30)
Two Score Distributions (N = 10) Figure 2.3. Two Score Distributions (N = 10)
Inferential Statistics Aid in testing hypotheses & making inferences from sample data to a larger sample/population Include t-test, F-test, chi-square test
Statistical Significance Defined in terms of a probability statement Threshold for significance is often set at .05 or lower Significance refers only to confidence that result is NOT due to chance, not strength of an association or importance of results.
Statistical Power Likelihood of finding statistically significant difference when true difference exists The smaller the sample size, the lower the power to detect a true or real difference
Concept of Correlation Positive Linear Correlation Figure 2.4 Correlation between Test Scores and Training Grades
Concept of Correlation (cont'd) Scatterplot Displays correlational relationship between 2 variables Regression Straight line that best “fits” the scatterplot and describes the relationship between the variables in the graph
Correlation Coefficient Statistic or measure of association Reflects magnitude (numerical value) & direction (+ or –) of relationship between 2 variables Ranges from 0.00 and 1.00
Correlation Coefficient Positive correlation → As one variable increases, other variable also increases & vice versa Negative correlation → As one variable increases, other variable decreases & vice versa
Scatterplots of Various Degrees of Correlation Figure 2.6. Scatterplots of Various Degrees of Correlation
Curvilinear Relationship If correlation coefficient is .00, one cannot conclude that there is no association between variables A curvilinear relationship might better describe the association
Curvilinear Correlation Figure 2.7 An Example of a Curvilinear Relationship
Multiple correlation coefficient Overall linear association between several variables & a single outcome variable
Meta-Analysis Statistical method for combining results from many studies to draw a general conclusion Statistical artifacts Characteristics of a particular study that distort the results Sample size is typically the most influential statistical artifact
Module 4: Interpretation Reliability Consistency or stability of a measure Test-retest reliability Calculated by correlating measurements taken at Time 1 with measurements taken at Time 2
High and Low Test-Retest Reliability Figure 2.8 Examples of High and Low Test-Retest Reliability: Score Distributions of Individuals Tested on Two Different Occasions
Equivalent forms reliability Reliability (cont'd) Equivalent forms reliability Calculated by correlating measurements from a sample of individuals who complete 2 different forms of same test Internal consistency Assesses how consistently items of a test measure a single construct
Inter-rater reliability Reliability (cont'd) Inter-rater reliability Can calculate various statistical indices to show level of agreement among raters Values in the range of .70 to .80 represent reasonable reliability Generalizability theory Simultaneously considers all types of error in reliability estimates
Validity Whether measurements taken accurately & completely represent what is to be measured Predictor Test chosen or developed to assess identified abilities or other characteristics (KSAOs) Criterion Outcome variable describing important performance domain
Figure 2.9: Validation Process from Conceptual and Operational Levels
Criterion-Related Validity Correlate a test score (predictor) with a performance measure; resulting correlation often called a validity coefficient Predictive validity design Time lag between collection of test data & criterion data Test often administered to job applicants
Criterion-Related Validity (cont'd) Concurrent validity design No time lag between collection of test data & criterion data Test administered to current employees, performance measures collected at same time Disadvantage: No data about those not employed by the organization
Content-Related Validity Demonstrates that content of selection procedure represents adequate sample of important work behaviors & activities or worker KSAOs defined by job analysis I-O Psychologists can use incumbents/SMEs to gather content validity evidence
Construct-Related Validity Investigators gather evidence to support decisions or inferences about psychological constructs Construct - concept or characteristic that a predictor is intended to measure; examples include intelligence, extraversion, and integrity
A Model for Construct Validity Figure 2.10. A Model for Construct Validity
Construct Validity Model of Strength and Endurance Physical Factors Figure 2.11