What does a test score of 85 (out of 100) indicate? When you get a test back in class, what is one of the first questions that you ask?

Test Score Distribution Why is this problematic? Low Variability * * * * * * * * * * * * * * * * * * * * * * * * * * * 0 10 20 30 40 50 60 70 80 90 100 Test Score Distribution

* Positively Skewed Distribution Negatively Skewed Distribution 40 45 55 60 70 75 80 90 100 40 45 55 60 70 75 80 90 100 Test Scores Test Scores

* Normal Curve Central Tendency Variability (Spread in Scores) -4 -3 -2 -1 Mean +1 +2 +3 +4 Central Tendency a) Mode (most frequent score) b) Mean (average score; [EX/N]) c) Median (midpoint of scores) Variability (Spread in Scores) a) Range (lowest to highest score) b) Standard Deviation c) Variance

Deviation scores (scores minus the mean Computation of Standard Deviation & Variance Deviation scores (scores minus the mean Squared deviation scores Test Scores X 10 20 30 40 50 x -20 -10 10 20 x2 400 100 EX2 = 1000 (Sum of the squared deviation scores) EX = 150 (EX/N) = 30 (Mean) Mean of the sum of the squared deviation scores EX2/N = 200 (the variance or s2) = standard deviation or s 200 s2 = 14.14 (standard deviation)

Z Score X – Mean ___________ SD

Relationships Among Different Types of Test Scores in a Normal Distribution Number of Cases 2.14% 0.13% 0.13% 2.14% 13.59% 34.13% 34.13% 13.59% -4 -3 -2 -1 Mean +1 +2 +3 +4 Test Score Z score T score CEEB score Deviation IQ (SD = 16) Stanine Percentile -4 -3 -2 -1 0 +1 +2 +3 +4 10 20 30 40 50 60 70 80 90 200 300 400 500 600 700 800 55 70 85 100 115 130 145 4% 7% 12% 17% 20% 17% 12% 7% 4% 1 2 3 4 5 6 7 8 9 1 5 10 20 30 40 50 60 70 80 90 95 100

* Correlation --- Some Key Concepts Consists of a set of ordered pairs Indicates both the magnitude and direction of the relationship between variables c) Range is from -1.0 to + 1.0

Fathers Height (in inches) 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 Example of basic concept of correlation * * * * * * * * * * * * * * * * * * * * * Son’s Height 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Fathers Height (in inches)

Widely Reported Correlations Smoking and lung cancer Media violence and aggression Condom use and sexually transmitted HIV Passive smoking and lung cancer at work Lead exposure and children’s IQ Nicotine patch and smoking cessation Calcium intake and bone mass Homework and academic achievement From Bushman, B.J., & Anderson, C. A. (2001). Media violence and the American public: Scientific facts versus media misinformation, American Psychologist, June/July, 477-489. Asbestos and laryngeal cancer Self-examination and breast cancer -.2 -.1 0 .1 .2 .3 .4

Positive Correlation Test Scores Performance Job * * * * * * * * * * * 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 * * Positive Correlation * * * * Performance Job * * * * * * * * * * * * * * * 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Test Scores

Absenteeism (in hours) 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 * * * * Negative Correlation * * * * Performance Job * * * * * * * * * * * * * * 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Absenteeism (in hours)

* Test Scores Correct Acceptances False Rejections Performance Job 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 * Correct Acceptances False Rejections * * * * Good * * * * * * * Performance Job * * * * * * * * * * * * * * Poor Correct Rejections False Acceptances 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Fail Pass Significant Correlation Test Scores

Test Scores Correct Acceptances False Rejections Performance Job Poor 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 Correct Acceptances False Rejections * * * * * * * * Good * * * * * Performance Job * * * * * * * * * * * * * * * * * * Poor * * * Correct Rejections False Acceptances 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Fail Pass No Correlation Test Scores

Job Performance Scores Test Scores 100 50 100 50

Effect of raising cutoff score? 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 False Rejections Correct Acceptances Effect of raising cutoff score? * * * * Good * * * * * * * Performance Job * * * * * * * * * * * * * * Poor False Acceptances Correct Rejections 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Fail Pass Test Scores

Effect of lowering cutoff score? 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 False Rejections Correct Acceptances Effect of lowering cutoff score? * * * * Good * * * * * * * Performance Job * * * * * * * * * * * * * * Poor False Acceptances Correct Rejections 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 Fail Pass Test Scores

Basic Steps in Research Observation Statement of the Problem (Research Question) State Hypotheses Use/Generate a Theory Design Study Measurement (Collect Data) Statistical Analysis Interpretation (Conclusion)

~ Tips for Choosing a Research Topic ~ Read, read, read (e.g., journal articles) to generate research ideas Pick a topic that interests you (it will be more fun to do) Be realistic (pick a topic that can be accomplished in a reasonable time frame) Improve on prior research (e.g., “limitations of present study,” “suggestions for future research”) Minimize problems, time delays in data collection phase of research

* Basic Scales of Measurement Measurement: The assignment of numerals to events or objects according to certain rules Nominal (Categorization or classification) Yes/No, True/False Male/Female Ordinal (Ranking) 1st, 2nd, 3rd Interval (Equal intervals exist between points on the scale) _______ _______ _______ _______ _______ 1 2 3 4 5 Ratio (an absolute zero point exists) Kelvin scale of temperature Time Height, Weight Secretariat Belmont Stakes (1973) Video

~ I-O Research ~ Measurement Limit collection of categorical data Age in Years: _______ Income: ____________ Age 0 - 18 19 – 25 26 – 35 36 – 45 46 – 55 56 – 65 85 & Above Income 0 ------ 10,000 10,001 – 25,000 25,001 – 35,000 35,001 – 50,000 50,001 – 75,000 75,001 – 100,000 100,000 & Above

~ I-O Research ~ Measurement (cont.) Limit collection of dichotomous data Yes _____ No _____ _____ _____ _____ _____ _____ 1 2 3 4 5 Highly Highly Disagree Agree

~ I-O Research ~ Measurement (cont.) Restrict possibility of missing data Scale Questions 1. 2. 3. 4. 5. Missing Computed score for scale or subscales containing questions #5 and #48 will also be missing 48 49 50 Missing

~ Key Terms ~ Predictor: One that can be manipulated or used to predict scores on the dependent variable Criterion: The variable of interest; the one you are attempting to understand or affect SAT, ACT scores used to predict success in college Interview scores used to predict performance in a job Predictor Criterion

Research Design Options Laboratory Experiment Control Realism Laboratory Experiment Manipulate independent variable Precise measurement of dependent variable Low generalizability Quasi-experiment Less control over variables More generalizability Case study Detailed information Low generalizability Naturalistic observation Low control Data used to identify themes; placed into categories Not common in IO Field study Survey research

Alternative Research Approaches Meta Analysis: Statistical analysis of past research findings More accurate estimate of the true “effect size” More “weight” to findings from larger studies Role of "file drawer" effect Predicting workplace aggression: A meta-analysis. The good, the bad, and the unknown about telecommuting: Meta-analysis of psychological mediators and individual consequences.

Alternative Research Approaches (cont.) Data Mining ("Big Data"): Statistical analysis of large, complex data sets to examine patterns (e.g., Internet activity, social network connections, consumer behavior) Example: Collaboration between Weather Channel & IBM to reduce economic impact of weather on business (about ½ billion/year). Weather data is collected from over 100,000 weather sensors, aircraft, smartphones, buildings, and moving vehicles. Combined data yields 2.2 billion unique forecast points, and an average of more than 10 billion forecasts on an active weather day. Some Uses: Companies can alter staffing and supply chain strategy Insurance companies can forewarn customers, limit damage and reduce costs Power companies can anticipate supply and demand Source: Lisa Morgan, Information Week (2015), LINK

~ Some I-O Research Suggestions ~ More use of “archival” data (many are of high quality with large sample sizes; e.g., government statistics on unemployment rates) Longitudinal studies (assessment of change over time) 3) Report confidence intervals and effect sizes in addition to significance levels (e.g., p < .05) Cohen's d (effect size example) here

~ I-O Research ~ Interesting fact: Substantial amount of I-O studies are non-experimental Overall Point: Best for research to be driven by theories and problem-solving approaches not by methodology/statistics Much research efforts in I-O focus on rather trivial questions that can be studied with “fancy” techniques Bulk of research has limited applied significance

Some Pre-Experimental Designs X = Treatment or Intervention O = Observation or Collection of Data One-Shot Case Study X O One-Group Pretest-Posttest Design O X O Static Group Comparison X O O

6-week training program between tests Did the program work to increase scores? Math Pretest 55 64 44 33 28 63 48 38 46 47 Math Posttest 56 66 46 38 29 63 50 40 48 47 English Pretest 33 35 43 36 20 60 40 31 52 64 English Posttest 35 37 47 36 21 62 40 31 56 66

“Lying” With Numbers % increase 100 90 80 70 60 50 40 30 20 10 Math Math English

An organization reports that accidents have decrease substantially since they began a drug testing program. In 1995, the year before drug testing, the number of accidents was 50. In 1996, the year testing began, the amount dropped to 40. In 1997, the year after drug testing the number of accident dropped to 29. What do you make of this? 55 50 45 40 35 30 25 20 15 10 5 * * * 1995 Drug Testing 1997

Given the illustration below, now what do you make of the effectiveness of the drug testing program? * 65 60 55 50 45 40 35 30 25 20 15 * * * * * * * * 1992 1993 1994 1995 1996 1997 1998 1999 2000

Some Quasi-Experimental Designs Non-Equivalent Control-Group Design O X O O O Time-Series Design O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10 Multiple Time-Series Design O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10 O1 O2 O3 O4 O5 O6 O7 O8 O9 O10

Some True Experimental Designs Pretest-Posttest Control Group Design R O X O R O O R indicates randomization Posttest-Only Control Group Design R X O R O

~ Basic Ethics in Research ~ Informed Consent Right to Privacy Anonymous, confidential treatment of data Protection from Deception (cost-benefit assessment) Debriefing e.g., (explanation of the purpose of the study)

At first glance, is there anything happening here? 50 45 40 35 30 25 20 15 10 5 J F M A M J Jul Aug S O N D

How about now? 10 9 8 7 6 5 4 3 2 1 J F M A M J Jul Aug S O N D