Problems with statistical methods in management research … and a few solutions Presentation for the INFORMS Annual Meeting, Austin, USA, November 2010.

Slides:



Advertisements
Similar presentations
Simple Linear Regression and Correlation by Asst. Prof. Dr. Min Aung.
Advertisements

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and l Chapter 12 l Multiple Regression: Predicting One Factor from Several Others.
Inference for Regression
Econ 140 Lecture 81 Classical Regression II Lecture 8.
Learning Objectives Copyright © 2002 South-Western/Thomson Learning Data Analysis: Bivariate Correlation and Regression CHAPTER sixteen.
Learning Objectives 1 Copyright © 2002 South-Western/Thomson Learning Data Analysis: Bivariate Correlation and Regression CHAPTER sixteen.
Objectives (BPS chapter 24)
Classical Regression III
Research Methods in Social Sciences
© 2010 Pearson Prentice Hall. All rights reserved Least Squares Regression Models.
Empirical Analysis Doing and interpreting empirical work.
1. Estimation ESTIMATION.
Class Handout #3 (Sections 1.8, 1.9)
Do not think the point of your paper is to interpret in detail every single regression statistic. Don’t give each of the independent variables equal emphasis.
Correlation and Simple Regression Introduction to Business Statistics, 5e Kvanli/Guynes/Pavur (c)2000 South-Western College Publishing.
DISMANTLING THE QUANTITATIVE – QUALITATIVE DIVIDE Comments On Hypothesis Testing, Induction, Statistics, Fiction And Epistemological Anarchy Presentation.
1 Pertemuan 13 Uji Koefisien Korelasi dan Regresi Matakuliah: A0392 – Statistik Ekonomi Tahun: 2006.
Pengujian Parameter Koefisien Korelasi Pertemuan 04 Matakuliah: I0174 – Analisis Regresi Tahun: Ganjil 2007/2008.
4-1 Statistical Inference The field of statistical inference consists of those methods used to make decisions or draw conclusions about a population.
Business Statistics - QBM117 Interval estimation for the slope and y-intercept Hypothesis tests for regression.
Statistics for the Social Sciences Psychology 340 Spring 2005 Hypothesis testing with Correlation and Regression.
Chapter 7 Forecasting with Simple Regression
Simple Linear Regression Analysis
QUANTITATIVE METHODS I203 Social and Organizational Issues of Information.
Introduction to Linear Regression and Correlation Analysis
Chapter 9 Statistical Data Analysis
Regression Analysis (2)
Lecture 14 Multiple Regression Model
Learning Objective Chapter 14 Correlation and Regression Analysis CHAPTER fourteen Correlation and Regression Analysis Copyright © 2000 by John Wiley &
Anthony Greene1 Correlation The Association Between Variables.
Fundamentals of Data Analysis Lecture 4 Testing of statistical hypotheses.
Do not think the point of your paper is to interpret in detail every single regression statistic. Don’t give each of the independent variables equal emphasis.
Regression. Idea behind Regression Y X We have a scatter of points, and we want to find the line that best fits that scatter.
© 2003 Prentice-Hall, Inc.Chap 13-1 Basic Business Statistics (9 th Edition) Chapter 13 Simple Linear Regression.
L 1 Chapter 12 Correlational Designs EDUC 640 Dr. William M. Bauer.
Spreadsheet simulation and trial and error methods in statistics Michael Wood University of Portsmouth, UK
Why isn’t the customer always right in academia? The peer review culture and the complication and irrelevance problems. Michael Wood
Copyright © 2010 Pearson Education, Inc Chapter Seventeen Correlation and Regression.
Various topics Petter Mostad Overview Epidemiology Study types / data types Econometrics Time series data More about sampling –Estimation.
Chapter 4 Linear Regression 1. Introduction Managerial decisions are often based on the relationship between two or more variables. For example, after.
AP Statistics Chapter 22 Notes “Comparing Two Proportions”
Section 9-1: Inference for Slope and Correlation Section 9-3: Confidence and Prediction Intervals Visit the Maths Study Centre.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 13 Multiple Regression Section 13.3 Using Multiple Regression to Make Inferences.
Chapter 16 Data Analysis: Testing for Associations.
Political Science 30: Political Inquiry. Linear Regression II: Making Sense of Regression Results Interpreting SPSS regression output Coefficients for.
Inferential Statistics. The Logic of Inferential Statistics Makes inferences about a population from a sample Makes inferences about a population from.
Section Copyright © 2014, 2012, 2010 Pearson Education, Inc. Lecture Slides Elementary Statistics Twelfth Edition and the Triola Statistics Series.
© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 1 Chapter 12 Testing for Relationships Tests of linear relationships –Correlation 2 continuous.
Testing Regression Coefficients Prepared by: Bhakti Joshi February 06, 2012.
Regression Analysis. 1. To comprehend the nature of correlation analysis. 2. To understand bivariate regression analysis. 3. To become aware of the coefficient.
UNIT 2 BIVARIATE DATA. BIVARIATE DATA – THIS TOPIC INVOLVES…. y-axis DEPENDENT VARIABLE x-axis INDEPENDENT VARIABLE.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13.
26134 Business Statistics Week 4 Tutorial Simple Linear Regression Key concepts in this tutorial are listed below 1. Detecting.
QUANTITATIVE METHODS I203 Social and Organizational Issues of Information For Fun and Profit.
BPA CSUB Prof. Yong Choi. Midwest Distribution 1. Create scatter plot Find out whether there is a linear relationship pattern or not Easy and simple using.
Simple Linear Regression and Correlation (Continue..,) Reference: Chapter 17 of Statistics for Management and Economics, 7 th Edition, Gerald Keller. 1.
26134 Business Statistics Week 4 Tutorial Simple Linear Regression Key concepts in this tutorial are listed below 1. Detecting.
Using the regression equation (17.6). Why regression? 1.Analyze specific relations between Y and X. How is Y related to X? 2.Forecast / Predict the variable.
as presented on that date, with special formatting removed
Paired Samples and Blocks
Inference for Regression (Chapter 14) A.P. Stats Review Topic #3
26134 Business Statistics Week 5 Tutorial
Political Science 30: Political Inquiry
Hypothesis Tests for 1-Sample Proportion
Elementary Statistics
LESSON 24: INFERENCES USING REGRESSION
Section 12.2: Tests about a Population Proportion
Do not think the point of your paper is to interpret in detail every single regression statistic. Don’t give each of the independent variables equal emphasis.
BIVARIATE ANALYSIS: Measures of Association Between Two Variables
BIVARIATE ANALYSIS: Measures of Association Between Two Variables
Presentation transcript:

Problems with statistical methods in management research … and a few solutions Presentation for the INFORMS Annual Meeting, Austin, USA, November 2010 Michael Wood, Portsmouth University, UK Presentation based on revised version of Wood (2010)

A case study of a typical journal paper leads to three conclusions: 1.The value of any statistical approach is seriously limited by various factors: e.g. difficulties of generalizing to contexts other than the sample studied. 2.The conventional hypothesis testing format makes results almost meaningless: instead, I suggest using confidence levels for hypotheses – and suggest two ways of doing this (one a bootstrap method on a spreadsheet). 3.The analysis should be far more user-friendly. I’ll start with 3, then 2. Probably no time for 1.

Case study: Glebbeek and Bax (2004)...wanted to test the hypothesis “that employee turnover and firm performance have an inverted U-shaped relationship: overly high or low turnover is harmful.” To do this, they analyzed the performance (profitability) of “110 offices of a temporary employment agency” by building regression models, using “net result per office” (p. 281) as the measure of performance, and staff turnover, and the square of turnover as independent variables, as well as three control variables.

Part of Table 2 in Glebbeek and Bax (2004)

Could easily be more user friendly: Results and curvilinear predictions for Region 1 and mean absenteeism and age (Model 4)

Or... Equivalent of Model 4 …

Null hypothesis testing and p values are not a good idea Many people don’t know what they mean or misinterpret them Don’t tell you how big the effect is, or how likely that the curvilinear hypothesis is right Inverted U-shape hypothesis too obvious to be worth proving? Instead … use confidence intervals, but only possibly for a single parameter, or …

Use bootstrapping : bold line is data prediction, other lines are predictions from resamples representing other samples from same source.

Obviously some of these resamples don’t confirm the inverse U-shape hypothesis, some do Spreadsheet shows that 65% of resamples produce inverted U-shape predictions So confidence in hypothesis is 65% … which is what we want to know! Notice that the two p values (both > 10%) don’t give you this information See Wood (2009b). Analysis uses this spreadsheet: (2009b)

Sometimes possible to use p values to work out confidence levels P value for Turnover in (linear) Model 3 is 0.7%, the unstandardized reg. coefficient is –1 778 So confidence that coefficient negative =99.65% … again avoids null hypotheses (Wood (2009a)Wood (2009a)

Consider whether any statistical approach is likely to be useful … The advantage of statistical methods is that they enable us to peer through the fog of noise variables to see patterns. But for a statistical analysis to be worthwhile it is necessary to check four issues 1.Whether the necessity to focus on easily measurable variables damages the credibility of the results 2.Whether the target population is likely to be of lasting interest 3.Whether the amount of variation explained is likely to be sufficient to justify the effort, and, taking these factors into consideration, 4.Whether the research makes a useful addition to existing knowledge (including “common sense”).

Conclusions 1.Consider usefulness of any statistical approach 2.Avoiding a series of hypotheses to test. Instead, look at size / nature of effects 3.Use confidence intervals not p values 4.Or confidence levels for hypotheses derived from p values derived from bootstrapping 5.Make results as user-friendly as possible

References Glebbeek, A. C., & Bax, E. H. (2004). Is high employee turnover really harmful? An empirical test using company records. Academy of Management Journal, 47(2), Wood, M. (2009a). Liberating research from null hypotheses: confidence levels for substantive hypotheses instead of p values. Wood, M. (2009b). Bootstrapping confidence levels for hypotheses about regression models. Wood, M. (2010). The use of statistical methods in management research: suggestions from a case study.

Thank you Any thoughts? All comments gratefully received