The job satisfaction-performance literature is biased

Slides:



Advertisements
Similar presentations
Q.
Advertisements

EVAL 6970: Meta-Analysis Vote Counting, The Sign Test, Power, Publication Bias, and Outliers Dr. Chris L. S. Coryn Spring 2011.
Potential of Public Health Systematic Reviews to Impact on Primary Research Professor Laurence Moore September 2007.
Introduction to Evidence-Based Inquiry
Communicate –Reason for evaluating performance Process of performance evaluation Who evaluates performance Method of performance evaluation Communicate.
‘Building Bridges’ An innovative tool to capture small health behaviour changes; the development process. Mills, H., Uphill, M., & Weed, M. Introduction.
International Auditing and Assurance Standards Board Accounting Estimates, Including Fair Value Accounting Estimates, and Related Disclosures ISA Implementation.
What is a Systematic review?. Systematic review  Combination of the best research projects in a specific area Selecting Identifying Synthesizing  Health.
Making all research results publically available: the cry of systematic reviewers.
Daniel Acuña Outline What is it? Statistical significance, sample size, hypothesis support and publication Evidence for publication bias: Due.
The Effect of Computers on Student Writing: A Meta-Analysis of Studies from 1992 to 2002 Amie Goldberg, Michael Russell, & Abigail Cook Technology and.
Introduction to Faculty of 1000 Services Find out how F1000 services can help you search the literature and publish your work By [Your name] F1000 Specialist.
The Campbell Collaborationwww.campbellcollaboration.org C2 Training: May 9 – 10, 2011 Introduction to meta-analysis.
Paper III Qualitative research methodology. Objective 1.3 To what extent can findings be generalized from qualitative studies.
Development and the Role of Meta- analysis on the Topic of Inflammation Donald S. Likosky, Ph.D.
CS 235: User Interface Design November 19 Class Meeting Department of Computer Science San Jose State University Fall 2014 Instructor: Ron Mak
CS 235: User Interface Design April 30 Class Meeting Department of Computer Science San Jose State University Spring 2015 Instructor: Ron Mak
The Validity of Conscientiousness is Moderately Overestimated Sven Kepes & Michael A. McDaniel Virginia Commonwealth University Paper presented to at IPAC.
ACTION RESEARCH By Toni McConnell Rachel Milliken Sarah Montefiore Biljana Milovanovic.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 27 Systematic Reviews of Research Evidence: Meta-Analysis, Metasynthesis,
CSWE Overview This resource highlights key aspects of the mission of the Commission on Research and its goals for the next 5 years. It will then.
1 Assessing the robustness of meta-analytic results: Why sensitivity analyses matter Sven Kepes George Banks Michael A. McDaniel Traci Sitzmann.
Meta-analysis Overview
I-squared Conceptually, I-squared is the proportion of total variation due to ‘true’ differences between studies. Proportion of total variance due to.
Evaluation Research and Meta-analysis
Systematic literature searching in social work: Case study on intimate partner violence. Tony McGinn, PhD Candidate, University of Ulster, Northern Ireland,
Research Methods in I/O Psychology
Performance Appraisal & Workplace Performance
Jacqueline A. Clauss, B.A., Jennifer Urbano Blackford, Ph.D. 
Publication bias evaluations are not routinely conducted in clinical oncology systematic reviews Publication bias and non-reporting found in majority of.
UOG Journal Club: October 2016
Evidence Synthesis/Systematic Reviews of Eyewitness Accuracy
Benefits and Pitfalls of Systematic Reviews and Meta-Analyses
Statistical Quality Control, 7th Edition by Douglas C. Montgomery.
The Literature Search and Background of the Problem
WELCOME TO THE PRESENTATION
Intergenerational Transmission of Work Values: A Meta-Analytic Review Ekin Seçinti, Zeynep Cemalcılar, Nebi Sümer, Murat Kezer INTRODUCTION Recent statistics.
Occupational Self-Efficacy and Job-Related Outcomes in Older Workers
Question 1: What is the baseline of high power?
Testing for moderators
A SYSTEMATIC REVIEW AND META ANALYSIS
Copyright (c) 2005 John Wiley & Sons, Inc.
Increased/Maintained UO
The Validity of Conscientiousness is Overestimated
FINDINGS CUMULATION AND REPORT WRITING
H676 Meta-Analysis Brian Flay WEEK 1 Fall 2016 Thursdays 4-6:50
Type of Data Qualitative data Quantitative data
NURSING RESEARCH “THE LITERATURE REVIEW”
CHAPTER OBJECTIVES The primary objective of this chapter is to show how to compute the matrix inverse and to illustrate how it can be.
Presented by Nancy Vargas.
Sue Todd Department of Mathematics and Statistics
Theoretical Framework & Hypothesis Development
Criticisms of Meta-Analysis
JAMA Pediatrics Journal Club Slides: Mobile Health Interventions for Improving Health Outcomes in Youth Fedele DA, Cushing CC, Fritz A, Amaro CM, Ortega.
The Use of Counterfactual Impact Evaluation Methods in Cohesion Policy
Reliability, Validity, and Bias
Meta-Analysis Information from various studies is used to develop a common metric (effect size) Effect size yields information about the existence of.
Sabine Wollscheid, Senior Researcher, Dr. phil.
The Rosabeth Moss Kanter Award Module 2, Class 2 A Teaching Module Developed by the Curriculum Task Force of the Sloan Work and Family Research Network.
Evaluating the Performance of Salespeople
McLeod 2007.
LO: Why did Churchill become PM in 1940?
EAST GRADE course 2019 Introduction to Meta-Analysis
Student Evaluations of Teaching (SETs)
Publication Bias in Systematic Reviews
D. Fletcher, V. Martinez  British Journal of Anaesthesia 
Meta-analysis, systematic reviews and research syntheses
Centre for Lifelong Learning
META-ANALYSIS PROCEDURES
The utility of optical coherence tomography for diagnosis of basal cell carcinoma: a quantitative review N. Reddy and B.T. Nguyen Department of Dermatology,
Presentation transcript:

The job satisfaction-performance literature is biased Sven Kepes and Michael A. McDaniel Introduction Meta-analytic findings are viewed as a primary means for generating cumulative knowledge and bridging the often lamented gap between research and practice. Concerns regarding meta-analytic results and our cumulative knowledge exist. Sensitivity analyses: Sensitivity analyses in the context of meta-analysis concern the degree to which the results and conclusions of a meta-analysis remain stable when conditions of the data or the analysis change (Greenhouse & Iyengar, 2009). To the extent that results and conclusions remain stable, they can be considered robust. Current study Publication bias analyses were conducted to assess the robustness of the job satisfaction-job performance relation. Method Data source: Judge, Thoresen, Bono, and Patton (2001) Publication bias analyses: Trim and fill analysis Contour-enhanced funnel plots Cumulative meta-analysis by precision Selection models (moderate and severe one-tailed) Results The job satisfaction-job performance relation is likely affected by publication bias. Publication bias led to the likely overestimation of the relation by, on average, .07 or 35%. This overestimation is likely due to suppression of low magnitude correlation estimates in Low complexity jobs (likely overestimated by up to 70%) Longitudinal studies (likely overestimated by up to 50%) Peer-subordinate ratings of performance (likely overestimated by up to 35%) Meta-analytic and publication bias results: Robustness of results and conclusions of the analyses: Forest plots (examples) Discussion The job satisfaction-performance relation is likely overestimated by ~35%. Results reported in ranked journals, especially when performance is measured by peer/subordinate ratings, may not be trustworthy. Our journal publication process is in need of revision. The findings illustrate the need for a rigorous quantitative assessment of the robustness of meta-analytic results. We encourage the use of sensitivity analyses in all meta-analytic reviews. Aligned with the approach of triangulation and customer-centric science, we recommend the reporting of the range of results.. Data from top-tier journals Data from other ranked journals Data from unranked journals