Sources of Measurement Errors in Business Surveys

Slides:

Advertisements

Similar presentations

Needs Assessment A brief overview of needs assessment in the context of using ID to plan instructional programs.

Advertisements

MICS4 Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop Survey Quality Control.

Investigation of Treatment of Influential Values Mary H. Mulry Roxanne M. Feldpausch.

Quality assurance Kari Kuulasmaa 1 st EHES Training Seminar, February 2010, Rome.

National training programmes EHES Training seminar, Rome, 12 February 2010 Päivikki Koponen.

Estimating the Level of Underreporting of Expenditures among Expenditure Reporters: A Further Micro-Level Latent Class Analysis Clyde Tucker Bureau of.

1 Session 20: Data Collection in the Field Ina F. Wallace RTI International IES/NCER Summer Research Training Institute, June 2007.

Data Quality Considerations

1 Measuring data quality by the use of a routine re-interview module Experiences from the Norwegian European Social Survey Øyvin Kleven and Frode Berglund.

Cross Cultural Research

Self-Administered Surveys: Mail Survey Methods ChihChien Chen Lauren Teffeau Week 10.

THE IMPLEMENTATION OF E-SURVEY IN THE DEPARTMENT OF STATISTICS MALAYSIA MEETING ON THE MANAGEMENT OF STATISTICAL INFORMATION SYSTEMS APRIL 2014 MANILA.

HOW TO PASS AS EVALUATING STATISTICAL REPORTS.

Robin L. Donaldson May 5, 2010 Prospectus Defense Florida State University College of Communication and Information.

Structured Survey Interviewing: Telephone and In-Person Surveys Behice Ece Ilhan Chihchien Chen.

Survey Design Steps in Conducting a survey.  There are two basic steps for conducting a survey  Design and Planning  Data Collection.

Research methods Basic concepts. Theoretical and technical research  Describe and formalise phenomenons mathematically  Make (non-trivial) proofs in.

Measurement Validity and Reliability. Reliability: The degree to which measures are free from random error and therefore yield consistent results.

Documentation and survey quality. Introduction.

Survey Research & Understanding Statistics

Sociological Research Chapter Two. Copyright © 2004 by Nelson, a division of Thomson Canada Outline  Why is Sociological Research Necessary?  The Sociological.

Responding driven sampling Principles of Sampling Session 1.

RESEARCH METHODS Lecture 19

En Route Surveys Chapter 38 Research Methodologies.

FINAL REPORT: OUTLINE & OVERVIEW OF SURVEY ERRORS

Cover Letters for Survey Research Studies

Diane K. Willimack U.S. Census Bureau Any views expressed on statistical, methodological, or technical issues are those of the authors and not necessarily.

SAMPLING:REQUIREMENTS OF A GOOD SAMPLE

Measurement Error.

Volunteer Angler Data Collection and Methods of Inference Kristen Olson University of Nebraska-Lincoln February 2,

Research Methods in Education

Chapter 29 conducting marketing research Section 29.1

Exploratory Research Design Week 02

Effectiveness of Monetary Incentives and Other Stimuli Across Establishment Survey Populations ICES III 2007 Montreal, Quebec Canada Danna Moore.

“The Effect of Patient Complexity on Treatment Outcomes for Patients Enrolled in an Integrated Depression Treatment Program- a Pilot Study” Ryan Miller,

1 Chapter 10 Fact-Finding Techniques Transparencies.

W HAT IS M&E  Day-to-day follow up of activities during implementation to measure progress and identify deviations  Monitoring is the routine and systematic.

Potential Errors In Epidemiologic Studies Bias Dr. Sherine Shawky III.

How do we assess whether we are improving instrument design? Alice McGee.

Intro to Survey Design and Issues Sampling methods and tips.

Discussion Session 5: Data collection Maiki Ilves Statistics Estonia

Sociological Research Methods. The Research Process Sociologists answer questions about society through empirical research (observation and experiments)

1. To what extent does the Correctional Education Association College of the Air (CEA/COA): a. Increase rates of participation in postsecondary and.

The Development and Validation of the Evaluation Involvement Scale for Use in Multi-site Evaluations Stacie A. ToalUniversity of Minnesota Why Validate.

Ch 15 Bias, Confounding, and Interaction

11 How Much of Interviewer Variance is Really Nonresponse Error Variance? Brady T. West Michigan Program in Survey Methodology University of Michigan-Ann.

Research Scholar Chandrashekara J Under the Guidens Dr. Adithyakumari H Asst. Prof., University of Mysore.

NCRM is funded by the Economic and Social Research Council 1 Interviewers, nonresponse bias and measurement error Patrick Sturgis University of Southampton.

1. 2 Issues in the Design and Testing of Business Survey Questionnaires: Diane K. Willimack U.S. Census Bureau Economic Census The International.

© 2004 McGraw-Hill Companies, Inc., McGraw-Hill RyersonSlide 8-2 TURNING MARKETING INFORMATION INTO ACTION C HAPTER.

IA-DMM Additional Evaluation Efforts. Evaluation questions focused on areas not addressed by FSU Evaluation Plan Evaluation Measures Evaluation Results.

Sources of Motivation in Business Surveys Vanessa Torres van Grinsven, Irena Bolko, Mojca Bavdaž,

Survey Research and Methodology UNL-GALLUP Research Center Evaluating Data quality in Time Diary Surveys Using Paradata Ana Lucía Córdova Cazar Robert.

MUTIA NISRINA SARI JOB ANALYSIS. DEFINITION OF JOB ANALYSIS Job analysis is the process of collection and examination of work activities.

Measurement & Data Collection

Experience of Serbia in conducting pilot Time Use survey

QUESTIONNAIRE DESIGN AND VALIDATION

Economics-VII (Research Methodology) Topic-Data Collection

WRITING AND PUBLISHING RESEARCH ARTICLES

Data Collection Methods

CH. 4 THE RESEARCH PROCESS

Kitchen Stories Getting Credibility Getting Consent Collecting Data

Survey phases, survey errors and quality control system

Survey phases, survey errors and quality control system

Chapter 7: Reducing nonresponse

Experiential Learning Cycle

RESEARCH METHODS Lecture 19

The European Statistical Training Programme (ESTP)

Usefulness of participants’ reports in studying and analyzing outcomes and learning effects in Erasmus+. Siru Korkala 29 Nov 2018.

Deciding the mixed-mode design WP1

Presentation transcript:

Sources of Measurement Errors in Business Surveys Mojca Bavdaz

Outline Measurement errors & their sources. Empirical study. Sources of measurement errors redefined. Implications.

Measurement errors ‘Theoretical’ definition of a measurement error: ‘deviation’ of the observed survey value from the true value (Groves, 1991); ‘difference’ or ‘discrepancy’ between the observed survey value and the true value (Hansen et al., 1951; Sukhatme and Sukhatme, 1970). ‘Practical’ definition of a measurement error: observational gap between the ideal measurement and the response obtained (Groves et al., 2004); error which occurs at the time of data collection (Biemer and Lyberg, 2003).

Sources of measurement errors (1) In household surveys (e.g. Groves, 1989): respondent; interviewer; instrument (the survey questionnaire); mode of data collection.

Sources of measurement errors (2) In business surveys, the addition of the fifth source: records (Ponikowski, Meily, 1989); information system (Biemer, Fecso, 1995; Biemer, Lyberg, 2003); organization (O'Brien, 2000).

Empirical study Business survey under study: Aim: Research methods: Quarterly Business Survey on Trade (SORS). Aim: examination of actual response process. Research methods: in-depth interview; observation. Sample size (n=28). Challenges of implementation.

Three-dimensional model of business survey response

Sources of measurement error redefined Existing sources Identified sources Respondent → + Business participants Interviewers Survey staff Survey questionnaire (Survey instrument) → Survey instrument Mode of data collection Survey characteristics Records/Information system/Organization Organizational context INTERACTIONS!

Implications Improved tools: Three-dimensional model for comprehensive & systematic analysis of response process in business surveys. Redefined sources of measurement errors for development of approaches to measurement error prevention & reduction.

Implications: Example 1 Self-administered business survey. No interviewer. But really no survey staff involved? One-time & rare contacts make huge impact. Training of (all) survey staff.

Implications: Example 2 Recurring business survey. No source of measurement error. But really no impact of recurrence? Learning curve effect & routine. Support & verification of first-time questionnaire completion (at business & respondent level).

Future research More research on specific aspects: comprehension of economic concepts; role of business participants & survey staff; circulation of survey instrument; impact of ‘soft’ issues; ... Quantification through experiments. Generalization to non-business organizational surveys.

Sources of Measurement Errors in Business Surveys Mojca Bavdaz Mojca.Bavdaz@ef.uni-lj.si