1 Selective data editing Development & implementation Q 2010 Helsinki Jörgen Svensson Process Owner Statistics Sweden (SCB)

Slides:

Advertisements

Similar presentations

Mexican Export and Import Unit Value Indices. Introduction Export and import price indices are useful for the analysis of foreign trade statistics. Besides.

Advertisements

By: Saad Rais, Statistics Canada Zdenek Patak, Statistics Canada

Innovation data collection: Methodological procedures & basic forms Regional Workshop on Science, Technology and Innovation (STI) Indicators.

Innovation data collection: Advice from the Oslo Manual South East Asian Regional Workshop on Science, Technology and Innovation Statistics.

Innovation Surveys: Advice from the Oslo Manual South Asian Regional Workshop on Science, Technology and Innovation Statistics Kathmandu,

Innovation Surveys: Advice from the Oslo Manual National training workshop Amman, Jordan October 2010.

Work Session on Statistical Data Editing Paris, France, April 2014 Topic (i): Selective editing / macro editing Experiences from Selective Editing.

SELEKT in the International Trade in Services. International Trade in Services Statistics on international trade in services, wages and transfers are.

Industrial Statistics In Afghanistan Economic Statistics Division Central Statistics Organization(CSO) Afghanistan 1.

METODOLOGI SIX SIGMA PERTEMUAN 9 ( Perhitungan Statistik) OLEH: EMELIA SARI.

1 Incorporating Statistical Process Control and Statistical Quality Control Techniques into a Quality Assurance Program Robyn Sirkis U.S. Census Bureau.

ECONOMIC STATISTICS AND NATIONAL ACCOUNT IN ETHIOPIA By Sehin Merawi Central Statistical Agency of Ethiopia.

Sampling Strategy for Establishment Surveys International Workshop on Industrial Statistics Beijing, China, 8-10 July 2013.

1.1 What is Econometrics? A set of techniques for measuring economic relationships. 1.What is an economic relationship? It is a relationship among economic.

15 de Abril de A Meta-Analysis is a review in which bias has been reduced by the systematic identification, appraisal, synthesis and statistical.

Dr. Michael R. Hyman, NMSU Sample Size (Click icon for audio)

GRAPHS OF MEANS How is a Graph of Means Constructed? What are Error Bars? How Can a Graph Indicate Statistical Significance?

Typical Sample Sizes Incidence Rate Calculation.

CONFIDENCE INTERVALS What is the Purpose of a Confidence Interval?

Measuring and Monitoring Poverty for the MDGs Johan A. Mistiaen Economist-Statistician Development Data Group The World Bank Overview of the Approach and.

1 Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. Section 7.2 Estimating a Population Proportion Objective Find the confidence.

The Research Process. Purposes of Research  Exploration gaining some familiarity with a topic, discovering some of its main dimensions, and possibly.

Maintenance of Selective Editing in ONS Business Surveys Daniel Lewis.

Data Editing United Nations Statistics Division (UNSD) 16 March 2011 Santiago, Chile.

Vienna, 23 April 2008 UNECE Work Session on SDE Topic (v) Editing on results (post-editing) 1 Topic (v): Editing based on results Discussants: Maria M.

Chapter 7 Confidence Intervals and Sample Sizes

The Edit Anders Norberg, Statistics Sweden (SCB) Work Session on Statistical Data Editing Ljubljana, Slovenia, 9-11 May 2011.

Ilie Dumitrescu National Institute of Statistics Third International Conference on Agricultural Statistics MEXSAI-ICAS-III Cancun- Mexico, 2-4 November.

Sampling January 9, Cardinal Rule of Sampling Never sample on the dependent variable! –Example: if you are interested in studying factors that lead.

Joint UNECE/Eurostat Meeting on Population and Housing Censuses (28-30 October 2009) Accuracy evaluation of Nuts level 2 hypercubes with the adoption of.

Q2010, Helsinki Development and implementation of quality and performance indicators for frame creation and imputation Kornélia Mag László Kajdi Q2010,

Business Research Methods William G. Zikmund Chapter 17: Determination of Sample Size.

ILO Department of Statistics1 ILO experience in quickly estimating the impact of financial crisis on the global labour market International Seminar on.

Deliverable 2.6: Selective Editing Hannah Finselbach 1 and Orietta Luzi 2 1 ONS, UK 2 ISTAT, Italy.

Update on Selective Editing and Implications for Staff Skills International Trade Conference September 2008 Ken Smart.

Applying Process Indicators to Monitor the Editing Process.

The application of selective editing to the ONS Monthly Business Survey Emma Hooper Office for National Statistics

Jeroen Pannekoek - Statistics Netherlands Work Session on Statistical Data Editing Oslo, Norway, 24 September 2012 Topic (I) Selective and macro editing.

Users’ Attitudes towards the Utilization of Online Government and Business Services in Lebanon.

2007 Commodity Flow Survey (CFS) John Fowler Chief, Commodity Flow Branch Service Sector Statistics Division (SSSD) Talking Freight Webinar December 10,

Statistical Data Editing Anders Norberg, Statistics Sweden (SCB)

Valentina Stoevska ILO Department of Statistics Workshop on MDG Data Reconciliation: Employment Indicators, Beirut, July

Compilation of Distributive Trade Statistics in African Countries Workshop for African countries on the implementation of International Recommendations.

Chapter 9: Inferences Based on Two Samples: Confidence Intervals and Tests of Hypotheses Statistics.

StatisticalData Editing Anders Norberg, Statistics Sweden (SCB)

Topic (iii): Macro Editing Methods Paula Mason and Maria Garcia (USA) UNECE Work Session on Statistical Data Editing Ljubljana, Slovenia, 9-11 May 2011.

Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 12 Analyzing the Association Between Quantitative Variables: Regression Analysis Section.

Topic (i): Selective editing / macro editing Discussants Orietta Luzi - Italian National Statistical Institute Rudi Seljak - Statistical Office of Slovenia.

Targeting of Public Spending Menno Pradhan Senior Poverty Economist The World Bank office, Jakarta.

A selective editing method considering both suspicion and potential impact, developed and applied to the Swedish foreign trade statistics Topic (ii), WP.

MARKET APPRAISAL. Steps in Market Appraisal Situational Analysis and Specification of Objectives Collection of Secondary Information Conduct of Market.

Copyright © 2010 Pearson Addison-Wesley. All rights reserved. Chapter 9 One- and Two-Sample Estimation Problems.

Testing the use of administrative data to edit the 2009 Agriculture Census Dolores Lorca National Statistical Institute of Spain.

INFO 4470/ILRLE 4470 Visualization Tools and Data Quality John M. Abowd and Lars Vilhuber March 16, 2011.

Evaluating the benefits of using VAT data to improve the efficiency of editing in a multivariate annual business survey Daniel Lewis.

Workshop on MDG, Bangkok, Jan.2009 MDG 3.2: Share of women in wage employment in the non-agricultural sector National and global data.

Clinicalmicrosystem.org Global Aim Assessment Theme Global Aim Change Ideas Specific Aim Measures SDSA P DS A P D S A P DS A PDSA

Luxembourg, October Workshop on Backasting SHORT-TERM BUSINESS STATISTICSwww.stat.gov.lt BACKCASTING OF TIME SERIES: PROBLEMS AND RESULTS.

Statistics Chapter 9: Inferences Based on Two Samples: Confidence Intervals and Tests of Hypotheses.

Random Variable.

HISTORICAL AND CURRENT PROJECTIONS

Chapter 9 One- and Two-Sample Estimation Problems.

Improving the efficiency of editing in ONS business surveys

Random Variable.

Web Panels at National Statistical Institutes?

Statistical Office of the Republic of Slovenia

Sampling Distribution of a Sample Proportion

Data processing German foreign trade statistics

Session 2: Work Programme of UNECA Related to National Accounts

New Editing Methods at Statistics Sweden

Presentation transcript:

1 Selective data editing Development & implementation Q 2010 Helsinki Jörgen Svensson Process Owner Statistics Sweden (SCB)

Standardization at SCB Decentralized production Development of CBM:s Editing costly, 33% of budgets Data collection departments, 2006 Standardization – the Lotta project, in

3 Nine case studies Purpose of the project: Try using selective data editing What is the potential gain using the method? Would it be possible to develop and use a common tool?

4 Some results from case studies Survey Reduction % Short term employment, private sector60 Business activity indicators50 Price indices in producer & import stages50 Short term statistics, wages & salaries, private sector 40 Wage & salary structures in the private sector25 Foreign trade (5) Structural business statistics---

SUSPICION SUSP(j, k) = Suspicion of variable j for unit k SUSP(j, k) = 0 if variable value falls within acceptance interval SUSP(j, k) → 1 as value deviates from acceptance limit 0 ≤ SUSP(j,k) ≤ 1

POTENTIAL IMPACT POTIMP = Potential impact POTIMP is weighted absolute difference between observed and predicted value : POTIMP(j,k,d) = for variable j, unit k in domain d w k is sampling weight,  k (d) is domain indicator SELEKT supports several ways to establish predicted value: from time series data and from cross sectional analysis within homogenous groups of units

Flagging suspected errors log(Potential impact) log(Suspicion) 20 Flagged

LOCAL SCORE Local (item) score LScore (j,k,d): LScore (j,k,d) = SUSP(j,k)*|POTIMP(j,k,d)|*Cello(j,d) Cello(j,d) is inversely proportional to the standard error based on previous data

GLOBAL SCORE Global (unit) score GScore(k) is obtained by aggregation of local scores LScore (k, j, d) → LScore (k, j) → GScore(k) → = Summation, Euclidian Summation or Maximum Only those units with GScore larger than a pre-decided threshold are followed up

SELEKT, EDIT and process data 10

Implementation of SELEKT So far three surveys: Business activity indicators Wage & salary structures in the private sector Commodity flow survey 11

12 Documentation A General Methodology for Selective Data Editing