1 1 Statistical registers by restricted neighbor imputation – An application to the Norwegian Agriculture Survey Nina Hagesæther and Li-Chun Zhang Statistics.

Slides:



Advertisements
Similar presentations
Estimation of Means and Proportions
Advertisements

Nonparametric estimation of non- response distribution in the Israeli Social Survey Yury Gubman Dmitri Romanov JSM 2009 Washington DC 4/8/2009.
The estimation strategy of the National Household Survey (NHS) François Verret, Mike Bankier, Wesley Benjamin & Lisa Hayden Statistics Canada Presentation.
Prediction and Imputation in ISEE - Tools for more efficient use of combined data sources Li-Chun Zhang, Statistics Norway Svein Nordbotton, University.
DATA MINING van data naar informatie Ronald Westra Dep. Mathematics Maastricht University.
UNECE Work Session on Statistical Data Editing Vienna April 2008 Topic ii – Editing Administrative Data and Combined Sources.
Chapter 8 Estimation: Additional Topics
Chap 9-1 Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chapter 9 Estimation: Additional Topics Statistics for Business and Economics.
STAT262: Lecture 5 (Ratio estimation)
A new sampling method: stratified sampling
Copyright © 2014 by McGraw-Hill Higher Education. All rights reserved.
Chapter 9 Hypothesis Testing.
Q USE OF PROCESS DATA TO DETERMINE THE NUMBER OF CALL ATTEMPTS IN A TELEPHONE SURVEY Annica Isaksson Linköping University, Sweden Peter Lundquist.
Increasing Survey Statistics Precision Using Split Questionnaire Design: An Application of Small Area Estimation 1.
Incomplete Graphical Models Nan Hu. Outline Motivation K-means clustering Coordinate Descending algorithm Density estimation EM on unconditional mixture.
Eurostat Statistical Data Editing and Imputation.
Chapter 7 Estimation: Single Population
Copyright 2010, The World Bank Group. All Rights Reserved. Estimation and Weighting, Part I.
AP Statistics Section 9.3A Sample Means. In section 9.2, we found that the sampling distribution of is approximately Normal with _____ and ___________.
1 Ratio estimation under SRS Assume Absence of nonsampling error SRS of size n from a pop of size N Ratio estimation is alternative to under SRS, uses.
1 Dealing with Item Non-response in a Catering Survey Pauli Ollila Statistics Finland Kaija Saarni Finnish Game and Fisheries Research Institute Asmo Honkanen.
The new multiple-source system for Italian Structural Business Statistics based on administrative and survey data Orietta Luzi, Ugo Guarnera, Paolo Righi.
Using administrative registers in sample surveys European Conference on Quality in Official Statistics 3-–6 May 2010 Kaja Sõstra Statistics Estonia.
1 Pattern Recognition Pattern recognition is: 1. A research area in which patterns in data are found, recognized, discovered, …whatever. 2. A catchall.
Calibrated imputation of numerical data under linear edit restrictions Jeroen Pannekoek Natalie Shlomo Ton de Waal.
EMIS 7300 SYSTEMS ANALYSIS METHODS FALL 2005 Dr. John Lipp Copyright © Dr. John Lipp.
Eurostat Statistical matching when samples are drawn according to complex survey designs Training Course «Statistical Matching» Rome, 6-8 November 2013.
Eurostat Statistical Matching using auxiliary information Training Course «Statistical Matching» Rome, 6-8 November 2013 Marco Di Zio Dept. Integration,
Eurostat Weighting and Estimation. Presented by Loredana Di Consiglio Istituto Nazionale di Statistica, ISTAT.
Chapter 7 Sampling Distributions Target Goal: DISTINGUISH between a parameter and a statistic. DEFINE sampling distribution. DETERMINE whether a statistic.
May 12-15, Evaluating the Integrated Census Israel Pnina ZADKA Central Bureau of Statistics Israel.
Integrated Approach Processing Marie Brodeur Director General, Industry Statistics Branch, Statistics Canada St. Lucia February, 2014 SNA seminar in the.
Representativity Indicators for Survey Quality Programme: Cooperation Theme: Socio-economic sciences and Humanities Activity: Socio-economic and scientific.
Could that be true? Methodological issues when deriving educational attainment from different administrative datasources and surveys Bart F.M. Bakker Manager.
[Topic 1-Regression] 1/37 1. Descriptive Tools, Regression, Panel Data.
- We have samples for each of two conditions. We provide an answer for “Are the two sample means significantly different from each other, or could both.
Statistics for Business and Economics 8 th Edition Chapter 7 Estimation: Single Population Copyright © 2013 Pearson Education, Inc. Publishing as Prentice.
Chapter 12 Vocabulary. Matching: any attempt to force a sample to resemble specified attributed of the population Population Parameter: a numerically.
Synthetic Approaches to Data Linkage Mark Elliot, University of Manchester Jerry Reiter Duke University Cathie Marsh Centre.
Chapter 9 Sampling Distributions 9.1 Sampling Distributions.
How to deal with quality aspects in estimating national results Annalisa Pallotti Short Term Expert Asa 3st Joint Workshop on Pesticides Indicators Valletta.
usually unimportant in social surveys:
Theme (i): New and emerging methods
Chapter 6 Inferences Based on a Single Sample: Estimation with Confidence Intervals Slides for Optional Sections Section 7.5 Finite Population Correction.
Modeling approaches for the allocation of costs
Calibrated estimators of the population covariance
The Language of Sampling
Sampling: Design and Procedures
Sampling with unequal probabilities
Two-Phase Sampling (Double Sampling)
Stratified Sampling STAT262.
Chapter 8 Estimation: Additional Topics
Prepared by Yuliia Halytsia
Treatment effect: Part 2
The European Statistical Training Programme (ESTP)
VAT data in Business Register and Business Statistics
The European Statistical Training Programme (ESTP)
Chapter 8: Weighting adjustment
MEASUREMENT OF THE QUALITY OF STATISTICS
Workshop on Residential Property Price Indices
Chapter 10: Selection of auxiliary variables
Non response and missing data in longitudinal surveys
New Techniques and Technologies for Statistics 2017  Estimation of Response Propensities and Indicators of Representative Response Using Population-Level.
Sampling and estimation
The European Statistical Training Programme (ESTP)
What do Samples Tell Us Variability and Bias.
Sampling Distributions
Chapter 6: Measures of representativity
Canonical Correlation Analysis
The European Statistical Training Programme (ESTP)
Presentation transcript:

1 1 Statistical registers by restricted neighbor imputation – An application to the Norwegian Agriculture Survey Nina Hagesæther and Li-Chun Zhang Statistics Norway

2 Outline Objective Method Empirical results Future work

3 Statistical register: Objective Known Unknown Units Variables

4 Statistical register: Objective Known Unknown Good quality Register variables Poor quality Units Known

5 Statistical register: Objective Known Unknown Units in sample Target variables Units outside sample Register variables

6 Objective Triple-goal criterion (Zhang and Nordbotten, 2008) 1.Efficient estimates 2.Correct covariance structure 3.Non-stochastic

7 The RENI Method REstricted Neighbor Imputation Restrictions: Totals are already estimated Donors = respondents Receivers = population – respondents Nearest neighbor (NN) = unit in same imputation class that satisfy

8 Algorithm Fine-tune phase (FT) –Donor among k nearest neighbors –Choose the donor that best satisfy the restrictions –An iterative process Jump-start phase (JS) –NN imputation for a given proportion of totals –Speeds up the process –Proportion can be reduced or JS omitted

9 Agriculture Survey units in the population, in the sample 84 target variables Publish: class of farmlands in decares (6), farming activity (FA, 12), county Important topics: leasing, investment, maintenance

10 Empirical results – Number of neighbors FA- 2: 2660 receivers, 727 donors FA- 4: 9984 receivers, 3266 donors FA-10: 384 receivers, 243 donors FA-11: 586 receivers, 340 donors

11 Empirical results (FA-4) – Restriction Donors: 3000, Receivers: Alt 1: Equal weight for all 84 restrictions when calculating delta (84) Alt 2: Chosen 12 restrictions 10 times higher weights Alt 3: 9 sets of sub-population restrictions in addition to alternative 2 (12x10) (12) (84)

12 Empirical results (FA-10) – Restriction Donors: 240, Receivers: 380 Alt 1: Equal weight for all 84 restrictions when calculating delta Alt 2: Chosen 12 restrictions 10 times higher weights Alt 3: 9 sets of sub-population restrictions in addition to alternative 2 (12x10) (12) (84)

13 Empirical results – CV RENI / Weighting

14 Empirical results - Correlations Farming activity 10

15 Future work Restriction –How to choose restrictions –How to calculate delta Adjust for partial non-response –Donor and receiver do not match on observed values of receiver –Partial missing in target variables –Unit missing of target variables as partial missing of combined auxiliary and target variables

16 Thank you for your attention! Statistics Norway in Oslo Statistics Norway in Kongsvinger

17 Empirical results – Computation time Farming activity 4 Farming activity 10

18 Empirical results – Computation time

19 Empirical results – Two-way classification Farming activity 10