How to deal with quality aspects in estimating national results Annalisa Pallotti Short Term Expert Asa 3st Joint Workshop on Pesticides Indicators Valletta - Malta January 2008 Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti 1. Dimension of quality 2. Most important aspects 3. How to deal with aspects Contents of presentation Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti This system is the sum of different actions, finalized to treat the non sampling errors. It is possible to summarize this actions in 3 important phases: Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Dimension of quality

Valletta, Malta January 200Annalisa Pallotti QUALITY ASPECTS Preventing actions Actions to estimate the non sampling errors Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey

Valletta, Malta January 200Annalisa Pallotti Non-sampling error is the error attributable to all sources other than sampling error. Non-sampling errors arise during the planning, conducting, data processing and final estimation stages of all types of survey. Non sampling errors Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Preventing actions PLANNING THE SURVEY  Characteristics and reference period  Questionnaire- an important role in developing the questionnaire is played by experts  Survey organization – training of interwievers  Population and frame  Sampling design

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey Sampling, data collection and data entry  Drawing the sample  Data collection  Data entry Data control

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey Data collection: What kind of control we do during data collection? 1.Interviewer (face to face or telephone) 2.List of PPP 3.Comparare data with administrative source

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey After data collection and before data processing the following control phases has to be performed:  Controls by interviewers coordinator  Controls during data entry

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey Controls by interviewers coordinator In this phase the main controls are following: 1.Presence of data in corresponding section 2.Code of the interviewer 3. Main coherence among different section of questionnaire 4.Presence of notes for non-respondent units If some important inconsistencies are found by the supervisor, it is performed a telephone check

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey Controls during data entry In this phase the main controls are automatic control performed by the computer system Example: area cultiveted >= area treated

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions during the survey Non – response the non response problem can be summarized as follows : 1. Partial non -response (method of imputation depends on the type of farms and on type of variable to be imputed) 2. Total non response (unit non-response problem is faced by reweighting the respondent units).

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Actions to estimate the non sampling errors 1.Methods for handling missing or incorrect data items 2.Methods of estimation 3.Evaluation of estimates

Valletta, Malta January 200Annalisa Pallotti Two approaches are used to handle missing or incorrect data items: 1. The first approach is used to face the problem for influent farms (usually the big farms). In these case two actions: a comparison between collected data and administrative data (if available); a telephone check performed by survey staff 2. The second approach is used for the other farms (usually small farms) and it consists of 2 stage, summarized as follows: Methods for handling missing or incorrect data items Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics FIRST PHASE ERROR LOCALIZATION Systematic and deterministic errors Deterministic approach Editing approach combined with grafical data rappresentation (Latouche at al.1992) Editing approach combined with grafical data rappresentation (Latouche at al.1992) Influential outliers

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics FIRST PHASE ERROR LOCALIZATION Non influential outliers random origin Probabilistic Approach (Fellegi and Holt, 1976) Probabilistic Approach (Fellegi and Holt, 1976)

Valletta, Malta January 200Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics SECOND PHASE CORRECTION AND IMPUTATION Both interactive treatment (for outliers and influential errors) and authomatic approach are used Both interactive treatment (for outliers and influential errors) and authomatic approach are used

Valletta, Malta January 200Annalisa Pallotti preliminary quantitative check of the collected questionnaires; Identification and correction of influential errors with respect to the main quantitative variables. To summarize the methods for handling missing or incorrect data items Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti The survey estimates of total for national domain is produced using the following estimator: Where s r,d is the set of respondent, w k is the final weight of unit k and y k. is the variable of interest. The final weights will be obtained as a product of three factors:. Methods of estimation Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti Where d k = 1/ k is the sampling weight. The second factor, y 1k, is computed as the inverse of the response rate on each stratum. The third factor, y 2k, is used to achieve the consistency of sample estimates with respect to some known totals of the population. It is computed using the calibration theory explained in Estevao, Hidiroglou and Särndal (1995). Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics Methods of estimation

Valletta, Malta January 200Annalisa Pallotti The results has to be compared with other statistical and administrative source available. For example, the results of PPP’s use can be compared with PPP’s distributed on the market, or can be compared with specific results of organizations Evaluation of the results Annalisa Pallotti Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics

Valletta, Malta January 200Annalisa Pallotti THANK YOU FOR THE ATTENTION! Multi-Beneficiary statistical Co-operation Programme for Bulgaria, Croatia, Romania and Turkey 2005 Lot 2: Agricultural and Environmental Statistics