Rotating Panels – Especially with Regards to Business Statistics Peter Tibert Stoltze Statistical Methodology Forum for Sample Survey and Estimation April 11, 2012

Selection of the Sample When population, stratification, and allocation are in place, it is easy to extract the sample by applying e.g. proc surveyselect in SAS proc surveyselect data=pop method=srs seed= n=sampsize out=samp; strata stratum; id cvrnr; run; However, sometimes there are restrictions as to the selection, e.g. if we want to control overlaps between samples by introducing rotating panels Forum for Survey Sampling and Estimation2

Rotating Panel Samples Controlled overlaps of samples Forum for Survey Sampling and Estimation3 AdvantagesDisadvantages Respondents gradually learn to fill out the questionnaire correctly (can be interpreted as a reduction of measurement error) A smaller degree of uncertainty with regard to the change due to positive correlation between replies at t and t-1 Systematic bias if the sample is not maintained correctly (e.g. no inclusion of new units in the population)

Rotating Panels in Research, Development and Innovation (R&D) Panel Forum for Survey Sampling and Estimation4

Structure of Rotating Panels With a panel divided into 5 parts, selected units participate as starting point for five years, and thereafter they are ”released” This ideally corresponds to an overlap of 80 pct. and a replacement of 20 pct. However, for this to be true neither growth nor shrinkage of the population is allowed, and the allocation has to remain unchanged Forum for Survey Sampling and Estimation5

Principle of Stratified Selection Allocation n h at stratum level as usual Substrata h i are defined according to the stratum to which the unit belonged last time (quadratic matrix with strong diagonal) or whether the unit is new Allocation n hi at substratum level by means of proportional distribution of n h according to N hi and with stochastic rounding Forum for Survey Sampling and Estimation6

Definition of substrata Stratum in previous survey Stratum in current survey. (new units) 123…H  1N 1,0 N 1,1 N 1,2 N 1,3 …N 1,H N1N1 2N 2,0 N 2,1 N 2,2 N 2,3 …N 2,H N2N2 3N 3,0 N 3,1 N 3,2 N 3,3 …N 3,H N3N3 …………………… HN H,0 N H,1 N H,2 N H,3 …NH,HNH,H NHNH ......N Forum for Survey Sampling and Estimation7

Allocation at substratum level for stratum h iN hi (population) N hi (sample) Typical result by ordinary rounding Example of stochastic rounding …… h h+101 …… H00  Forum for Survey Sampling and Estimation8

Design weights for substrata Forum for Survey Sampling and Estimation9

Rotating Panels in R&D 2011 The four most recent panels (5-8 in R&D 2010) maintain their panel number, while all others (including the just released panel 4) are assigned panel number 0 Within substrata, the units are sorted in random order – however, the highest panel numbers are at the top The first n hi units are selected for the sample – Sometimes, we do not succeed in including everyone from panels 5-8 (especially, if there are many new units) – Selected units in addition to panels 5-8 make up the new panel 9 (can contain in principle units just released from panel 4) The principle implies that the design weight at stratum level N h /n h represents correctly new units and stratum changes Forum for Survey Sampling and Estimation10

data samplepop; retain n nn; set samplepop; sample_2011 = 0; /* initializing with npophi (N_hi) and nstikhi (n_hi) */ by stratum stratum_old; if first.stratum_old then do; n = nstikhi; nn = npophi; end; /* if there is more sample left, then you can select */ if (n>0) then do; /* choosing from panels first */ if panel in (5,6,7,8) then do; sample_2011 = 1; n = n-1; end; /* alternatively, you can select outside panel */ else do; if (n/nn > ranuni(763476)) then do; sample_2011 = 1; panel = 9; n = n-1; end; /* always decrease nn by 1 */ nn = nn-1; run; Forum for Survey Sampling and Estimation11

Surveys at Statistics Denmark with Rotating Panels Implemented 1 Quarterly surveys – Job vacancies (panel divided into 12 parts) – Survey of pigs (panel divided into 8 parts) Annual surveys – Research, Development and Innovation (panel divided into 5 parts) 1) Panels which are presented here Forum for Survey Sampling and Estimation12