Weighting opt-in samples Some preliminary results from Sweden

The Citizen Panel at LORE Respondents collected through VAAs in 2014: approx 30,000 Probability based recruitment to online panel through postcards: approx 10,000 Other various opt-in recruitments: approx 20,000

Our strategy today We compare several data sources/samples: Basic demographics from Statistics Sweden (population register) The SOM Insitute postal probability survey (quasi benchmark) Our Citizen panel: Mixed opt-in recruitments Opt-in recruitments from 2014 VAA Probability based part of the panel

Various possibilities for weighting and sample adjustments Various ways to adjust samples/panels to be representative (potentially) pre-stratification simple post-stratification (”multiplicative”, cell values/combinations unknown) ”raking” (iterative proportional fitting) post-stratification (”multivariate”/”cell based”) sample matching/propensity score matching

Our data today Citizen panel wave from November/December SOM Insitute data from 2013 (2014 soon available) CP various opt-in: 10,555 responses CP VAA opt-in: 20,286 responses CP probability sample: 5,395 responses This wave included a larger set of variables with identical questions wordings as in other data sources

About our two benchmarks Population register data the true benchmark (Statistics Sweden) But we mainly rely on a quasi benchmark, the SOM Institute surveys Annual mail survey with about 50 percent response rates based on the best possible sampling frame

Some descriptive comparisons to the true benchmark

Gender SwedenSOM MP-OI variousMP-OI VAMP-PROBMP-TOT Woman Man Other Total100 Average absolute deviation from actual benchmark (percentage points)19626 Average absolute deviation from SOM benchmark (percentage points) 10737

Age groups, 4 groups SwedenSOM MP-OI various MP-OI VA MP- PROBMP-TOT years years years years Total100 Average absolute deviation from actual benchmark (percentage points) Average absolute deviation from benchmark (percentage points)5343

Education SwedenSOM MP-OI variousMP-OI VAMP-PROBMP-TOT low medium low medium high high Total100 Average absolute deviation from actual benchmark (percentage points) Average absolute deviation from benchmark (percentage points)

Municipalilty size SwedenSOM MP-OI various MP-OI VA MP- PROBMP-TOT inhabitants inhabitants inhabitants Total100 Average absolute deviation from actual benchmark (percentage points) Average absolute deviation from benchmark (percentage points)154108

Some results Post-stratification weighting in three different ways

First, we use the following variables: Age, gender, education, urbanisation Next, we also add how often they discuss politics

Switching to comparing the different weighting strategies

Conclusions Fewer cells or more fine grained post-stratification hardly makes any difference The probability panel benefits most from this basic weighting Basic demographic weighting doesnt help the opt-in samples much The real action happens when also weighting on a variable related to politics Different effects on different outcome variables

Implications? Quasi benchmark is necessary Or, we need methods like raking, thus combining population register data with quasi benchmarks Or potentially other techniques such as sample matching for example