Calculation of Sampling Errors MICS3 Data Analysis and Report Writing Workshop

Background The sample selected in a survey is one of the many samples that could have been selected (with same design and size). Sampling errors are measures of the variability between all possible samples, which can be estimated from survey results.

Background Calculation of sampling errors is very important; -Provides information on the reliability of your results -Tells you the ranges within which your estimates most possibly fall -Provide clues as to the sample sizes (and designs) to be selected in forthcoming surveys

Background MICS3 sample designs are complex designs, usually based on stratified, multi-stage, cluster samples. It is not possible to use straightforward formulae for the calculation of sampling errors. Sophisticated approaches have to be used New versions of SPSS (13 or 14) is used for this purpose. SPSS uses Taylor linearization method of variance estimation for survey estimates that are means or proportions. This approach is used by most other package programs: Weswar, Sudaan, Systat, EpiInfo, SAS

Background In MICS3, the objective is to calculate sampling errors for a selection of variables, for the national sample, as well as selected sub-populations, such as urban and rural areas, and regions Sampling errors will be presented as part of the final report, in an appendix

Background

Value of the estimate should be the same as that in the corresponding table

Background Standard error is the square root of the variance – a measure of the variability between all possible samples

Background Coefficient of variation (relative error) is the ratio of SE to the estimate

Background Design effect is the ratio between the SE using the current design and the SE that would result if a simple random sample was used. A DEFT value of 1.0 indicates that the sample is as efficient as a SRS

Background Upper and lower confidence limits are calculated as p +/- 2.SE Indicate the ranges within which the estimate would fall in 95 percent of all possible samples of identical design and size

How SPSS works COMPLEX SAMPLES module Can be used to select a sample, or indicate the design of the sample from which the data set comes, so that sampling error estimates can be calculated Calculations can be done for means and proportions, ratios, frequencies and crosstabs. Also possible to use general linear models and logistic regression.

How SPSS works Prepare an analysis file to indicate the parameters that define the sample design. CSPLAN ANALYSIS /PLAN FILE='micsplan.csplan' /PLANVARS ANALYSISWEIGHT=hhweight /PRINT PLAN /DESIGN STRATA= strat CLUSTER= HH1 /ESTIMATOR TYPE=WR. Using the plan file, calculate sampling errors. Complex Samples Descriptives. CSDESCRIPTIVES /PLAN FILE = 'micsplan.csplan' /SUMMARY VARIABLES =treated iodized /MEAN /STATISTICS SE CV COUNT DEFF DEFFSQRT /MISSING SCOPE = ANALYSIS CLASSMISSING = EXCLUDE.

Problems with using SPSS Need to pair clusters and create pseudo-strata. Cannot handle normalized weights – multiply the weights by 1,000,000 before analysis. Provides estimates for subpopulations only when the data file used contains only cases for the subpopulation in question Provides incorrect confidence limits Cannot report on sampling errors for variables coming from different data sets

SPSS Output