WORKSHOP FOUR MAKE DATA MEANINGFUL: MAKE THE INVISIBLE, VISIBLE Small Area Estimates: an Illustration for Family Planning using Nepal as a Case Study I Sabrina Juran Ph.D. Technical Specialist, Data and Population Analysis United Nations Population Fund

Family Planning Indicator (SDG3.7.1) SDG Indicator 3.7.1: Percentage of women of reproductive age (15-49 Years) who have their need for family planning satisfied with modern methods mCPR CPR + Unmet need X100 SDG3.7.1 = Contraceptive prevalence rate (CPR) - percentage of women who are currently using, or whose sexual partner is currently using, at least one method of contraception, regardless of the method used. mCPR - contraceptive prevalence rate using modern methods Unmet need - percentage of women of reproductive age, either married or in union, who want to stop or delay childbearing but are not using any method of contraception. SDG 3 Health Target 3.7 to ensure universal access to SRH services One indicator to measure prigress in SDG 3.7.1 Agency working on SRH – we are responsible for global monitoring and supporting countries in their efforts to report on this target. More importantl however, as an operational agency with programmes in more than 150 countties we need to make sure that our interventions are most eeffeciemtyl targeted. If we want to know where to invest in terms of awareness raising, sexual education, service provision, we need to know the actual situation of family planning on the ground. Primary data source for FP data are DHS and MICS.

Disaggregation Challenges General data disaggregation All data sources provide possibilities for data disaggregation but all have limitations Geographic data disaggregation Most of the data used for analysis of contraceptive dynamics in developing countries is collected via household surveys. Sample limitations allow disaggregation only to regional level, at best provincial/state level General data disaggregation challenges All data sources (administrative records, CRVS, census, surveys, etc.) provide possibilities for data disaggregation but all have limitations (completeness, periodicity, access, sampling, etc.) Geographic data disaggregation challenges Most of the data used for contraceptive dynamics analysis in the developing world is collected via household surveys (DHS and MICS). Sample limitations allow disaggregation only to regional level, at best provincial/state level

Family Planning Status in Nepal By year: 1990-2015 By Age Groups

Nepal 5 Regions; 15 Sub-regions; 75 Districts 2011 Nepal Demographic and Health Survey (DHS) represent for 13 sub-regions (these 3 sub-region in DHS are aggregated as “western mountain”)

Sub-regional direct estimates of contraceptive prevalence rate using DHS 2011 Far-western, mid-western and western mountain apply the same data analysis results

Geographic disaggregation problems Number Min Individual MAX Individual MEAN Individual Total Individual Clusters 289 3 153 42 12,023 Kathmandu DHS: Data disaggregation below the sub-regional or provincial level involves substantial level of uncertainty and thus not recommended Census: Do not have family planning indicators

Small Area Estimates (SAE) as an Alternative Using SAE to produce estimates of Family Planning (FP) indicators for each one of the 75 districts in Nepal The SAE technique takes advantage of the existing correlation between FP indicators and a set of common variables in the 2011DHS and the 2011 Population Census Public Use Microdata Sample data (age, number of children, urban/rural residence, education, water and sanitation, etc.) to predict values for contraceptive dynamics indicators at the district level using regression models.

Key Steps of SAE and Application Small Area Estimation (SAE) Data Analysis and Assessment Regression model coefficients for predicting probability of using contraception for an individual woman are obtained from the 2011 NDHS data The national coefficients from DHS are applied to the 2011 Census Data to predict the probability of using contraception for individual women The individual contraceptive use probabilities from census data are aggregated (using average) to district level Application of SAE Estimate the number of women aged 15-49 (married or in union) in need of contraception Identify the priority districts Major Steps: 1. Identify common variables associated with contraceptive indicators in DHS and census data DHS CENSUS 2. Develop a model for predicting the probability of individual contraceptive use using DHS data 3. Apply the model to census and estimate the probability of individual contraceptive use 4. Aggregate the estimation of individual contraceptive use to small area administrative level and map results Variable 1 Variable 2 Variable 3 …… Data Analysis and Assessment Identify a set of common variables associated with contraceptive dynamics indicators in 2011 NDHS and 2011 Population Census. 2011 NDHS and 2011 Population Census data assessment and comparison Final variable identification Logistic Models Build log regression models for contraceptive use and unmet need using the 2011 NDHS Accuracy assessment for the model at individual level Individual Estimation Apply the coefficients from the log regression models to the 2011 Population Census data to predict probability of contraceptive use and unmet need at individual level Small Area Level Estimation Aggregate induvial probability to estimate CPR and UNR at district level Estimation at sub-regional level and accuracy assessment by comparing to DHS data

15 Variables Selected for SAE of CPR Variable category Variable List Age   Age 15-24 Age 25-34 Age 35-49 Education Level of education is No Education Level of education is Primary Level of education is Some secondary Level of education is SLC above Number of children Number of living children is 0 Number of living children is 1-2 Number of living children is 3 and more Place of residence Place of residence is urban Place of residence is rural Household amenities Household has radio Household has TV Household has motorcycle/scooter Household has bicycle Household has refrigerator

15 Variables Selected for SAE of CPR Variable category Variable List House floor Type of foundation of the house is cement bonded bricks/stone House wall Type of outer wall of the house is cement bonded bricks/stone House roof Type of roof of the house is galvanized sheet/metal Drinking water Main source of drinking water is tap/piped drinking water Electricity Usual source of lighting is electricity (including solar) Toilet Type of toilet is flush toilet (septic tank) Religion   Religion is Hindu Religion is Bouddha Religion is Islam Religion is other than the above four Mother tongue Mother tongue is Nepali Sex of the head of household Sex of head of household is Male Sex of head of household is Female Relation to the head of household Relation to the household head is Head Relation to the household head is Head’s wife Relation to the household head is Head’s daughter Relation to the household head is other than Head’s wife or daughter

Results Assessment: Model Results Classification Table Observed Predicted data Using Contraceptive Method Percentage Correct No Yes (any method) 2931 1513 66% 1243 3381 73% Overall Percentage   70% The classification table is another method to evaluate the predictive accuracy of the logistic regression model. In this table the observed values (the data from DHS), and the predicted values (the estimated data based on model at a user defined cut-off value, usually p=0.50) are cross-classified. The model correctly predicts 70% of the cases. Now the models have enhanced, with overall 70% to 80% accuracy. 

Results Assessment: Regional level Direct & SAE comparison   DHS (2011) Census SAE Value Confidence Intervals Value (2011) R R-2SE R+2SE CPR Any method National Total 49.7 47.6 51.8 48.7 1 Eastern 46.4 42.4 50.3 49.51 2 Central 54.7 59.1 50.65 3 Western 46.1 41.2 51.0 46.75 4 Mid-Western 46.9 42.1 51.6 45.66 5 Far-Western 51.9 46.8 57.0 48.92 CPR Modern method 43.2 41.0 45.3 42.3 36.2 31.8 40.6 43.1 49.9 45.9 54.0 43.9 38.7 33.7 43.7 40.0 42.8 38.4 47.2 39.3 47.1 41.7 52.6 43.8 Unmet need for family planning rate 27.0 -- 25.3 30.0 24.1 21.6 23.0 34.0 29.5 26.1 27.5 24.4 Here you see the aggregates of the results of the model estimation at regional (or provincial) level, and the comparison between the regional level estimation with the regional level DHS data (because DHS regional level data are considered as reliable). Only one region which estimated data does not fit into the confident interval of the DHS data. Now we are in the process of enhancing the model by adding a factor of spatial autocorrection, and the model accuracy has been enhanced.

SAE Results: mPDS at District Level

Identification of Priority Districts: Total Number of Women with Unmet Need and Proportion of Demand Satisfied with modern methods Taking both relative numbers (%) and absolute numbers (total number) into consideration 25 districts with the lowest % of demand satisfied and 25 districts with the highest number of women with unmet need for FP are highlighted using the darkest color

Summary and The Way Forward SAE methodology is an alternative to obtain estimates of key population indicators at lower geographic levels. The application of SAE is helpful to policy and programming, and contributes to SDG monitoring and other agendas. Further combine geographic data disaggregation using SAE with age or other disaggregation, e.g. women aged 15-24, to targeting the locations of the most vulnerable groups. Apply the SAE methodology into other thematic areas such as maternal mortality, antenatal care, skilled birth attendance, etc. Capacity development activities on SAE to enhance the capacity of National Statistical Systems.

Thank you