1. WORKSHOP FOUR MAKE DATA MEANINGFUL: MAKE THE INVISIBLE, VISIBLE
Small Area Estimates: an Illustration for Family Planning using Nepal as a Case Study I
Sabrina Juran Ph.D.
Technical Specialist, Data and Population Analysis
United Nations Population Fund

2. Family Planning Indicator (SDG3.7.1)
SDG Indicator 3.7.1: Percentage of women of reproductive age (15-49 Years) who have their need for family planning satisfied with modern methods
mCPR
CPR + Unmet need
X100
SDG3.7.1 =
Contraceptive prevalence rate (CPR) - percentage of women who are
currently using, or whose sexual partner is currently using, at least one method of
contraception, regardless of the method used.
mCPR - contraceptive prevalence rate using modern methods
Unmet need - percentage of women of reproductive age, either married or in
union, who want to stop or delay childbearing but are not using any method of
contraception.
SDG 3 Health
Target 3.7 to ensure universal access to SRH services
One indicator to measure prigress in SDG 3.7.1
Agency working on SRH – we are responsible for global monitoring and supporting countries in their efforts to report on this target.
More importantl however, as an operational agency with programmes in more than 150 countties we need to make sure that our interventions are most eeffeciemtyl targeted.
If we want to know where to invest in terms of awareness raising, sexual education, service provision, we need to know the actual situation of family planning on the ground.
Primary data source for FP data are DHS and MICS.

3. Disaggregation Challenges
General data disaggregation
All data sources provide possibilities for data disaggregation but all have limitations
Geographic data disaggregation
Most of the data used for analysis of contraceptive dynamics in developing countries is collected via household surveys.
Sample limitations allow disaggregation only to regional level, at best provincial/state level
General data disaggregation challenges
All data sources (administrative records, CRVS, census, surveys, etc.) provide possibilities for data disaggregation but all have limitations (completeness, periodicity, access, sampling, etc.)
Geographic data disaggregation challenges
Most of the data used for contraceptive dynamics analysis in the developing world is collected via household surveys (DHS and MICS). Sample limitations allow disaggregation only to regional level, at best provincial/state level

4. Family Planning Status in Nepal
By year:
By Age Groups

5. Nepal 5 Regions; 15 Sub-regions; 75 Districts
2011 Nepal Demographic and Health Survey (DHS) represent for 13 sub-regions
(these 3 sub-region in DHS are aggregated as
“western mountain”)

6. Sub-regional direct estimates of contraceptive prevalence rate using DHS 2011
Far-western, mid-western and western mountain apply the same data analysis results

7. Geographic disaggregation problems
Number
Min Individual
MAX Individual
MEAN Individual
Total Individual
Clusters
289 3
153 42
12,023
Kathmandu
DHS: Data disaggregation below the sub-regional or provincial level involves substantial level of uncertainty and thus not recommended
Census: Do not have family planning indicators

8. Small Area Estimates (SAE) as an Alternative
Using SAE to produce estimates of Family Planning (FP) indicators for each one of the 75 districts in Nepal
The SAE technique takes advantage of the existing correlation between FP indicators and a set of common variables in the 2011DHS and the 2011 Population Census Public Use Microdata Sample data (age, number of children, urban/rural residence, education, water and sanitation, etc.) to predict values for contraceptive dynamics indicators at the district level using regression models.

9. Key Steps of SAE and Application
Small Area Estimation (SAE)
Data Analysis and Assessment
Regression model coefficients for predicting probability of using contraception for an individual woman are obtained from the 2011 NDHS data
The national coefficients from DHS are applied to the 2011 Census Data to predict the probability of using contraception for individual women
The individual contraceptive use probabilities from census data are aggregated (using average) to district level
Application of SAE
Estimate the number of women aged (married or in union) in need of contraception
Identify the priority districts
Major Steps:
1. Identify common variables associated with contraceptive indicators in DHS and census data
DHS
CENSUS
2. Develop a model for predicting the probability of individual contraceptive use using DHS data
3. Apply the model to census and estimate the probability of individual contraceptive use
4. Aggregate the estimation of individual contraceptive use to small area administrative level and map results
Variable 1
Variable 2
Variable 3 ……
Data Analysis and Assessment
Identify a set of common variables associated with contraceptive dynamics indicators in 2011 NDHS and 2011 Population Census.
2011 NDHS and 2011 Population Census data assessment and comparison
Final variable identification
Logistic Models
Build log regression models for contraceptive use and unmet need using the 2011 NDHS
Accuracy assessment for the model at individual level
Individual Estimation
Apply the coefficients from the log regression models to the 2011 Population Census data to predict probability of contraceptive use and unmet need at individual level
Small Area Level Estimation
Aggregate induvial probability to estimate CPR and UNR at district level
Estimation at sub-regional level and accuracy assessment by comparing to DHS data

10. 15 Variables Selected for SAE of CPR
Variable category
Variable List
Age
Age 15-24
Age 25-34
Age 35-49
Education
Level of education is No Education
Level of education is Primary
Level of education is Some secondary
Level of education is SLC above
Number of children
Number of living children is 0
Number of living children is 1-2
Number of living children is 3 and more
Place of residence
Place of residence is urban
Place of residence is rural
Household amenities
Household has radio
Household has TV
Household has motorcycle/scooter
Household has bicycle
Household has refrigerator

11. 15 Variables Selected for SAE of CPR
Variable category
Variable List
House floor
Type of foundation of the house is cement bonded bricks/stone
House wall
Type of outer wall of the house is cement bonded bricks/stone
House roof
Type of roof of the house is galvanized sheet/metal
Drinking water
Main source of drinking water is tap/piped drinking water
Electricity
Usual source of lighting is electricity (including solar)
Toilet
Type of toilet is flush toilet (septic tank)
Religion
Religion is Hindu
Religion is Bouddha
Religion is Islam
Religion is other than the above four
Mother tongue
Mother tongue is Nepali
Sex of the head of household
Sex of head of household is Male
Sex of head of household is Female
Relation to the head of household
Relation to the household head is Head
Relation to the household head is Head’s wife
Relation to the household head is Head’s daughter
Relation to the household head is other than Head’s wife or daughter

12. Results Assessment: Model Results
Classification Table
Observed
Predicted data
Using Contraceptive Method
Percentage Correct No
Yes (any method)
2931
1513
66%
1243
3381
73%
Overall Percentage
70%
The classification table is another method to evaluate the predictive accuracy of the logistic regression model.
In this table the observed values (the data from DHS), and the predicted values (the estimated data based on model at a user defined cut-off value, usually p=0.50) are cross-classified.
The model correctly predicts 70% of the cases.
Now the models have enhanced, with overall 70% to 80% accuracy. 

13. Results Assessment: Regional level Direct & SAE comparison
DHS (2011)
Census SAE
Value
Confidence Intervals
Value (2011) R
R-2SE
R+2SE
CPR Any method
National Total
49.7
47.6
51.8
48.7 1
Eastern
46.4
42.4
50.3
49.51 2
Central
54.7
59.1
50.65 3
Western
46.1
41.2
51.0
46.75 4
Mid-Western
46.9
42.1
51.6
45.66 5
Far-Western
51.9
46.8
57.0
48.92
CPR Modern method
43.2
41.0
45.3
42.3
36.2
31.8
40.6
43.1
49.9
45.9
54.0
43.9
38.7
33.7
43.7
40.0
42.8
38.4
47.2
39.3
47.1
41.7
52.6
43.8
Unmet need for family planning rate
27.0 --
25.3
30.0
24.1
21.6
23.0
34.0
29.5
26.1
27.5
24.4
Here you see the aggregates of the results of the model estimation at regional (or provincial) level, and the comparison between the regional level estimation with the regional level DHS data (because DHS regional level data are considered as reliable).
Only one region which estimated data does not fit into the confident interval of the DHS data.
Now we are in the process of enhancing the model by adding a factor of spatial autocorrection, and the model accuracy has been enhanced.

14. SAE Results: mPDS at District Level

15. Identification of Priority Districts: Total Number of Women with Unmet Need and Proportion of Demand Satisfied with modern methods
Taking both relative numbers (%) and absolute numbers (total number) into consideration
25 districts with the lowest % of demand satisfied and 25 districts with the highest number of women with unmet need for FP are highlighted using the darkest color

16. Summary and The Way Forward
SAE methodology is an alternative to obtain estimates of key population indicators at lower geographic levels.
The application of SAE is helpful to policy and programming, and contributes to SDG monitoring and other agendas.
Further combine geographic data disaggregation using SAE with age or other disaggregation, e.g. women aged 15-24, to targeting the locations of the most vulnerable groups.
Apply the SAE methodology into other thematic areas such as maternal mortality, antenatal care, skilled birth attendance, etc.
Capacity development activities on SAE to enhance the capacity of National Statistical Systems.

17. Thank you