1. Nobuo Yoshida
SWIFT program lead and Lead Economist, World Bank
July 19, 2017

2. ?
What is SWIFT?

3. What is SWIFT
SWIFT (Survey of Well-being via Instant and Frequent Tracking) is a new household survey instrument
SWIFT collects non-consumption data (X)
SWIFT is designed for producing welfare indicators from X in a cost- effective, timely, and user-friendly manner
SWIFT can be used for
Increasing frequency of poverty data
Estimating poverty for a specific area or group of people
Monitoring the welfare effects of government’s investment or program

4. How does SWIFT works (1) SWIFT Survey
SWIFT survey does not collect consumption or income directly
SWIFT survey includes 15 to 20 simple questions (X) that are selected by models developed from LSMS
We collect this SWIFT survey! X

5. Collect data (X) by a SWIFT survey
How does SWIFT work? (2)
LSMS
SWIFT survey C X
Ĉ=F(X) X
Develop projection model (regress C on X)
Impute Ĉ
Identify X
C=F(X)
Collect data (X) by a SWIFT survey
C: Consumption
X: Household variables (e.g. education, employment) Ĉ=F(X): Projected Consumption data

6. Is SWIFT reliable?

7. SWIFT Guideline Version 2.0
SWIFT Guideline V 2.0 includes two main parts:
Modeling and simulations
Data collection (sampling, logistics, questionnaire design, and training)

8. Modeling Create a model by running regressions
𝑙𝑛𝑦 ℎ =𝛼+ 𝛽 1 ∗ 𝑥 1ℎ + 𝛽 2 ∗ 𝑥 2ℎ +…+ 𝛽 𝑘 ∗ 𝑥 𝑘ℎ + 𝜀 ℎ
Left hand side variable: log of household expenditure per capita (or per adult equivalence)
Select right hand side variables from the group of poverty correlates and estimate the coefficients using “Stepwise” regression
Estimate distributions of the coefficients and errors

9. Stepwise model selection
Mechanically look for variables that are statistically significant
Need to look for the level of significance
Usually, 5 percent
But, SWIFT selects an optimal level of significance

10. Simulation stage
Simulate HH expenditure for each household in SWIFT Survey
Randomly drawing coefficients ( ) and errors ( ) from the estimated distributions
Simulation is repeated times
Compute poverty headcount rates using the simulated HH expenditures for each round
Average poverty rates as poverty rates
Standard errors of the poverty rates are estimated from the distribution of poverty rates
Use software
𝑙𝑛𝑦 ℎ =𝛼+ 𝛽 1 ∗ 𝑥 1ℎ + 𝛽 2 ∗ 𝑥 2ℎ +…+ 𝛽 𝑘 ∗ 𝑥 𝑘ℎ + 𝜀 ℎ
Repeat this simulation process 100 times
Identify poor households using the simulated HH exp for each round
Calculate poverty headcount rates for each round and use means as the poverty estimates
Use the standard deviation of 100 poverty headcount rates as standard errors of poverty estimates

11. Issues Over-fitting Multi-collinearity
A model performs very well in LSMS but might not outside
Multi-collinearity
Stepwise regression is vulnerable to multi-collinearity
Stability of coefficients over time
Models developed in LSMS might not be no longer valid
Misspecification of error structure
Error distributions can be very complex
Estimation of standard errors
Formula of poverty mapping is not accurate

12. A model might not be stable over time
2010
2015
LSMS
SWIFT
LSMS
LSMS (Modeling)
SWIFT
SWIFT (Simulations)

13. A within-sample test might not be reliable
LSMS 2012/13
LSMS 2012/13
𝐶,𝑋
𝐶 ,𝑋
C=f(X)
𝐶 =f(X)
A dataset used for developing a model is the same as a dataset which the model is applied for
If an over-fitting problem exists, the within-sample performance can be very different from the out-of-sample performance

14. Distribution might not be normal
Actual
Simulated with normal distribution
Log of household expenditure per capita
Note: data from Sri Lanka HIES 2012/13, Central province.

15. To improve SWIFT modeling & simulations
Cross Validation
Multicollinearity checks
Stability test using Backward/Forward Imputation
Addressing distributional issues with a combination of PovMap and MI

16. 1. Cross-Validation
Cross-Validation is used to see the out-of-sample performance rather than within-sample performance
The risk of overfitting problem rises as more variables are included
Using the cross-validation approach, we try to find the optimal number of variables
To ease programing, we search the optimal p-value for the stepwise regression

17. Cross-Validation: Step 1
Randomly Split by three
GLSS 2012/13
𝐶,𝑋
𝐶 1 , 𝑋 1
𝐶 2 , 𝑋 2
𝐶 3 , 𝑋 3

18. Cross-Validation: Step 2
Randomly Split by three
Training Data
Testing Data
GLSS 2012/13
𝐶,𝑋
𝐶 1 , 𝑋 1
𝐶 2 , 𝑋 2
𝐶 3 , 𝑋 3
modeling
Compare
𝐶 =𝑓( 𝑋 3 )
𝐶=𝑓 𝑋 𝑖

19. Cross-Validation: Step 3
𝐶 1 , 𝑋 1
𝐶 3 , 𝑋 3
𝐶 2 , 𝑋 2
𝐶 1 , 𝑋 1
Training Data
𝐶 3 , 𝑋 3
Testing Data

20. Cross-Validation: Statistics of interest
Mean Squared Errors
1 𝑁 𝑖=1 𝑁 ( 𝑌 𝑖 − 𝑌 𝑖 ) 2
Average Squared (or absolute) difference between actual and projected poverty rates
𝐻 1 − 𝐻 𝐻 2 − 𝐻 𝐻 3 − 𝐻

21. Cross-Validation: Select the best p-value for stepwise regression
P-value = 0.06 is the best
Average of absolute differences between actual and projected poverty rates
Mean Squared Errors

22. 2. Are signs of coefficients reasonable?
Variables
2010 Rural model
Coef.
Std. Err.
Intercept
16.87
0.06
Household size
-0.22
0.02
Household size 2
0.01
0.00
Dependency ratio
-0.77
0.16
Dependency ratio 2
0.52
0.17
Head: Male
0.10
0.03
Head: Grades enrolled 2
Cooking: coal/wood
0.21
Own: Car
0.32
0.09
Own: TV
Own: Vent
0.12
0.04
Me-Zochi dist.
0.15
Cantagalo dist.
0.07

23. 3. Backward/Forward Imputation to Test Stability of Models
2012/13
2005/6
2006/7
2007/8
2008/9
2009/10
2010/11
2011/12
C,X
C,X
Compare
Modeling
Ĉ=f(X)
C=f(x)
Simulation
C,X
Consumption and non-consumption data collected by LSMS
Projected consumption data in LSMS05/06
Ĉ=f(X)

24. Example of Backward/Forward Imputation from Afghanistan analysis
i) Backward/Forward imputation
ii) Final Estimation for 2013/14
Poverty Rate
Survey year
95% Confidence Interval
Actual
36.3
35.8
[34.94, 37.60]
[34.14, 37.40]
Imputed
37.2
35.2
39.1
[35.75, 38.63]
[33.56, 36.78]
[37.71, 40.55]

25. 4. Estimation of poverty rates
There are two major simulation approaches
Poverty Mapping Method or ELL
Developed by Elbers, Lanjouw and Lanjouw (2003) (ELL)
It is often called ELL
Multiple Imputation Method (MI)
Developed by Rubin (1987) and Harvard Univ.
It is often called MI

26. Comparisons between ELL and MI
ELL: Good for incorporating a complex error structure of regression models
MI: Easy to incorporate sampling errors and produce an accurate estimation of standard errors of poverty estimates
Ideally, we should combine these two methodologies
First, estimate models and simulate household expenditures with ELL
Then, estimate poverty statistics and the standard errors with MI

27. Rubin-Schafer’s formula
For a scalar population parameter Q:
𝑄 = 1 𝑚 𝑗=1 𝑚 𝑄 𝑗
𝑉𝑎𝑟 𝑄 = 1+ 1 𝑚 𝐵+ 𝑈
where B is the between-imputation variance
𝐵= 1 𝑚−1 𝑗=1 𝑚 ( 𝑄 𝑗 − 𝑄 ) 2
and 𝑈 is the average of within-imputation variances
𝑈 = 1 𝑚 𝑗=1 𝑚 𝑈 𝑗

28. Simulations with a more flexible distribution using PovMap
Actual
Simulated by PovMap
Log of household expenditure per capita
Note: data from Sri Lanka HIES 2012/13, Central province.

29. Steps for SWIFT modeling & simulation
Cross-validation to decide the optimal p-value
Run the stepwise regression using the optimal p-value determined by the cross-validation
Check the coefficients of the final model
Simulate household expenditures using PovMap or MI
Estimate poverty rates using the simulated expenditures and MI’s formula using “mi estimate”
To check stability, conduct backward imputation analysis

30. Review process of SWIFT
Selected as one of the Innovation Challenge Program in 2013
SWIFT Guideline V 1.0
Concept Note Review Meeting in July 2014
Decision Meeting in June 2015
SWIFT Guideline V 2.0
Decision Note
Decision Meeting for FY16
Papers
Serbia telephone data experimentation (soon PRWP)
Bangladesh High frequency poverty data collection (PRWP)
Sri Lanka Survey to Survey Imputation (PRWP)
Ethiopia Survey to Survey imputation (seeking permission from the govt)
Afghanistan Survey to Survey imputation (soon PRWP)
A note on Paraguay SWIFT Water survey (published in CPF)
A paper on Bangladesh Energy-SWIFT Survey (under preparation)

31. Research SWIFT
Econometrics of Projections (Y) rather than Econometrics of structural estimation (beta)
Explore LASSO and other estimators
Effects of the number of simulations
The impact of increasing the number of simulations on the selection of the optimal p
SWIFT V 2.0 for Iraq
Collection of consumption data as well

32. Advantages of SWIFT

33. Estimation of income growth
Advantages of SWIFT
Creation of Quintiles
Estimation of poverty
Estimation of income growth
Equity Tool
Yes No
PPI
SWIFT

34. Comparison of different methodologies (Sharma et al. 2014)

35. Improving marketing with partners

36. Training for SWIFT We have training for SWIFT
It includes introduction and theory of SWIFT (half day)
Hands-on training with a demo data (1.5 days)
Hands-on training with your data (3 days)
A new online course is now available!

37. Training for SWIFT