1. Sudarno Sumarto The SMERU Research Institute Jakarta - Indonesia www.smeru.or.id Local Monitoring System in Indonesia: With Special Reference to the BKKBN System

2. 2 Background  Efforts to monitor social impacts and development projects in Indonesia have traditionally relied on the national socio-economic survey (SUSENAS ) or village potential census (PODES).  The economic crisis in 1997 forced Indonesia to establish a social safety net program (SSN) for the first time in its history.  SUSENAS and PODES were unable to respond to the quickly-changing crisis environment. By the time survey results came out, they were already obsolete.

3. 3 Background (continued)  In addition, both SUSENAS & PODES can not be used to identify eligible individual beneficiaries.  To meet the need for micro (household) level data, the government turned to BKKBN data.

4. 4 Areas and Major Programs of SSN AreaProgram Description and BenefitsTargetingFY 1998/99FY 1999/00 Food Security OPK program: sale of subsidized rice to targeted households. Eligible households can purchase 10-20 kg of rice at Rp. 1,000/kg (market price is Rp. 2,500 – 3,000/kg) GeographicNone HouseholdBKKBN listBKKBN list with flexibility Community Empowerment PDM-DKE: a ‘community fund’ program that provides block grants directly to villages for either public works or revolving credit funds. GeographicPre-crisis dataUpdated with regional data HouseholdLocal decision Making Local decision making Employment Creation “Padat karya”: a loose, uncoordinated collection of several ‘labor intensive’ programs in various government departments. GeographicNone, various Ministries Urban areas, based on employment HouseholdWeak self selection Self selection EducationScholarships and block grants: providing  Scholarships of Rp. 10,000/month for elementary (SD) students, Rp. 20,000/month for lower secondary (SLTP) students, and Rp. 30,000/month for upper secondary (SMU) students  Block grants to selected schools GeographicOld data on enrollment Poverty data updated to 1998 HouseholdSchool Committees applying criteria School committees applying criteria HealthJPS-BK: a program providing subsidies for  Medical services  Operational support for health centers  Medicine and imported medical equipment  Family planning services  Nutrition (supplementary food)  Midwife services GeographicBKKBN pre- prosperous rates Pre-prosperous rates updated to 1999 HouseholdBKKBN listBKKBN list with flexibility

5. 5 Inventory of Monitoring Efforts during the Crisis  CRP’s NGO Mapping  SMERU’s Kecamatan Crisis Impact Survey  SMERU’s Community-based Monitoring  BPS’ SUSENAS  BPS & Unicef’s 100 Village Survey  RAND’s IFLS2+

6. 6 BKKBN (National Family Planning Coordinating Agency) Background  It was established in 1970 to monitor the FP (Family Planning) program.  The first national FP survey began in 1985 and has been conducted annually ever since.  Family welfare and demographic characteristics sections were added starting in 1994.

7. 7 BKKBN Family Welfare Section  BKKBN maintains a list of every family in the country with an indicator of “welfare” status based on a mix of variables (‘able to eat twice a day’ ‘change of clothing’ ‘able to fulfill religious obligations’).  Overall there are 23 welfare indicators.  The indicators are used to categorize families into 5 welfare status (‘pre-prosperous’ ‘just prosperous’ ‘ prosperous 2’ ‘prosperous 3’ ‘prosperous 3 plus’).

8. 8 BKKBN Family Welfare Section (continued)  In addition to the SSN program, other programs that have used the data from the BKKBN family welfare: Family Savings Program (Takesra)Family Savings Program (Takesra) Family Credit Program (Kukesra)Family Credit Program (Kukesra) National Foster Parents Movement (GN-OTA)National Foster Parents Movement (GN-OTA) Medical assistance for the poorMedical assistance for the poor Housing assistance for the poorHousing assistance for the poor

9. 9 BKKBN Data Gathering System PreparationData CollectionMapping Evaluation ReportingStakeholder meeting Supporting programs Analysis

10. 10 Data Collection & Reporting Process

11. 11 Strengths of BKKBN Data  The only micro data in Indonesia that includes data from almost every household.  As of 2002, the data cover 197.5 million individuals (total population 210 million).  Collected by locals, which ensures its accuracy and comprehensiveness.  In 2000, about 1 million cadre groups conducted the survey, assisted by 35,000 FP supervisors.

12. 12 Weaknesses of BKKBN Data  Even with computerization starting in 2001, mistakes are unavoidable due to the large amount of data.  High variation in the ability of enumerators affects consistency and quality.  Local subjectivity could play a role.  The data do not capture transitory shocks to income.

13. 13 Weaknesses of BKKBN Data (continued)  Questionable subjective welfare criteria.  Composition of data is susceptible to changes by local government officials.  Suffers from lack of funding.

14. 14 How Accurate is BKKBN Data? Comparison with consumption-based measures of household welfare at the district level in 3 provinces Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators Province BKKBN % Pre- prosperous (economic) % Pre- prosperous % Pre-Prosperous + Prosperous 1 (economic) % Pre-Prosperous + Prosperous 1 East Java 0.6932 ** 0.6188 ** 0.7390 **0.7394 ** N = 37 North Sumatra0.44210.4018 0.7579 **0.7491 ** N = 19 South Sulawesi0.3992 0.5326 **0.4279 *0.5958 ** N = 23 Note: ** Significant at 1 percent level * Significant at 5 percent level

15. 15 How Accurate is BKKBN Data? (continued) Comparison with consumption-based measures of household welfare at the household level using the 100 Village Survey (SSD) Cross-tabulation between BKKBN’s Welfare Indicator with SSD’s “Non-Poor/Poor” (20% Poor) SSD Non-PoorPoorTotal BKKBN Welfare Category Prosperous5,1979206,117 % Row8515100 % Column54.238.351 Pre-Prosperous4,4001,4805,880 % Row74.825.2100 % Column45.961.749 Total9,5972,40011,997 % Row8020100 % Column100

16. 16 How Accurate is BKKBN Data? (continued) Cross-tabulation between BKKBN’s Welfare Indicator with SSD’s “Non-Poor/Poor” (49% Poor) SSD Non-PoorPoorTotal BKKBN Welfare Category Prosperous3,5852,5326,117 % Row58.641.4100 % Column58.74351 Pre-Prosperous2,5233,3575,880 % Row42.957.1100 % Column41.35749 Total6,1085,88911,997 % Row5149100 % Column100

17. 17 How Accurate is BKKBN Data? (continued)  The conclusion is that although BKKBN is quite accurate at the district-level, it misses the target at the household-level.  There is quite a large discrepancy between BKBBN welfare classification and welfare measured by consumption at the household level.  Since interventions are aimed at households, this means using BKKBN for such interventions is insufficient.

18. 18 Concluding Remarks  Targeting is not as easy as it is often suggested.  The BKKBN monitoring system was the most widely used targeting tool as it is the most comprehensive CBM system in the country.  At the household level, there is a large degree of discrepancy between BKKBN welfare classification and consumption-based welfare classification.  In order to remedy this discrepancy, a new scoring system or a composite index needs to be explored by combining a number of BKKBN’s 23 welfare variables with other variables.