Measurement Issues Regarding Estimation of Sub-National Health System Efficiency Ajay Tandon Asian Development Bank Development Indicators and Policy Research Division Economics and Research Department September 29 th 2006

Asian Development Bank Sub-National Health System Efficiency Entails measurement of health system outcomes relative to resource inputs at lower administrative tiers (e.g., at district or province level). Entails measurement of health system outcomes relative to resource inputs at lower administrative tiers (e.g., at district or province level). Arguably, of greater local relevance and often more useful than cross-country analyses for policy-makers. Arguably, of greater local relevance and often more useful than cross-country analyses for policy-makers. Poses significant measurement challenges: relevant data often not available sub-nationally. Poses significant measurement challenges: relevant data often not available sub-nationally.

Asian Development Bank Sub-National Data Challenges Sub-nationally representative surveys are expensive to conduct and – even if available – such surveys are not conducted as regularly as may be needed. Sub-nationally representative surveys are expensive to conduct and – even if available – such surveys are not conducted as regularly as may be needed. Relevant sub-national data may only be available from administrative records and this may not be reliable. Relevant sub-national data may only be available from administrative records and this may not be reliable. Data may not be available for pertinent variables. Data may not be available for pertinent variables.

Asian Development Bank Possible Solutions Use of innovative sampling strategies to reduce cost of conducting surveys, e.g., EPI cluster surveys and Lot Quality Assurance Sampling (LQAS). Use of innovative sampling strategies to reduce cost of conducting surveys, e.g., EPI cluster surveys and Lot Quality Assurance Sampling (LQAS). Use of econometric models and Bayesian data combinatorial techniques for estimation purposes. Use of econometric models and Bayesian data combinatorial techniques for estimation purposes. Use factor analytical techniques to compute proxy indexes to capture relevant quantities of interest. Use factor analytical techniques to compute proxy indexes to capture relevant quantities of interest.

Asian Development Bank Econometric Models Quantity of interest (Y) may be available at a national level, but its determinants (Xs) are available both at the national and sub-national level. Quantity of interest (Y) may be available at a national level, but its determinants (Xs) are available both at the national and sub-national level. Y = f (X): model at national level. Predict Y at sub-national level using X. Y = f (X): model at national level. Predict Y at sub-national level using X. Disadvantage: model dependence is high as estimates of Y at the sub-national level will be sensitive to choice of predictors. Disadvantage: model dependence is high as estimates of Y at the sub-national level will be sensitive to choice of predictors.

Asian Development Bank Bayesian Data Combination Techniques Derive priors of quantity of interest using national survey data. Derive priors of quantity of interest using national survey data. Augment priors with micro-samples at the sub- national level (the likelihood). Augment priors with micro-samples at the sub- national level (the likelihood). Combine the priors with the likelihood to estimate posterior estimates at the sub- national level. Combine the priors with the likelihood to estimate posterior estimates at the sub- national level. Borrow strength using prior information. Borrow strength using prior information.

Asian Development Bank Bayesian Data Combination Techniques

Asian Development Bank Proxy Indexes Constraint type VariableIndicator Financial Lack of demand due to insufficient purchasing power; Low levels of public/private spending GDP per capita, PPP Household Lack of demand for health Female literacy Health services delivery Shortage and maldistribution of staff; Weak systems for managing supplies and staff; Poor accessibility DPT3 coverage; Access to health services Environmental characteristics (1) Corruption; Weak governance; Lack of public accountability Government effectiveness index Environmental characteristics (2) Geographical/epidemiological constraints Proportion of population in tropics Source: Ranson, Hanson, Oliveira-Cruz, and Mills (2003)

Asian Development Bank Indonesia Sub-National Application Output index by district: Output index by district: Coverage of complete immunization. Coverage of complete immunization. Skilled birth attendance rate. Skilled birth attendance rate. Iodized salt consumption. Iodized salt consumption. Life expectancy. Life expectancy. Extent of protection from catastrophic spending. Extent of protection from catastrophic spending. Input index by district: Input index by district: Income. Income. Female education. Female education. Nurses per 100,000. Nurses per 100,000. Out-of-pocket health expenditure. Out-of-pocket health expenditure. Access to health facilities. Access to health facilities.

Asian Development Bank Indonesia Sub-National Application

Asian Development Bank Indonesia Sub-National Application

Asian Development Bank Indonesia Sub-National Application