Performance assessment & benchmarking IBNET as tool for utility management and policy decisionmakers Caroline van den Berg November 24, 2008

What is IBNET? IBNET started as an initiative to collect data on performance of WSS utilities to improve policy dialogue in the early/mid-1990s IBNET started as an initiative to collect data on performance of WSS utilities to improve policy dialogue in the early/mid-1990s Since then, it has developed in three separate products that can be found at Since then, it has developed in three separate products that can be found at

The Elements of IBNET 1. IBNET Toolkit is a suite of software and guidance documents to help utilities compile, analyze and share performance information 2. IBNET website includes a searchable database with indicators from more than 2500 utilities from more than 80+ countries:  Toolkit can be downloaded in different languages  Database with indicator search mechanism & reporting formats  A section with links and resources to assist measurement and benchmarking 3. IBNET helpdesk at

Basic principles of IBNET  Focus on utilities, while also of use for governments, consumer regulators, consumer organizations, donors  Voluntary participation  Easy to implement tool, focusing on a basic set of performance indicators: create a common language between financial and operational staff  Low cost tool  Learning by doing approach  Information on indicators is easily accessible – sharing lessons between utilities

What IBNET can tell you? Understanding utility performance on the basis of a set of objective indicators: Understanding utility performance on the basis of a set of objective indicators: performance of the individual utility over time performance of the individual utility over time performance of the individual utility with other utilities operating in the sector, or similar utilities elsewhere in the world performance of the individual utility with other utilities operating in the sector, or similar utilities elsewhere in the world Analysis to see how the performance of the utility and/or the sector can be improved upon Analysis to see how the performance of the utility and/or the sector can be improved upon Advanced research and comparisons: Advanced research and comparisons: Trends in utility performance (benchmarking) Trends in utility performance (benchmarking) Development of new indicators that reflect the specific utility needs/sector needs Development of new indicators that reflect the specific utility needs/sector needs Econometric work -- frontier and productivity analysis Econometric work -- frontier and productivity analysis

Utility managers and IBNET data Common language for technical and financial staff Common language for technical and financial staff Performance assessment: Performance assessment: Set-up baseline: where are we now?: detecting weaknesses Set-up baseline: where are we now?: detecting weaknesses Set-up performance objectives: where we want to go?: setting priorities and targets Set-up performance objectives: where we want to go?: setting priorities and targets Performance monitoring: are we getting there? Performance monitoring: are we getting there? Advocacy – detect what impedes progress and what has to be done to overcome the impediments Advocacy – detect what impedes progress and what has to be done to overcome the impediments Comparisons between utilities, best practices, tool for analysis Comparisons between utilities, best practices, tool for analysis

Basic Analytical Tools in IBNET

Utility Scorecard

Benchmarking utilities: Operational Cost Coverage Ratio – multi-city, large utilities

Country scorecard

Sector Scorecard

Examples of Other Analyses: The Case of Moldova

Impact on efficiency improvements: convergence Staff Ratio

Impact on efficiency improvements: convergence (2) Collection Ratio

Impact on efficiency improvements: convergence (3) Non Revenue Water m3/conn/day

Impact on efficiency improvements Two impacts: Two impacts: Efficiency levels: trend is going in the right direction; Efficiency levels: trend is going in the right direction; Convergence of performance: the “bad” performing utilities get better Convergence of performance: the “bad” performing utilities get better Benchmarking as tool for yardstick competition pays off Benchmarking as tool for yardstick competition pays off

Impact on policy development Sector reform has been pushed supported by benchmarking The efficiency of the sector has increased as implicit subsidies have declined significantly over time

Examples of Other Analyses: The Impact of the Triple Crisis

Impact of the Triple Crisis O&M Cost per m3 water sold

Demand for IBNET Even though no major dissemination has taken place so far, with the push for dissemination, access to website has grown significantly from 7,000 hits in Sep05 to about 80,000 in Oct08 Even though no major dissemination has taken place so far, with the push for dissemination, access to website has grown significantly from 7,000 hits in Sep05 to about 80,000 in Oct08 Use of IBNET data in projects, reports and other publications in-and outside the Bank Use of IBNET data in projects, reports and other publications in-and outside the Bank Use of IBNET by countries to improve their benchmarking Use of IBNET by countries to improve their benchmarking Standard tool for assessment of utility performance in Moldova, and Latin America’s ADERASA Standard tool for assessment of utility performance in Moldova, and Latin America’s ADERASA Use of IBNET by countries to improve their benchmarking Use of IBNET by countries to improve their benchmarking Working with many different partners – both national and regional partners Working with many different partners – both national and regional partners

Issues in IBNET Data collection issues: Data collection issues: Use of standardized data definitions and data tools to ensure comparability of data within and between countries Use of standardized data definitions and data tools to ensure comparability of data within and between countries Looking for national partners that can collect data and ensure future data collection efforts: the benefits of institutionalization Looking for national partners that can collect data and ensure future data collection efforts: the benefits of institutionalization Data quality requires a lot of attention: Data quality requires a lot of attention: Capacity building with data collecting agencies: process takes time Capacity building with data collecting agencies: process takes time Control mechanisms to check on inconsistencies in the collected data Control mechanisms to check on inconsistencies in the collected data Need for follow-up on data collection with analysis of collected data and actual assistance to utilities wherever is needed Need for follow-up on data collection with analysis of collected data and actual assistance to utilities wherever is needed

Conclusions By providing access to comparative information key stakeholders will get the information to do their jobs better : By providing access to comparative information key stakeholders will get the information to do their jobs better : Utility managers and employees can identify areas for improvement, adopt realistic targets and—not least—convince authorities of the need for change; Utility managers and employees can identify areas for improvement, adopt realistic targets and—not least—convince authorities of the need for change; Governments can monitor and adjust sector policies and programs; Governments can monitor and adjust sector policies and programs; Regulators can ensure that customers get value, and providers have incentives to perform; Regulators can ensure that customers get value, and providers have incentives to perform; Customer groups and NGOs can exercise “voice” in an informed way; Customer groups and NGOs can exercise “voice” in an informed way;

Conclusions (2) IBNET: IBNET: Use of IBNET as a tool with a tested methodology with adjustments and expansions Use of IBNET as a tool with a tested methodology with adjustments and expansions Sharing data to ensure that learning and communication can take place Sharing data to ensure that learning and communication can take place We hope to ensure your participation in IBNET – directly or indirectly so that this tool can help to improve the sector to provide better and more universal access to its services We hope to ensure your participation in IBNET – directly or indirectly so that this tool can help to improve the sector to provide better and more universal access to its services Collaboration with WOP – we hope that WOP can move the agenda forward with focusing on Collaboration with WOP – we hope that WOP can move the agenda forward with focusing on 1. further analysis with secondary data links; and 2. create the link with performance improvements

THANK YOU For more information, please contact: Caroline van den Berg Alexandar Danilenko