Benchmarking of Indian Urban Water Sector: Performance Indicator System versus Data Envelopment Analysis By: Dr. Mamata Singh, Dr. Atul K. Mittal, and Dr. V. Upadhyay Presented by: Dr. V. Upadhyay Professor Indian Institute of Technology, Delhi, Hauz Khas, New Delhi – 110016, India

Problems of Water Sector Exponential population growth Industrialization and urbanization Infrastructure inadequacies Inadequate funds Inefficient water use

Objective of the Paper To demonstrate the benchmarking approach using DEA and PIS methods For the selected 12 Indian urban water utilities (municipal bodies) of Maharashtra state/province Compare and analyze the results

Benchmarking Concept A “benchmark” is a reference or measurement standard used for comparison “Benchmarking” is the continuous activity of identifying, understanding and adapting best practice and processes that will lead to superior performance

Benchmarking Methods Performance Indicator System - PIS (partial metric method) Performance Relative to a Model Company (engineering approach –viz. DEA) Performance Scores based on Production or Cost Estimates (“total” methods) Process Benchmarking (involving detailed analysis of operating characteristics) Customer Survey Benchmarking (identifying customer perceptions); and SWOT (strengths, weaknesses, opportunities and threats) analysis

Performance Indicator System (PIS) PIS comprises a set of performance indicators (PIs) and related data elements which represents the real instances of utilities’ context Performance indicator (PI) is simply the ratio of an output to an input, or an input to an output Each PI reflects only one input and one output level, so difficult to view overall performance of a DMU Need to adopt a summary (or overall) measure of performance values Methods of aggregating PIs are generally subjective PIs may be tailored for different objectives and priorities

Data Envelopment Analysis (DEA) Charnes, Cooper and Rhodes (1978) first introduced the term data envelopment analysis (DEA) CCR considered constant returns to scale (CRS) model with input orientation DEA is a linear programming technique that incorporates multiple inputs and outputs for assessing relative efficiencies The most efficient utilities are rated to have an efficiency score of one, while the less efficient utilities score between zero and one The utilities lying on efficient frontier are identified as best practice utilities by DEA

Inputs and Outputs in DEA Resources utilized by the units or conditions affecting their operation are typical inputs Measurable benefits generated or service levels of the utility constitute the outputs Number of DMUs to be considered for DEA should at least be three times the sum total number of inputs and outputs There should be positive correlation between inputs and outputs The basic CRS-DEA model with an input orientation has been considered and detailed below

Discussion: PIS versus DEA Technique DEA measures of performance are based on simultaneous consideration of multiple inputs and outputs, while each PI measures performance in relation to one input and one output only A unit offering unremarkable values on individual PIs can still be deemed a good performer in the context of DEA when its all-round performance is taken into account DEA is suitable for setting targets which would render a DMU relatively efficient but offers no indication of how relatively efficient DMUs might improve their performance PIs do give indications as to the specific aspects of performance that a DMU might strengthen DEA generally reflects overall, and PIs, factor-specific performance

Performance Indicators considered for the study

Performance Indicator System Relative Performance Score (RPS) of a DMU = Overall Performance Score of that DMU/ Highest Overall Performance Score The DMUs are then ranked on the basis of their RPS

DEA Input Oriented CRS DEA Model DEA is performed using free version of DEAF software (by Joe Zhu) to obtain their relative efficiency scores DMUs are ranked on the basis of their relative efficiency scores

Inputs and Outputs for DEA

Relative Performance Score and Ranking using PIS

PIS Results Wardha has a highest RPS of 1.0 (100%) and is ranked first Yavatmal is ranked the last with least RPS of 0.34 There are 3 DMUs with RPS < 50% and 4 DMUs with RPS > 75% ≤ 100% For 5 DMUs RPS varies between 50-75% Minor difference in overall performance scores of better performing utilities Wardha and Bhusawal, Bhusawal and Aurangabad, Aurangabad and Chandrapur.

TE Score and Ranking using DEA

DEA Results 4 DMUs Aurangabad, Bhusawal, Parbhani and Wardha have highest TE of 1.0 Chandrapur has higher TE score of 0.850 Yavatmal has the least TE score of 0.487 and is the only DMU with TE score < 50% 2 DMUs with TE score < 75% and > 100% 5 DMUs with TE score ranging between 50 - 75%

Potential for Input Reduction Maximum for Yavatmal Wardha, Bhusawal, Chandrapur and Prabhani deliver higher output levels at relatively lower input usage For most of the DMUs, potential for % reduction in total expenditure and staff size are same (Amravati, Chandrapur, Kolhapur, Nanded Waghala, Solapur and Yavatmal) or closer (Nashik) except for a DMU Dhule For Dhule, potential for % reduction in staff size is lesser than that of total expenditure as also implied by its PI values

Note: Higher is the ranking – lower is the performance level.

PIS vs DEA Ranking Same for 6 UWUs (Amravati, Dhule, Nanded - Waghala, Solapur, Wardha and Yavatmal) Differs maximum by six positions for the UWU Parbhani Differs by one position for 2 UWUs (Bhusawal and Chandrapur) Differ by two to four positions for rest of the 3 UWUs Rank correlation coefficient is 0.833. The high correlation further indicates the fact that PIS and DEA methods agree strongly on the UWUs performance rankings UWUs with lower ranking positions (Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik) under both PIS and DEA methods

Conclusion Better performing UWUs under both PIS and DEA are - Wardha, Aurangabad, Bhusawal and Chandrapur {except Prabhani) Performance of Yavatmal is found to be lowest in both the methods UWUs having lower performance levels under both PIS and DEA are - Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik PIS and DEA results are complimentary DEA efficient DMU Prabhani may focus on reducing its staff size but not expenditure level as can be observed from PI values DEA efficient DMU Bhusawal indicates its superior performance as regards staff size but reasonable scope for reduction in total expenditure

Recommendation DEA inefficient DMUs need to focus on reducing their input (total expenditure and staff size) usage altogether DEA efficient DMUs need to explore the opportunity for reduction in specific input usage (total expenditure or staff size or both) by analyzing PIS outcomes as well Similar analysis may be performed by the state/independent regulator to devise suitable performance linked incentive mechanism

Thank You