1. Benchmarking of Indian Urban Water Sector: Performance Indicator System versus Data Envelopment Analysis
By: Dr. Mamata Singh, Dr. Atul K. Mittal, and Dr. V. Upadhyay
Presented by:
Dr. V. Upadhyay
Professor
Indian Institute of Technology, Delhi,
Hauz Khas, New Delhi – , India

2. Problems of Water Sector
Exponential population growth
Industrialization and urbanization
Infrastructure inadequacies
Inadequate funds
Inefficient water use

3. Objective of the Paper
To demonstrate the benchmarking approach using DEA and PIS methods
For the selected 12 Indian urban water utilities (municipal bodies) of Maharashtra state/province
Compare and analyze the results

4. Benchmarking Concept
A “benchmark” is a reference or measurement standard used for comparison
“Benchmarking” is the continuous activity of identifying, understanding and adapting best practice and processes that will lead to superior performance

5. Benchmarking Methods
Performance Indicator System - PIS (partial metric method)
Performance Relative to a Model Company (engineering approach –viz. DEA)
Performance Scores based on Production or Cost Estimates (“total” methods)
Process Benchmarking (involving detailed analysis of operating characteristics)
Customer Survey Benchmarking (identifying customer perceptions); and
SWOT (strengths, weaknesses, opportunities and threats) analysis

6. Performance Indicator System (PIS)
PIS comprises a set of performance indicators (PIs) and related data elements which represents the real instances of utilities’ context
Performance indicator (PI) is simply the ratio of an output to an input, or an input to an output
Each PI reflects only one input and one output level, so difficult to view overall performance of a DMU
Need to adopt a summary (or overall) measure of performance values
Methods of aggregating PIs are generally subjective
PIs may be tailored for different objectives and priorities

7. Data Envelopment Analysis (DEA)
Charnes, Cooper and Rhodes (1978) first introduced the term data envelopment analysis (DEA)
CCR considered constant returns to scale (CRS) model with input orientation
DEA is a linear programming technique that incorporates multiple inputs and outputs for assessing relative efficiencies
The most efficient utilities are rated to have an efficiency score of one, while the less efficient utilities score between zero and one
The utilities lying on efficient frontier are identified as best practice utilities by DEA

8. Inputs and Outputs in DEA
Resources utilized by the units or conditions affecting their operation are typical inputs
Measurable benefits generated or service levels of the utility constitute the outputs
Number of DMUs to be considered for DEA should at least be three times the sum total number of inputs and outputs
There should be positive correlation between inputs and outputs
The basic CRS-DEA model with an input orientation has been considered and detailed below

9. Discussion: PIS versus DEA Technique
DEA measures of performance are based on simultaneous consideration of multiple inputs and outputs, while each PI measures performance in relation to one input and one output only
A unit offering unremarkable values on individual PIs can still be deemed a good performer in the context of DEA when its all-round performance is taken into account
DEA is suitable for setting targets which would render a DMU relatively efficient but offers no indication of how relatively efficient DMUs might improve their performance
PIs do give indications as to the specific aspects of performance that a DMU might strengthen
DEA generally reflects overall, and PIs, factor-specific performance

10. Performance Indicators considered for the study

11. Performance Indicator System
Relative Performance Score (RPS) of a DMU = Overall Performance Score of that DMU/ Highest Overall Performance Score
The DMUs are then ranked on the basis of their RPS

12. DEA Input Oriented CRS DEA Model
DEA is performed using free version of DEAF software (by Joe Zhu) to obtain their relative efficiency scores
DMUs are ranked on the basis of their relative efficiency scores

13. Inputs and Outputs for DEA

14. Relative Performance Score and Ranking using PIS

15. PIS Results Wardha has a highest RPS of 1.0 (100%) and is ranked first
Yavatmal is ranked the last with least RPS of 0.34
There are 3 DMUs with RPS < 50% and 4 DMUs with RPS > 75% ≤ 100%
For 5 DMUs RPS varies between 50-75%
Minor difference in overall performance scores of better performing utilities
Wardha and Bhusawal,
Bhusawal and Aurangabad,
Aurangabad and Chandrapur.

16. TE Score and Ranking using DEA

17. DEA Results
4 DMUs Aurangabad, Bhusawal, Parbhani and Wardha have highest TE of 1.0
Chandrapur has higher TE score of 0.850
Yavatmal has the least TE score of and is the only DMU with TE score < 50%
2 DMUs with TE score < 75% and > 100%
5 DMUs with TE score ranging between %

18. Potential for Input Reduction
Maximum for Yavatmal
Wardha, Bhusawal, Chandrapur and Prabhani deliver higher output levels at relatively lower input usage
For most of the DMUs, potential for % reduction in total expenditure and staff size are same (Amravati, Chandrapur, Kolhapur, Nanded Waghala, Solapur and Yavatmal) or closer (Nashik) except for a DMU Dhule
For Dhule, potential for % reduction in staff size is lesser than that of total expenditure as also implied by its PI values

19. Note: Higher is the ranking – lower is the performance level.

20. PIS vs DEA Ranking
Same for 6 UWUs (Amravati, Dhule, Nanded - Waghala, Solapur, Wardha and Yavatmal)
Differs maximum by six positions for the UWU Parbhani
Differs by one position for 2 UWUs (Bhusawal and Chandrapur)
Differ by two to four positions for rest of the 3 UWUs
Rank correlation coefficient is The high correlation further indicates the fact that PIS and DEA methods agree strongly on the UWUs performance rankings
UWUs with lower ranking positions (Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik) under both PIS and DEA methods

21. Conclusion
Better performing UWUs under both PIS and DEA are - Wardha, Aurangabad, Bhusawal and Chandrapur {except Prabhani)
Performance of Yavatmal is found to be lowest in both the methods
UWUs having lower performance levels under both PIS and DEA are - Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik
PIS and DEA results are complimentary
DEA efficient DMU Prabhani may focus on reducing its staff size but not expenditure level as can be observed from PI values
DEA efficient DMU Bhusawal indicates its superior performance as regards staff size but reasonable scope for reduction in total expenditure

22. Recommendation
DEA inefficient DMUs need to focus on reducing their input (total expenditure and staff size) usage altogether
DEA efficient DMUs need to explore the opportunity for reduction in specific input usage (total expenditure or staff size or both) by analyzing PIS outcomes as well
Similar analysis may be performed by the state/independent regulator to devise suitable performance linked incentive mechanism

23. Thank You