1. WOPs: how to make them even better? An approach to inform about WOPs’ performance M.Pascual, S.Veenstra, U.Wehn, R.van Tulder, G. Alaerts 30th May 2013

2. What are WOPs? WOPs encompass a great range of collaboration approaches, each able to foster different type of results Source: IWA & UNHABITAT., 2009 “Any form of simple or structured partnership aimed at capacity building on a not-for-profit basis” (IWA, UN-HABITAT and VEI,2009) "(...) a structured programme of cooperation among water operators, based on mutual support and on a not-for-profit basis (UN-HABITAT, 2007). “Any form of simple or structured partnership aimed at capacity building on a not-for-profit basis” (IWA, UN-HABITAT and VEI,2009) "(...) a structured programme of cooperation among water operators, based on mutual support and on a not-for-profit basis (UN-HABITAT, 2007). Minimum of P-WOPs Performance based non-delegation driven partnerships P-WOPs Performance based non-delegation driven partnerships

3. Key elements for WOPs’ success WOP evaluation studies converge in the following recommendations for WOPs to be effective mechanisms for change – Long term collaboration The transformation of new knowledge into organisational change is a slow process and requires time. – Strong engagement from local management and sector stakeholders The change process has to be led by the local organisation – Funds for investments KT is not enough to enable change. Limited operational improvements are feasible if no funds for investments are available to scale up small steps into change. – Flexibility to emerging circumstances WOPs as a CD intervention require flexibility to adapt to the unpredictable evolution of the change process in order to ensure relevance and effectiveness – High relational quality between partners Particularly in WOPs, where there is no delegation of decision making power to the external partner, change is socially constructed by partners interaction. – Report on achievements of WOPs during implementation of the project In order to gain and keep confidence of partners and stakeholders to keep supporting the process

4. State of the art on reporting performance of WOPs Change in KPIs are rarely achieved by WOPs except for long term structured comprehensive WOPs associated to investment programs – Yet, in those cases, KPI improvements take years to be achieved Even in cases of delegated management through PPPs KPI improvements take at least 4-5 years to be achieved. I.e. Senegal PPP,10 years or Mali, 5. – KPIs are insensitive to improvements demonstrated in pilot areas and to nascent changes like strengthened capacity and improved processes, typical outputs of first years of WOPs implementation Hence, other measures are commonly used to report WOPs achievements. However, results reported are : – Highly heterogeneous, which hinder comparative analysis, learning and dissemination – The dominant approach is to report on the basis of Inputs: – No and type of trainings – No of exposure visits, etc., and not so much on the basis of outputs: – No of employees that gained specific knowledge – Degree of change in working routines, etc. WOPs have been criticised for being an accepted mechanism based on their solidarity nature despite the lack of empirical evidence of their effectiveness (Boag & McDonald, 2010) There is need for an innovative standardised system to report performance of WOPs

5. Multi-path approach to inform about WOPs’ performance* Project inputs consolidation into change Degree of satisfaction KPIs changes Partners’ relational capital Partners’ interaction Partners’ rolesDominant governing mechanismsConflict management Satisfaction with the project Selected KPIs of the targeted water operator Most valuable aspects Capacity changes Specific KPI- related capacity Plausible attribution to the partnership - + Time Other effects of the projectSustainability of the change trendProject activities-related knowledge consolidation into change * performance of a WOP different from performance of water operator WOP performance is measured through the ability of the partnership to effectively support the local operator in achieving ‘progress towards impact’

6. Cases in which the multi-path approach to WOP performance 2 parallel projects Same type of contract: 4-years service contract 2009-2013 – no delegation – VEI and Lilongwe Water Board (LWB) – VEI and Blantyre Water Board (BWB) 2 main components: – (i) CD and operational support – (ii) support in the supervision of the implementation of an investment program VEI tasks – 1. Organisational structure and staffing – 2. Financial and commercial management – 3.Management information system – 4. Reduction of NRW – 5. Operation and maintenance plans – 6. Optimising pumping regimes – 7. Extension of water supply services to LIAs – 8. Access to water facility – 9. Public Relations works – 10. HIV/AIDS programme – 11. Implementation of investments programmes for improvement of production capacity, reduction of NRW and extension of services to LIA KPIs targets – NRW, Working Ratio, No kiosks built, Production capacity (only in BWB) Information collected after 2 years of implementation of the project

7. KPI achievements in each project KPIs targets were not met for NRW. Only No of kiosks built in BWB and Working Ratio in LWB were met, however, partners argued the limited impact of VEI inputs in the achievement of those 2 KPIs.

8. Change in capacity to reduce NRW and plausible attribution BWBLWB Highest capacity change Organisational structure, availability of resourcesStrategic management, resources and capability to reduce leakages and overflows in storage tanks Project strongest contribution Change of organisational structure, organisational motivation, resources, capability to reduce leakages, and learning capability Organisational motivation, resources, capability to reduce leakage and learning capability

9. Project inputs consolidation into change Szulanski (2000) defines KT as – “ an unfolding process comprised of several stages that start with the opportunity identification for knowledge transfer and ends once the recipient unit is able to maintain satisfactory performance derived from the integration of knowledge into working routines”

10. Project inputs consolidation into change - Project tasks-related knowledge translation into change-

11. Project inputs consolidation into change - Other effects from the project - BWBLWB CEO A force supporting organisational change via: -Reducing internal resistance to change; - Limiting political interference; - Increasing accountability for results; A more accurate picture of the current status of LWB WBs’ staff -Motivation to improve and engagement of staff; -Willingness to learn and improvement of staff attitude -Sense of urgency to improve; -Commitment of the different sections with a role in NRW reduction; -Improvement of management and coordination skills -Stronger commitment to reduce NRW throughout the organisation; -Higher sense of urgency to improve

12. Project inputs consolidation into change - Degree of sustainability of perceived change - – BWB members showed mixed opinions while LWB staff argued that if the project stopped the urge to improve would not be there any more. – The conditions identified to sustain the change trend are mostly shared by both WBs: BWBLWB - Strong support from BWB management to the project and increase efforts to complete implementation of improvement plans - Capacity to retain qualified staff - Maintain joint work with VEI - Keep the momentum of change and spirit of improvement of the local staff - Enhance management and operational skills - Executive management engagement and higher commitment and ownership of the project, particularly in implementation of plans - Increased ability to retain staff - Foster good relationship and understanding with VEI to favour joint work - Stronger communication, coordination between different sections and team spirit and good working relations within LWB - Additional training

13. Partners’ relational capital - Interaction, roles, governing mechanism, conflict- BWBLWB Formal interactionManagement: high to medium Operations: high to high Management : high to low Operations: high to low Informal interactionManagement: low to medium Operations : low to high Management: low to low Operations: low to medium Roles external partnerConsultant -> colleagueConsultant -> controller Engagement of local partnerLow to highLow to low Dominance of governing mechanisms Contract -> contract + trustContract -> contract Conflict occurrence and impact in the partnership Yes (KPIs, role of VEI ) Positive impact on the partners relationship Yes, (KPI, role of VEI, engineering tasks take over) Negative impact in relationship

14. Satisfaction of partners and stakeholders Degree of satisfaction Observations from partners and stakeholders – MIWD, EIB, EU Appreciate the innovative approach of an external operator working together with the WBs in strengthening their capacity while there needs to be accountability for results. Positively valued in terms of sustainability of the achieved improvements They call for the need to go beyond KPIs to evaluate progress of project – WBs BWB, appreciation of the project at all levels. LWB, appreciation of the project at operational levels, but not at executive management. – VEI RPM BWB emphasised the critical role of the CEO’s support and its translation into approx. 10 members of WB actively supporting the change process. LWB points at the lack of support from executive management, which translates into low commitment at operational levels (only 2 members engaged) BWBLWB WBs’ executive management HighLow WB’s staff at operational and management level High VEI RPM (resident), VEI PDHighMedium EIB, EU, MIWDHighLow- Medium

15. Conclusions Findings – KPIs achievements provided vague information on the achievements of the recipient local operator as well as effectiveness of the partnership – The proposed complementary paths of evidence proved highly informative on the progress of each project and the effectiveness of the partnership The proposed paths of evidence serve different purposes Data sources are highly based on perceptions. Reliability of findings requires: – An objective (external) evaluator – A standard multi-method for data collection – A standard method for data analysis – Representative sample of key informants from both partners and stakeholders – Triangulation of information Progress towards impactEffectiveness of the WOPLearning and adapting KPIsX Capacity changes and attribution XXX Knowledge consolidation into change and obstacles XXX Partners’ relational capitalXX Partners’ satisfactionXX

16. Statements for future improvement of WOPs 1.P-WOPs have the potential to be a slow but firm route to public driven utility reform processes, yet they have to be accountable for results and demonstrate its effectiveness in order to become a widely legitimised approach. 2.A widely used standard system to report P-WOP performance will not only build credibility of WOPs but also enable learning towards improvement of its effectiveness. 3.P-WOP contracts should be designed according to the notion of WOP performance and not merely to performance of the local water operator – Hence, incorporate ( shared ) incentives based on the multiple evidence paths proposed

17. Purpose of 5th Symposium Thank you for your attention. Maria Pascual UNESCO-IHE m.pascual@unesco-ihe.org