GSF Results and Financial Monitoring Workshop Value for Money Study (2016) GSF Results and Financial Monitoring Workshop Dear es Salaam

Content What is VFM analysis? Adapting VFM analysis to sanitation and hygiene programmes Applying the methodology: Country case studies and desk reviews Reflections on applying the methodology

08/11/2018 What is VFM? Value for money (VFM): The optimum combination of whole-life cost and quality (or fitness for purpose) to meet the user’s requirement. VFM is about striking the best balance between the “three E’s” − economy, efficiency and effectiveness. A fourth “E” – equity – is now also sometimes used to ensure that value-for-money analysis accounts for the importance of reaching different groups. “S” – Sustainability as additional dimension for sustained outcomes VFM is not a tool or a method, but a way of thinking about using resources well. So why do projects or organisations do VFM -

Internal VFM / External VFM 08/11/2018 How can VFM analysis be used in development context? To help managers better understand and analyse performance issues they see on the ground, and their associated cost To demonstrate results and attract funding based on evidence To create a culture of transparency around programme results – what are our costs, what are our results. What do they mean? So why do projects or organisations do VFM - Internal VFM / External VFM

VfM dimensions across the WASH results chain The financial costs of inputs Eg. organizational expenditure at all levels

VfM dimensions across the WASH results chain The resources used in terms of finance and staff time (capital and labour)

VfM dimensions across the WASH results chain Economy = a measure of the expenditure for resources needed for the programme

VfM dimensions across the WASH results chain The process by which inputs are transformed into results. The process can be the object of a programme evaluation Eg. Pre-triggering, triggering, post-triggering follow-up

VfM dimensions across the WASH results chain Outputs: the direct deliverables of the programme Eg. number of triggering meetings, number of improved latrines constructed, communities declared ODF Assumed outcomes: assumed outcomes resulting from the outputs Eg. number of people using latrines, living in ODF environments

VfM dimensions across the WASH results chain Efficiency = a measure of how well resources (inputs) are being used to achieve programme deliverables (outputs)

VfM dimensions across the WASH results chain Cost-efficiency = a measure of how well costs are transformed into programme deliverables

VfM dimensions across the WASH results chain The actual change in people’s lives over time (appropriately measured) Eg. As assumed outputs, but measured/verified as opposed to assumed

Cost-efficiency = a measure of how well costs are transformed into sustained programme outcomes

Objectives of the study 1) Assess mature GSF-funded sanitation and hygiene programmes to gauge current levels of economy, efficiency and effectiveness by applying standard VFM analysis procedures and producing current unit costs of outcomes. 2) Recommend a better cost classification structure and aggregation procedure to facilitate future Value for Money analyses and standardise this for cross-sector benchmarking. 3) Compare findings from GSF programmes with existing data in the sanitation/hygiene sector so as to benchmark GSF performance Assumptions Limitations

Understanding GSF financing and data sources

Cost classification – programme costs Capital expenditure software Activity + implementation expenditure on demand creation Expenditure on direct support Expenditure directly supporting implementation activities Expenditure on indirect support Institutional, or other staff expenditure not directly linked to sub-grantee implementation Sub-grantee operating costs UNOPS hosting fee Sub-grantee activity and programme staff expenditure CPM costs EA management fee and EA operating costs GSF Staff cost & operating costs Executing agency implementation Cost classification – non-programme costs Capital expenditure hardware i) HH expenditure on latrine construction ii) Expenditure by other agencies in the implementation area

Step 1: Align sub-grantee financial data with indicators of interests. 08/11/2018 Summary of methodology and process Step 1: Align sub-grantee financial data with indicators of interests. Coding of a sample of sub-grantee financial reports by expenditure type and activity (overheads, programme staff costs, expenditure on triggering, post-triggering, School WASH, BCC sessions etc…). Step 2: Attributing global level expenditure to country programmes and activities Separate processes applied for global expenditure, EA expenditure, and SG overhead expenditures Step 3: Combining coded financial data with monitoring data for VfM Indicators Indicators of interest are those which are monitored

Cost per ODF village – Cambodia and Madagascar

Approximate village size: Cambodia 250 Madagascar 10-20 Cost per ODF village – Cambodia and Madagascar Approximate village size: Cambodia 250 Madagascar 10-20 ODF village conversion rates per sub-grantee Madagascar Cambodia

Example of the output of activity coding (Nepal) Activity coded expenditure (output of step 1) Example of the output of activity coding (Nepal)

Results of the financial analysis (output of step 2)

Results of the financial analysis (output of step 2)

Desk reviews – Cost-efficiency

Desk reviews – cost- efficiency

Desk reviews – findings economy/efficiency Economy indicators are hugely sensitive: both to context, definitions in the monitoring data and the methodology (codes used). Dispersion around the mean reduces as you move from economy to cost-efficiency. Partially as monitoring indicators become more closely aligned. Figures capture key dimension of programme activity but not all. Different programmes have different amounts of programme expenditure ‘on the books’.

Reflections on applying the methodology The method of allocating of different types of expenditure to activities varies in each country, but in all cases we saw SG’s recorded the expenditure types prior to EA aggregation  the data are hard to get at the moment but there if the GSF want them! Only an implicit theory of change embedded in the results framework  but results framework does not necessarily link to the financial reporting or monitoring data; nor harmonised across countries Potential for two tiered methodology: one analysis tailored to the country context, a more general global methodology  but there is a tension between the two.

Can it be done = yes with some reservations Reflections on applying the methodology Can it be done = yes with some reservations ++ Excellent financial reporting mechanisms at sub-grantee level – these are largely activity based, and subject to strong check and balances. Significant opportunity for sector learning. Initially in terms of internal VFM – assessing the economy and efficiency costs of different innovative approaches to sanitation and hygiene promotion; or the cost of reaching the hard to reach communities – the last 20%. -- At present deriving VFM results from current reporting is difficult and very time consuming – especially on an SG by SG basis. VFM analysis of EA level is limited by differing and unclear methods of aggregation of SG data. Finding are very sensitive to variations in how monitoring data is collected and how key results are calculated.

08/11/2018 Recommendations 17 Recommendations, grouped into five thematic areas: i) Three on more clearly establishing causal relationships between programme inputs and outputs/outcomes. ii) Two on the collection and treatment of financial data. iii) Four on the collection and treatment of the financial data. iv) Three on the perspective and purpose of future analyses; internal or external VfM analyses. v) Five on cost classifications for future VfM analyses.

Next steps GSF M&E system review & strengthening 08/11/2018 Next steps GSF M&E system review & strengthening Results Framework update Financial Reporting Tool finalization Roll-out Applying more articulated VfM considerations in future programming WSSCC VFM of policy advocacy

Thank you. More information: www.wsscc.org