1. The DEA assessment procedure guideline Assessment of development impacts from energy interventions Example: Wind power system development ≠ the ”physical” wind power system development = improvements among energy users from access to the services enabled by the wind power system improvement = e.g. swifter/cheaper production or consumption, higher incomes, reduced drudgery, higher livingstandards, better health

2. development ”impact” formalized (for now) Behaviors and practices over a period of time Outcome for the agent(s) or other agent(s) Agents (region, village, households, individuals) Modified behaviors and practices over a period of time Modified outcome for the agent(s) or other agent(s) Agents (other/later) Intervention/ program/project The difference between outcomes is the impact we seek. Impact

3. Impact from hypothetical wind powered irrigation project Practice: Rainfed agriculture Outcome: Harvest size/quality/ diversification Agent: Rural villagers Modified practice: Irrigated agriculture Modified outcome: Greater/improved/more diversified harvests Agent: Rural villagers (other/later) Wind powered irrigation project The difference in harvest size/quality/- diversification is an impact Impact

4. Mediating circumstances Practice: Rainfed agriculture Outcome: Harvest size/quality/ diversification Agent: Rural village Modified practice: Irrigated agriculture Modified outcome: Greater/improved/ more diversified harvests Agent: Rural village (other/later Wind powered irrigation project Impact Mediating circumstances E.g. location- or time-specific rainfall, temperatures, ínfrastructure functionality, customs/traditions, prices Mediating circumstances Harvest sizes/quality/diversification affected by mediating circumstances? How assess the impact from the intervention? Attribution problem

5. General ambition Empirically assess a causal linkage fromthe technological input of a given energy intervention to observable development results via the utlization of the (improved) energy services provided by the intervention while keeping an eye on mediating processes

6. Causality challenges (1) Energy does not in itself e.g. quench, feed, house, or clothe people Energy services may be a vital input into meeting those needs => long and complex chain of causality Energy services often bring about improvements in many aspects of life – difficult to capture or conceive of all such improvements Impacts of energy access may take many years to become manifest

7. The type/range of energy services made available varies across energy interventions (different services from wind driven water pumps, solar homes, improved stoves, solar geezers, MFPs) Households, enterprises, or institutions differ in abilities and willingness to invest in new technologies energy – related preferences, traditions and behaviors  Existing demand patterns vary geographically across regions, districts, settlements within districts, and users within settlements  Impacts that are observed in one context do not necessarily appear in another Causality challenges (2)

8. Key requirements to assessment methodology One –size–fits–all approach not possible Need for flexible approach/methodology – adaptable to different types of interventions and contexts knowledge of ”realities on the ground” – energy services needed and used among the specific end-users affected by the intervention systemize interventions into inputs brought in and observable results – causes, effects, and circumstances that intervene  Draw on methodology used for interventions in other sectors, that has been adapted to energy by a group of experts assembled by GVEP.

9. Assessment procedure Design stage Focus, conceptualization, and modeling 1.Identify the stakeholders of your project and their IA needs. 2.Define/delimit the intervention and your focus on the intervention. 3.Illustrate and model the project with results and chains of causality. Bridge model to the real world situation 4.Choose observable indicators for all elements in the results chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders. Implementation stage 7.Collect data. 8.Analyze the data. 9.Draw conclusions and write report. 10.Present the results to your stakeholders.

10. The implementation stage 7. choice of data collection methods – contingent on study-specific circumstances in the design stage 8. data analysis methods and types of conclusions depend on data collection methods. 9.–10. means of presentation to stakeholders - also case specific

11. Where can we provide universally applicable guidance? Design stage Focus, conceptualization and modeling 1.Identify the stakeholders of your project and their IA needs. 2.Define/delimit the intervention and your focus on the intervention. 3.Illustrate and model the project with results and chains of causality. Bridge model to reality 4.Choose indicators for all elements in the causality chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders. Implementation stage 7.Collect data. 8.Analyze the data. 9.Draw conclusions and write report. 10.Present the results to your stakeholders.

12. Stakeholders Conceivable stakeholders domestic public authorities share holders, financial partners donors donor country public authorities beneficiaries/users/clients project management team women’s groups community based organisations researchers in academia the impact assessor him-/herself Factors that affect the assessment design : what kind of information different stakeholders require the intended use of that information how stakeholders would like the information communicated to themselves and other stakeholders

13. General format guidance possible Design stage Focus, conceptualization and modeling 1.√ Identify the stakeholders of your project and their IA needs. 2.Define/delimit the intervention and your focus on the intervention. 3.Illustrate and model the project with results and chains of causality. Bridge model to reality 4.Choose observable indicators for all elements in the causality chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders.

14. Intervention – define, delimit and focus Does the intervention: provide/improve energy utilization for one or several purposes? involve one or several types of technology? Single technology, single purpose - wind driven water pump for irrigation Single technology, multi-purposes - rural solar PV charge stations offering battery charging services for several types of equipment Multi-technology, single purpose – Solar PV and diesel powered water pumps for rural irrigation Multi-technology, multi-purposes - grid electrification: lighting, communication water access, refrigeration and entertainment for homes, agricultural sector, health sector and schools. Which are your stakeholders’ main areas of interest?

15. General format guidance possible Design stage Focus, conceptualization and modeling 1.√ Identify the stakeholders of your project and their IA needs. 2.√ Define/delimit the intervention and your focus on the intervention. 3.Illustrate and model the project with results and chains of causality.. Bridge model to reality 4.Choose observable indicators for all elements in the causality chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders.

16. Illustration and modeling – the “results chain” terminology Inputs: the material, financial, human, and social resources used Activities: actions taken in order to produce specific outputs Outputs: resultant products, capital goods and services (domestic, collective or institutional) Outcome: short- and medium-term effects of an intervention’s outputs effect - a “change, intended or unintended, due - directly or indirectly - to an intervention” Impacts: positive and/or negative long-term effects, subject to greater influence of mediating circumstances than outcomes (inverse “log–frame”)

17. Revised terminology Outcome: Agents practicing irrigated agriculture Impact:: Higher agricultural income (”further downstream”) Input : Wind power equipment Mediating circumstances Output : Irrigation services

18. Solar home system components Lighting Sales, distribution, and installation of SHS components Installed solar home systems Income generating activties in the evening Income poverty reduction Input Use of output Activity Output Outcome Impact Alternative application of results chain : solar home systems (focus: income generation) + +

19. General format guidance possible Design stage Focus, conceptualization and modeling 1.√ Identify the stakeholders of your project and their IA needs. 2.√ Define/delimit the intervention and your focus on the intervention. 3.√ Illustrate and model the project with results and chains of causality. Bridge model to reality 4.Choose observable indicators for all elements in the causality chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders.

20. Indicators A measurable key characteristic of the agents, their behaviour, or their circumstances, that can be related to the intervention’s desired outcome. Operational objective of the assessment: Capture differences in indicator values between agents that have experienced the intervention and those that have not.

21. Choosing indicators – some considerations Pertinence to your project Pertinence to national or regional (aggregate) development objectives Interest for project stakeholders Ease and cost of measurement or data collection Possibilities for triangulation between sources of information Information format - not necessarily numeric. Other examples; pictures videos voice recordings change in behavior or attitudes

22. Solar home system components Lighting Sales, distribution, and installation of SHS components Installed solar home systems Income generating activties in the evening Income poverty reduction Input Use of output Activity Output Outcome Impact Solar home systems with indicators: focus on income generation - no of solar panels - no of inverters - hours of lighting - no of light bulbs used - no of SHS sold - no of SHS installed - no of SHS operational - evening working hours - goods/services production - change in household income + +

23. General format guidance possible Design stage Focus, conceptualization and modeling 1.√ Identify the stakeholders of your project and their IA needs. 2.√ Define/delimit the intervention and your focus on the intervention. 3.√ Illustrate and model the project with results and chains of causality. Bridge model to reality 4.√ Choose indicators for all elements in the causality chains. 5.Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders.

24. Data collection methods Assessment depends crucially on data quality! Each indicator is ideally measured using more than on data collection methods, e.g.: Physical measurement (satellite data on forest cover) Data extraction from public statistics (school attendance or agricultural production) Interviews Extraction of accounting or administrative data of a public or private organization (ESCO customer records) Focus groups and other participative methods Household or firm surveys Often combinations of methods!

25. General format guidance possible Design stage Focus, conceptualization and modeling 1.√ Identify the stakeholders of your project and their IA needs. 2.√ Define/delimit the intervention and your focus on the intervention. 3.√ Illustrate and model the project with results and chains of causality. Bridge model to reality 4.√ Choose indicators for all elements in the causality chains. 5.√ Specify the appropriate data collection methods for each link. 6.Construct a research plan and discuss it with your stakeholders.

26. General format guidance possible Design stage Focus, conceptualization and modeling 1. √ Identify the stakeholders of your project and their IA needs. 2. √ Define/delimit the intervention and your focus on the intervention. 3. √ Illustrate and model the project with results and chains of causality. Bridge model to reality 4. √ Choose indicators for all elements in the causality chains. 5. √ Specify the appropriate data collection methods for each link. 6. √ Construct a research plan and discuss it with your stakeholders. Implementation stage 7.Collect data. 8.Analyze the data. 9.Draw conclusions and write report. 10.Present the results to your stakeholders.

27. Concluding remarks Impact assessments of energy interventions are possible – but challenging!! Huge variety in types of energy interventions Multitude of resultant energy services Complex demand side – difficult to trace development impacts. Need for systemizing intervention and consequences; modelling and design requires creativity, analytical skills, and knowledge of the reality on the ground. Collection of high quality data and dissemination – possibly the most difficult components!