Performance Management Planning Development Grants Program (DGP) Detailed Implementation and Management Planning (DIMP) Workshop PART 1 Dakar, Senegal January 2010

Course Objectives:  To understand what a Performance Management Plan (PMP) is  To practice the development of results logic from your project activities  To examine what makes good indicators and apply that to your project results 2

Course Objectives:  To understand the use of an indicator resume  To use a practical approach to target setting based on your workplan  To start planning Performance Management tasks 3

Session I Introducing the Performance Management Plan (PMP)? Key Definitions

5 Project  General definition: carefully planned temporary endeavor aimed to achieve a particular change in the situation of the target population.  ADS definition: “structured undertaking of limited duration and broad scope. Through its component activities, a project mobilizes inputs such as commodities, technical assistance, training … to produce outputs or results that will contribute to achieving an Assistance Objective.” (Def. p. 69)

6 Activity  General definition: the clearly identifiable and measurable means of implementing the project. (What you do)  ADS definition: “…structured undertaking of limited duration and narrow scope. It mobilizes inputs such as commodities, technical assistance, training … to produce outputs or results that will contribute to achieving an Assistance Objective.” (Def. p. 58)

What is Performance Management? 7 It is the systematic process of:  monitoring the achievements of program operations;  collecting and analyzing performance information to track progress toward planned results;  using performance information and evaluations to influence decision making and resource allocation; and  communicating results achieved, or not attained, to advance organizational learning….” (Def. p 67)

“Domino effect” in Performance Management context 8

9

10

“Domino effect” in Performance Management context 11 ?

Performance Management 12  Being busy is not the same as producing results  Results not achieved guide evaluations to improve decision making The major contribution of performance management is the focus on achieving results—a measurable change in the condition of a customer, directly or indirectly.

Performance Management means we learn from the real world to improve effectiveness 13

What is a Performance Management Plan (PMP)? 14 ADS 200 definition:  A tool used by USAID Mission/Office and assistance objective team to plan and manage the process of assessing and reporting progress toward achieving an Assistance Objective (ADS p. 67)

Development Grants Program (DGP) Annual Program Statement (APS) 2009 The technical approach should include discussion of:  Problem Analysis  Objectives  Methods  Plan  Monitoring and Evaluation Approach There should be a clear logic linking the objectives to the problem analysis as well as linking the methods to the objectives. 15

Contents of a PMP  Narrative description of the specific cause-and-effect logic among project Intermediate Results and the highest level objective  Graphic illustrating the cause-and-effect logic among Project intermediate results and the highest level objective. It includes Indicators to measure project performance and any critical assumptions  Indicator Resumes containing information about measures and methods of collecting data on project performance 16

Contents of a PMP (continued)  Table with information (data) on the situation before project implementation and the planned level of results to be achieved within the explicit timeframe  Calendar in the form of a Gantt chart with a list of management tasks and deadlines in order to develop and implement the PMP 17

Starting a Results Framework 18

Review the project activities and project objective(s) as stated in your Cooperative Agreement: 1.Ask: who is my customer and what measurable change(s) in the condition of the customer do we propose? 2.From these answers, draft the project objective and then write this on a card and attached at top of flip chart page. 3.Select key project activities (what you do) to achieve the change(s for which you are willing to be held accountable). 4.Write each activity on a separate card and attach the cards across the bottom of the same flip chart. NGO Task I: Starting a Results Framework Working time: 10 min. 19

Session II What connects Activities to the Project Objective?

Understanding the Problem 21 Problem Analysis: a systematic process used to determine causes and effects of a problem and to link them in a cause—effect relationship.

Cause and effect relationship Example: Infant mortality reduced 22 ANALYSIS OF PROBLEMPROBLEM LEVEL High rates of mortality among infants 0-12 mos. WHY? MORTALITY Diarrhea disease rates are very high among infants 0-12 mos; many die from diarrhea. WHY? DISEASE Mothers are giving breast milk substitutes to infants under 6 months Mothers are taking infants 7 – 12 mos. off the breast and feeding plain porridge prepared with unclean water. WHY? BEHAVIOR Mothers believe that breast milk alone does not satisfy infants 0-6 mos. Mothers believe that porridge is an adequate food for infants 7 –12 mos. Grandmothers insist on following traditional practices. WHY? BELIEF Mothers do not know that unclean water will make infants sick Mothers do not know that an infant does not need other foods besides breast milk up to 6 mos. Mothers do not know that an infant benefits from breast milk up to two years. KNOWLEDGE

Defining specific activities to fix the causes of the problem 23 Which causes are more responsible for a larger part of the problem (whose reduction or elimination will contribute more to reducing the overall problem) ? Which causes do we have capacities to address?

Cause and effect relationship Example: Infant mortality reduced 24 ANALYSIS OF PROBLEMSOLUTION LEVEL High rates of mortality among infants 0-12 mos. To reduce infant mortality Diarrheal disease rates are very high among infants 0-12 mos; many die from diarrhea. To reduce incidence of diarrhea WHY? Mothers are giving breast milk substitutes to infants under 6 months Mothers are taking infants 7 – 12 mos. off the breast and feeding plain porridge prepared with unclean water. To change feeding behavior WHY? Mothers believe that breast milk alone does not satisfy infants 0-6 mos. Mothers believe that porridge is an adequate food for infants 7 –12 mos. Grandmothers insist on following traditional practices. To change beliefs WHY? Mothers do not know that unclean water will make infants sick Mothers do not know that an infant does not need other foods besides breast milk up to 6 mos. Mothers do not know that an infant benefits from breast milk up to two years. Family training in potable water and breast feeding WHY?

25 Climate change has affected the watershed basins of Santa/Piura Chira, Peru. Without conservation measures and improved water resources management, water will become scarce in the long-term. Populations affected are essentially divided into two groups: the highland people the lowland people. Understanding the “Peru Water Project” Problem/Case

Understanding the “Peru Water Project” Problem/Case (continued)  Land clearing for agriculture and investments in water management infrastructure is controlled by the Municipality and guided by the interest of the local community.  Conflicts exist between the two groups as decisions made do not necessarily include the interests of the other group.  The project intends to address the problem by creating opportunities for joint highland/lowland decision-making processes based on scientific information.  The objective is reached by these Activities: –Promotion of Stakeholder Conferences –Facilitation of Action Groups. 26

27 What is Causal Logic? Causal Logic is a way to demonstrate your theory of how the Activities delivered by your project lead to the Objective (“change in the condition of the customer”).

28 Intro to Table Exercise 1: Developing a Causal Logic Chain

Table Exercise 1 29 Each table receives an envelope with cards. Task: Organize the cards in a causal logic chain that flows from the bottom up with the “Activity” (at the bottom) and the “Project Objective” at the top; post, in order, on a flipchart page. Working time: 5 min.

30 Project Objective: Strengthening basic climate change knowledge, capacity, and decision-making of key stakeholders in two Watersheds Activity 1: Facilitate action groups Activity 2: Promote stakeholder conferences To assure good representation at the conference To establish effective decision- making processes To avoid conflicts over water To promote politically balanced decisions To increase investment in improved water resources management To generate the political will to change water management decision making

Assistance Objective (AO)  The most ambitious result that a USAID Mission/Office, along with its partners, can materially affect, and for which it is willing to be held accountable (Def. p. 59) Project Objective  The highest level of measurable change in a condition of a target audience for which a project is willing to be held accountable. Result  A significant, intended, and measurable change in the condition of a customer, or a change in the host country, institutions, or other entities that will affect the customer directly or indirectly (Def. p. 69)

32 Intro to NGO Task II: Building a Causal Logic Chain

 Use your flip chart from NGO Task I that shows your Project Objective at the top and 2-4 Activities listed across the bottom.  Build the causal logic for each activity by asking the question “WHY” from the bottom up.  Stop when the answer to the question “WHY” is the Project Objective. Working time: 20 min. NGO Task II: Building a Causal Logic Chain for Your Project 33

Session III What is a Project Result?

35

What Is a Result? A Result = Amount of change achieved or a problem solved  Def. p 69, Result  Def. p 60, Customer 36 Results are expressed in terms of a desired future condition, often ending or beginning with the words ‘increased’ or ‘decreased’.

Types of Results: USAID Definitions Assistance Objective (Def. p 59, Strategic Objective – same) The most ambitious result that a USAID Operating Unit, along with its partners, can materially affect, and for which it is willing to be held accountable. Intermediate Result (IR) An important result that is seen as an essential step to achieving an Assistance Objective. IRs are measurable results that may capture a number of discrete and more specific results. IRs may also help to achieve other IRs. 37

Characteristics of a good Result Statement 38 To the extent possible, a result should be uni-dimensional and SMART: S pecific – It should be exact, distinct and clearly stated. M easurable – It should be measurable in some way, involving qualitative and/or quantitative methods. A chievable – It should be realistic with the human and financial resources available. R elevant – It should relate to the project focus. T ime-bound – It should be achieved in a stated time-frame.

Some rules to keep in mind when developing results statements 39 Avoid overly broad statements. Which specific aspects of the result will project activities emphasize? State results as future completed actions. The statement should describe the end state that is desired as the outcome of the activity. Use strong action verbs. This makes results easier to implement: for example, constructed, eradicated, or reduced. Weak action verbs include: enhanced, liaised, or coordinated.

40 Be clear about what type of change is implied. What is expected to change—a situation, a condition, the level of knowledge, an attitude, or a behavior? Identify precisely who or what should change. Who: Is change expected to occur among individuals, families, groups, communities, regions? What: Income, water quality, laws, regulations, level of conflict? Some rules to keep in mind when developing results statements (continued)

41 Objective: Strengthening basic climate change knowledge, capacity, and decision-making of key stakeholders in two watersheds. Results Statement: Basic climate change knowledge, capacity, and decision-making of key stakeholders in two watersheds strengthened. Water resources management in the watershed basins of Santa/Piura Chira improved

42 Water resources management in the watershed basins of Santa/Piura Chira improved To facilitate action groups Decision-making process functions effectively Political will to change water management decision- making generated To have effective decision-making processes To avoid conflicts over water To promote politically balanced decisions Conflicts over water avoided Politically balanced decisions made OBJECTIVESRESULTS STATEMENTS -

43 To promote stakeholder participation in conferences To generate the political will to change water management decision- making To increase investment in improved water management Investment in improved water resources management increased Political will to change water management decision making generated Representation of all key stakeholders at conferences secured Water resources management in the watershed basins of Santa/Piura Chira improved OBJECTIVESRESULTS STATEMENTS I

NGO Task III: Turning the Project Causal Logic Chain into Results Statements 1.On a card, revise the Project Objective at the top of the flip chart to read as though the change were already achieved. 2.Check this Results Statement against good characteristics (SMART) and revise if needed. 3.Replace the Objective card at the top of the flip chart with the new Results Statement card. 4.Restate the other items in your causal logic from Task II as Results, check them against the SMART criteria and write each on a card. Attach them on the flip chart. 35 min. 44

Session IV What is a Results Framework ?

Results Framework (as we know it so far) 46 IF THEN

47 Capture the key elements of the development hypothesis for achieving an Assistance Objective (AO) (program intent and content) Clearly state results at AO and Intermediate Result (IR) levels Provide sufficient detail and causal linkages to diagram the development hypothesis Use results statements that are measurable and feasible during the planned life of the AO given anticipated resource levels Incorporate critical assumptions What Should a Results Framework do? (ADS )

Results Framework (as we know it so far) 48 IF THEN What Else? Context? Assumptions? Risks? How? Why?

To refine what we have into a complete RF: 1. Articulate causality (cause and effect) 2. Apply the “Necessary and Sufficient Rule” 3. Define critical assumptions 49

50 An example of causality  Jobs created  Incomes increased  People trained  Studies completed  Experts, equipment, funds  Health conditions improved  Livelihoods improved Inputs/Activities Outputs Outcomes Project Result/Impact IRs Project Result

51 Results at Different Levels  ADS Def. p. 63: A higher level or end result at the AO level. An outcome is expected to have a positive impact … and lead to change …  ADS Def. p. 67: A tangible, immediate, and intended product or consequence of an activity within USAID control. Ex.: people fed, trained, etc.  ADS Def. p. 63: A resource, such as technical assistance, commodities, training … used to create an output  ADS Def. p. 63: Impact refers to the effects, usually medium and long term, produced by a project/program. The impacts can be intended or unintended, positive or negative Input/ActivityOutput Outcome Project Result/Impact

Outcomes: The intended or achieved short-term and medium-term effects of a project’s outputs. Outcomes represent changes in development conditions (for customers) which occur between the completion of outputs and the achievement of Impact. Outputs: The products and services which result from the completion of activities within a development intervention. 52 In other words…

USAID View of Cause and Effect (Development Hypothesis)  Ultimate customer or customer’s environment impact 1-20 years  Do not always succeed but produce learning from experience  Produce data on the road of causality traveled to benefiting customers: new knowledge & impact 53

What Is a Development Hypothesis? (Def/ p.61) 54 A narrative description of the specific causal links between Intermediate Results (IR) and the Assistance Objective (AO) (Project Result). Refer to the Development Hypothesis of the Peru Case to see an example

The Results Framework and the Development Hypothesis Looking at the Results Framework in the Peru Case and answer the following question:  What is the difference between the Results Framework and the Development Hypothesis? 55

Results Framework The Necessary and Sufficient Rule 56 IF THEN

What is a Critical Assumption? A general condition under which the development hypothesis, or strategy for achieving the objective, will hold true. Critical assumptions …  Are outside the control or influence of USAID and its partners.  May exist from one level of the causal chain of a results framework to the next.  May apply to the entire strategy. 57

Building a Results Framework: Some Advice 58  The linkages flowing up the Results Framework should be causal—achievement of one result is necessary for, and contributes to, achievement of the other.  The causal connection between two IRs or between an IR and an AO in a Results Framework should also be direct. You should not need to infer additional IRs to understand the linkage between two results, or accept many or broad assumptions to move from a “lower” result to a “higher” result or Strategic Objective (SO).

Table Exercise 2 59 Each table receives an envelope with cards. Task: 1.Organize the cards in a Results Framework going top down with the Project Result at the top of a flipchart page and the Activities at the bottom. 2. At each level apply the Necessary and Sufficient Rule. 3. When satisfied with the logic, place causal arrows. Working time: 15 min.

60 Water resources management in the watershed basins of Santa/Piura Chira improved Water resources management improved Politically balanced decisions made Investment in improved water resources management increased Conflicts over water avoided Political will to change water management decision making generated Effective decision making processes function Stakeholder capacity increased Good representation at the conference secured Facilitation of action groups Promotion of Stakeholder conference

NGO Task IV: Creating a Results Framework Task: 1.Turn your Project Results Statements into a Results Framework by rearranging your Causal Logic Chain. 2.Apply the Necessary and Sufficient Rule at each level. 3.Rearrange Results, add or subtract Results, until satisfied that the logic meets the Necessary and Sufficient Rule at all levels. 4.Place causal arrows. 5.Add any critical assumptions. 60 min. 61