1. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Leadership for Results Evaluating Leadership Interventions Kabell Konsulting ApS, Dorte Kabell

2. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Leadership work in Madagascar Cabinet level retreats: 2003, 2004 and 2005 Leadership training program for ministers and SGs (AoLF, IPAC): 2004 Transformational leadership program (Dean Williams): 2004-2006 Rapid Results Initiatives: 2005-2006 Training of Chefs de régions: 2005

3. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Evaluation in Madagascar What: Evaluation of AoLF Immediately after event 6 months later A year after Evaluation of IPAC Immediately after event A year later Evaluation of Cabinet Retreats Immediately after event How: Needs assessment (by IPAC) Questionnaires (pre- and post) Structured and open interviews (with participants, non- participants and donors) Document review Direct observation

4. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Key Evaluation Findings (1) 2. Giving self confidence and empowerment at individual level through knowledge of useful management tools and practices, making participants better able to prioritize and manage time and hence stress 4. Facilitating the introduction and internalization of a new results oriented culture, as evidenced in the national vision, budget and strategy and various other new documents. 3. Improving cooperation among individuals and their respective institutions and clarifying roles and responsibilities 1. Creating a strong foundation for increased performance in terms of institutional structure and performance culture as well as a widely shared vision

5. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Key Evaluation Findings (2) The “burning platform for change” A clear commitment at the highest level of government A shared vision for the way forward A well coordinated donor response including synergies between WB/WBI initiatives An increasingly client-owned program where needs have been clearly identified and addressed Issues to consider for replicability:

6. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Evaluating Evaluation Limited participation from beneficiary in evaluation, but strong interest in results Retrospective evaluation, but limited monitoring data available Evaluation focused on “product”, not process Outcome was lessons learned. Limited capacity building - drawing lessons learned does not mean learning to do things differently Lack of a common conceptual framework

7. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Evaluation Recommendations Incorporate M&E from the start - a process from conception to follow-up Evaluation should contribute to building capacity Evaluate process & product Realistic expectations – mindful of trade- offs Common conceptual framework for complementary initiatives

8. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 WBI OPCS/LICUSPREM Evaluation Country Focused Evaluation Framework

9. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Integrated Country Focused Evaluation Framework Capacity development at 3 levels IndividualOrganizationalInstitutional/ Enabling Environment Objectives, based on diagnostic work Improved Diagnostic capacity Improved Coordination, shared goals Improved Trust (Demand) Interventions Dean Williams, AOLF IPAC Cabinet Retreats, Rapid Results PGDI, Judicial reform (Supply) Expected results/Indicators (Does S meet D?) Example: Madagascar

10. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Intervention specific M&E Design of M&E adjusted to each intervention Transformational leadership Peer-to-peer engagement Collective learning events/retreats

11. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Align M&E with design & delivery Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Baseline Results- framework Observation, questionaires Analysis Feedback M&E

12. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Evaluation Design Consider Trade-offs Approach (participative) Methodology Method (self- assessment) Tools (facilitated focus groups)

13. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Tool-box Survey, questionnaire, interviews, content analysis Internal (participants), external (key informants) Evocative (“I feel I have x, y, z) Evidential (“It seems she has x, y, z”) “Work stories” Self assessments

14. Kabell Konsulting ApS - presentation WBI BBL 1.3.06 Issues for Debate Is it feasible to include M&E from start? Is it realistic to expect that evaluation can build capacity? How to determine trade-offs between flexibility and accountability? How to value un-anticipated outcomes and effects?