American Evaluation Association Conference Evaluation Capacity Building Lessons Learned from the Experience of Botswana Robert Lahey REL Solutions Inc American Evaluation Association Conference San Antonio: November 2010

Background on Botswana Southern African country (to the North of South Africa) Land-locked; Kalahari Desert (5% arable land) Small, sparsely populated (less than 2 M) Independence in 1966 – Significant growth & transformation over last 20 years 2 Key Sectors: Mining (diamonds) & Public Sector Middle Income country - But still major challenges (poverty, unemployment, HIV/AIDS) REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Some Critical Success Factors for M&E Development & Implementation Drivers – Is there a ‘real’ need? Leadership & Commitment – Is there a high-level ‘champion’; sustainability? Capacity ‘to do’ – Is there infrastructure to supply M&E information? Capacity ‘to use’ – Are there incentives to demand & use M&E information? Oversight – Will implementation be monitored? REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Drivers – Is there a ‘real’ need? Two Key Initiatives ‘Vision 2016’ - Long-Term Vision for Botswana 2. Public Sector Reform REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Drivers – Is there a ‘real’ need? ‘Vision 2016’ - “Towards Prosperity for All” Developed in 1996, jointly by public sector, private sector & civil society 7 Pillars setting expectations for the Nation A very public expectation of national performance reporting against Vision pillars Vision Council has mandate/requirement to monitor and report on ‘progress’ REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Drivers – Is there a ‘real’ need? 2. Public Sector Reform Aim - more accountability in the public sector & management of individual Ministries Several cross-government initiatives launched over the 2000s More systematic macro and sector-specific planning Desire to raise focus from expenditures/activities to ‘results’ Timing – last leg of Vision 2016 & desire to base new National Plan more systematically on Vision Pillars REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Leadership & Commitment – Is there a ‘Champion’ & Sustainability? Vision 2016 Vision Council - a high-level group representing all 3 sectors – champion for both ‘M’ and ‘E’ Vision Secretariat & Coordinator – Ongoing operational support and M&E advocate Public Sector Reform Office of the President setting the tone & rules Key for introducing concept & ensuring development New President – reinforced drive to increased accountability & discipline in the public sector REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E? Some critical deficiencies: Insufficient HR with M&E or analytical skills 2. Too little of the ‘right data’ – credible, quality, results-oriented data, and, when needed, at a sub-national level 3. Little/no experience with the conduct of ‘Evaluation’ studies REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E? (cont) But, some elements to build on: 1. Some ‘pockets’ of M&E and analytical expertise across the system 2. Some understanding of RBM and role of M&E across public sector 3. A Central Statistics Office (CSO) – data experts 4. A number of IPDET-trained public servants - beginnings of a ‘network’ 5. M&E unit in a few Ministries (unfilled positions; limited experience) 6. A version of a Vision 2016 M&E system – BUT, not functioning well REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E? (cont) Some achievements to date: Modifications to Vision M&E System Vision Pillars Key Result Areas (KRAs) expected results More systematic linking of Sector/Ministry development plans to Vision Pillars More ‘results’-oriented indicators, (both outputs and outcomes) 4. Intent to develop plans for future ‘Evaluation’ studies 5. Partnering to increase M&E training – U of Botswana; World Bank REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Capacity ‘to use’ – Are there incentives to demand & use M&E information? National Performance Reporting: All sectors (& media) raising question: ‘Where are we?’ re Vision 2016 Pillars and goals First comprehensive results-based ‘progress’ report on nation’s performance against Vision 2016 - ‘Botswana’s Performance’ (April 2009) High-level audience: * President and High Level Consultative Committee National performance report identifies areas for possible future Evaluation (e.g. poverty; youth unemployment; HIV/AIDS) REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Capacity ‘to use’ – Are there incentives to demand & use M&E information? (cont) Ministry-Level Use: Link of Ministry programs to national goals and KRAs identified in latest National Development Plan (NDP 10) Mid-term review of NDP 10 to report on ‘results’ ‘M&E chapter’ in NDP 10 identifies broad need for Ministries to embrace results-oriented M&E But, accountability has not been a part of the culture of organizations * How quickly can/will Ministries embrace M&E? Capacity constraints (HR and data) REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Oversight – Will implementation be monitored? Vision Council and Secretariat Lobbying senior officials to fill M&E capacity gaps – particularly data deficiencies and HR training needs Plan to formalize role in establishing a ‘centre of excellence for evaluation’ Office of the President New unit to provide oversight Progress on M&E implementation will be seen in mid-term review of NDP 10 REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Some Lessons for Capacity Building Importance of senior leadership & political support → a champion to drive change and provide an enabling environment Position M&E as more than a technical issue → a mechanism to support public sector management (M&E is not an end in itself) Links to need to build capacity ‘to use’ M&E information within and across organizations → key users and uses of M&E need to be clarified REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Some Lessons for Capacity Building Manage expectations of senior officials → seeking to achieve in 1 or 2 years what other countries have been working at for some 20 to 30 years Recognize 3 audiences for M&E training & orientation → (1) Senior Officials (2) Program Managers (3) M&E Officers; Analysts Often, too little focus on (1) and (2) REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Some Lessons for Capacity Building Clarify major challenges early on (M&E ‘Readiness Assessment’) → will impact pace of roll-out and identify the need for new investment Moving from ‘concept to reality’ → Need to detail & resource an Implementation Plan (& monitor) The ‘E’ often overlooked → Evaluation strategy & plan and Evaluation policy centre would help in positioning and resourcing the function REL Solutions Inc - Nov. 2010 RELahey@rogers.com

Some Lessons for Capacity Building Too few skilled HR → pilot, using a phased implementation rather than government-wide roll-out of M&E Too little ‘quality’ data → national data development strategy & plan and involve national statistical agency Limited resources → identify potential partners & network with ‘pockets’ of analytical expertise Building support for sustainability → need to navigate the political landscape; an awareness beyond technical skills REL Solutions Inc - Nov. 2010 RELahey@rogers.com

REL Solutions Inc - Nov. 2010 RELahey@rogers.com Contact Coordinates Robert Lahey REL Solutions Inc. Ottawa, Canada E-mail: RELahey@rogers.com Telephone: +1 613 728-4272 REL Solutions Inc - Nov. 2010 RELahey@rogers.com