World Meteorological Organization Working together in weather, climate and water WMO OMM WMO Monitoring and Evaluation of Capacity Development

WMO OMM GSN ST n ER n1 KO n12 KO n11 KO n13 Del Act Budget SP OP ( ) SECRETARIAT RBB ( ) Linkage between the next SP, OP and RBB

WMO OMM ST 1ER 1 KOs of TC 1 … TC 8 Indicators, Baselines Targets KOs of RA I … VI Indicators, Baselines, Targets Secretariat Departments Activities and Budget ER 2 For every ER, each RA and TC will define specific Key Outcomes together with Indicators, Baselines and Targets; based on these plans and in close coordination the Secretariat develops deliverables, activities and budget proposals. Framework of the Results-based WMO Operating Plan and Secretariat Budget Strategic level Operational level (5)(8) ER 1 ER 2

WMO OMM KOs, KPIs and Priority Areas ST2: Advancing scientific research and application as well as development and implementation of technology ER4:Enhanced capabilities of Members to access, develop, implement and use integrated and interoperable surface-based and space-based systems for weather, climate and hydrological observations, as well as related environmental observations, based on world standards set by WMO. KO 4.1: WMO Integrated Global Observing System (WIGOS) is implemented KPI – Completion of WIGOS milestones according to WIGOS implementation plan., – Increased availability of observations for users Priority : WIGOS and WIS are priority areas Total of 24KOs, most of them have not more than 2KPIs. Additional 7 KOs on GFCS and DRR as major initiatives

WMO OMM 1.WMO Strategic Plan : High-level statement of strategic directions and priorities expressed through Strategic Thrusts and organization-wide Expected Results and Performance Indicators addressing the Global Societal Needs; 2.WMO Operating Plan: converts the organization-wide ERs into specific outcomes/ deliverables, indicators and targets 3.Secretariat Implementation Plan and budget: converts the organization-wide ERs into Secretariat specific outcomes/ deliverables, indicators and targets and links them to the planned activities and the Secretariat budget; 4.WMO Strategic Executive summary: In a language appealing to those outside WMO and those making decisions to fund NMHSs. and Tool/System for Monitoring & Performance Evaluation – Tracks implementation and provides for control and corrective actions for achieving the ERs. Strategic planning documents

WMO OMM WMO Strategy for Capacity Development : Why? Establish a CD Strategy to ensure, in a holistic approach, all actors in Met/Hydro/Climate work towards the same overall objective : Facilitating sustainable development of NMHS Resolution Cg-XVI/B/WP 11.2 Congress noted that a coordinated and cohesive approach for capacity development would be needed to enhance capabilities of NMHSs in developing countries to meet growing Societal Needs at different levels. Congress noted that capacity building is a crosscutting activity and contributes to all Expected Results (ER), especially ER 6 and Strategic Thrust 3 of the Strategic Plan.

WMO OMM Definitions and Logic of WMO strategic planning approach GSNs : Global Societal Needs STs : Strategic Thrusts ERs : Expected Results KOs : Key Outcomes Actions and Budget WMO Strategic Plan WMO OP

WMO OMM

Strategic Thrust n3 Expected Result n6

WMO OMM Tools to Monitor and Evaluate Progress Types: Empirical, Qualitative, Index, breadbasket, case histories, Indicators (available economic, social or environmental) Example Collection tools: –Self reporting: Surveys, Visits of PR and national staff –Country assessments –National Records –Reports: OPAGS, Regional WGs, Fellowship Evaluations, project evaluation –Country Profile Data Base

WMO OMM Conceptual Framework for Monitoring and Evaluation Inputs WMO governance decisions TCs and RAs human resources Financial & human resources: WMO Secretariat WMO Joint Programmes Members Partner Organizations contributions Activities Activities as stipulated in the WMO Operating Plan based on contributions of the TCs, RA and the Secretariat Activities of WMO Joint Programmes Activities of Members Deliverables Deliverables as stipulated in the WMO Operating Plan based on contributions of the TCs, RAs and the Secretariat Deliverables of WMO Joint Programmes Deliverables of Members Expected Results Expected Results as stipulated in the WMO Strategic Plan Expected Results of WMO Joint Programmes M&E Feedback and learning WMO Secretariat enabling processes and coordination Key Outcomes Key Outcomes as stipulated in the WMO Operating Plan Key Outcomes of WMO Joint Programmes

WMO OMM Capacity Development – and ER 6 Expected Result 6: Enhanced capabilities of NMHSs, in particular in developing and least developed countries, to fulfil their mandates. Looking at Key Outcomes (Kos) and Key Performance Indicators (KPIs) How do we measure?

WMO OMM EC-WG/SOP Questionnaire The working group reviewed in 2011 a standardized questionnaire to assess the impacts of the achieved results. Such a survey could help create baselines and targets.

WMO OMM Draft Survey on WMO Impacts of Achieved Results on Members Survey Questions for Capacity Development, ER 6 Key Outcomes KO 6.1: NMHSs and Regional Centers are improved, particularly in developing and least developed countries

WMO OMM KPI 6.1.1: Number of NMHSs and WMO Regional Centers with defined roles in national and regional socioeconomic development activities, in cooperation with international and regional development agencies

WMO OMM KPI 6.1.2: Number of NMHSs with programmes and networks for collaborating on service delivery at regional levels

WMO OMM KO 6.2: Infrastructure and operational facilities of NMHSs and Regional Centres are improved, particularly in developing and least developed countries

WMO OMM KPI 6.2.1: Number of NMHSs with improvement in infrastructure and operational facilities to address identified deficiencies

WMO OMM KPI 6.2.2: Number of NMHSs with modernized capacities to develop and disseminate products to their customers

WMO OMM KO 6.3: Education and Training Development activities at national and regional levels are improved, especially in developing and least developed countries

WMO OMM KPI 6.3.1: The number of RTCs providing education and training support for GFCS related activities To be completed by WMO Secretariat using other sources.

WMO OMM KPI 6.3.2: The degree to which Members are getting value for money from the WMO Fellowship Programme

WMO OMM KPI 6.3.3: The degree to which the RTCs support the regional training demands in particular for students from developing and least developed countries

WMO OMM KO 6.4: Capacities of NMHSs are enhanced through cooperation and partnerships with other national and regional organizations

WMO OMM KPI 6.4.1: Development projects and activities funded through voluntary and extra-budgetary resources To be completed by WMO Secretariat using other sources.

WMO OMM Your inputs…ideas? Questions in Survey…. –Are we measuring what needs to be measured to show the capacities of NMHS are being developed? –Are the questions Too simple (yes/no) to be of use? Too difficult to answer with enough specificity to show improvement in 4 years? About right –What other tools does WMO need to develop/improve (if any) to evaluate and monitor CD?

WMO OMM Additional Sources Survey on the Role and Operation of Meteorological Services (2011) Regional Surveys Programmatic Surveys Information from assessments done on bilateral basis, other development agencies or at sub- regional levels as part of project development

WMO OMM Annex 1 KPI reference sheet to be completed

WMO OMM Annex 2 Example of a duly completed KPI Reference Sheet