Monitoring and Evaluation of RA II Strategic Operating Plan (SOP) L.S. Lee, Hong Kong, China Chairman of the Task Team on RA II Strategic Planning

The last regional survey (2010–2011) An RA II Survey Questionnaire on the basic capability of NMHSs in RA II ( ) for monitoring the progress in the implementation of RA II Strategic Plan was developed by the RA II Task Team on Strategic Planning. The survey questionnaire covered 12 main topics including management, observing systems, telecommunications, forecasting system, natural disaster prevention and mitigation, climate services, aeronautical meteorological services, hydrological services and partnership. The survey was carried out in Responses were received from 30 out of 35 Members. Possible reasons for the Members not responding to the survey included: –ambiguous questions –communication problem –different ministries handling weather, climate, water and environment services.

The last regional survey (2010–2011)

Analysis showed that there was overall improvement of weather, climate and water services by Members, including service delivery capability, infrastructure for observation, telecommunication and forecast products. However, differences among Members were also noted. –Insufficient observational infrastructure and capacity to produce and provide reliable and timely forecast and warning services by some Members. –Many LDCs could not afford to have qualified maintenance technicians for observation instruments or the communication infrastructure for real- time delivery of observations. –Further support for NMHSs in developing countries was required, in particular LDCs in the Region, to enable them to implement quality management systems and cost-recovery of services.

The last regional survey (2010–2011) The results of survey were used to form the baseline of achievements of the deliverables in the RA II Operating Plan , with a view to monitoring and evaluating the progress of the implementation of the deliverables.

Development of RA II Strategic Operating Plan for 2012–2015 The Strategic Operating Plan for the Enhancement of NMHSs in RA II (Asia) for 2012–2015 was developed by the RA II Task Team on Strategic Planning. It took into account: –previous RA II Strategic Plan 2009–2011 –WMO Strategic Plan 2012–2015 (WMO-No. 1069) –WMO Operating Plan 2012–2015 –Cg-XVI decisions –evolving needs and deficiencies identified through the 2010–2011 regional survey. The RA II Strategic Operating Plan 2012–2015 is composed of: –Strategy part: including introduction describing the purpose and how to use the plan; priority areas of RA II; and monitoring and evaluation mechanism; and –Operating Plan: containing Regional Key Outcomes (RKOs) and corresponding Key Performance Indicators associated with WMO’s eight Expected Results. There are 114 priority deliverables, associated activities, relevant Programmes and technical commissions, responsibility, implementation schedule, and baselines and targets defined from the 2010–2011 survey.

RA II Strategic Operating Plan for 2012–2015

Monitoring and Evaluation of RA II SOP Monitoring and evaluation are tools to measure the performance of implementation of the Strategic Operating Plan. They also contribute to the identification of good practices and lessons learned with respect to implementation, as well as policy, strategy and programmatic design that will inform the next phase of strategic planning. A Task Team on RA II Strategic Planning was established by the ad hoc meeting of RA II Management Group (MG) in Doha during RA II-15, with the composition being: –L.S. Lee (Chair: Hong Kong, China) –Xu Xianghua (China) –M. Jabbari (Islamic Republic of Iran) –T. Kimura (Japan) –S-W Kim (Republic of Korea) –A. Al-Mannai (Qatar)

Monitoring and Evaluation of RA II SOP The Task Team was requested to develop a new survey questionnaire for implementation in 2014 and 2016 (Ref: Annex III of the Final Report of the 7th Session of the RA II MG). The survey will help analyze the likely trends, developments, evolving needs and deficiencies of the Region. Members are requested to take a proactive approach in responding to the diverse areas of questions of the survey and make consultations with relevant national authorities when responding to the survey to guarantee collection of appropriate information.

Design of Survey Questionnaire for RA II SOP Capability indicators (with baseline and target) surveyed in the past are suggested to be kept in the new questionnaire. New survey items as suggested in the SOP should also be added. In light of language problem for Members in Central Asia, Russian Federation had offered to assist in translating the questionnaire and transmitting to the Members concerned. Drawing from the experience from the last survey ( ), the new survey questionnaire can be further improved by the following: –For each KPI, it can be indicated clearly in the survey form the achievements of all Members based on last survey result, so that Member can immediately compare how close or far it is from the others when filling in the survey form, and then find ways to gear up itself. –When a Member does not meet a certain KPI / expected result, instead of simply answering "no" in the survey form, it should indicate if there is any difficulty encountered, whether it has any plan to make improvement and what the schedule is.

Design of Survey Questionnaire for RA II SOP (Cont’d) Drawing from the experience from the last survey ( ), the new survey questionnaire can be further improved by the following: –A section on feedback can be added at the end of the survey form for Members to raise any issues in connection with the SOP to the Regional Association. –A follow-up focal point should be assigned by each Member for the Regional Association to contact during the analysis of survey information. –Besides following the format of the last survey form, other new ways of information collection can be adopted, e.g. two-step answer questions, multiple-choice questions. –Experience of RA V in developing its survey form can also be drawn from (Section 4 to help identify main priorities for RA V ).

Design of Survey Questionnaire for RA II SOP Other points to consider: –Survey the status as at 2014 only, or back to 2012 and 2013? –Structure/flow following survey or SOP ? –Survey for members only or also for WG, WG, ICT….?

Thank you!