Midterm Evaluation of GEOSS Highlights and Key Messages from the Evaluation Report Charles Hutchinson Chair, GEOSS Midterm Evaluation Team GEO Executive.

Slides:



Advertisements
Similar presentations
ENTITIES FOR A UN SYSTEM EVALUATION FRAMEWORK 17th MEETING OF SENIOR FELLOWSHIP OFFICERS OF THE UNITED NATIONS SYSTEM AND HOST COUNTRY AGENCIES BY DAVIDE.
Advertisements

Guidance Note on Joint Programming
Delivering as One UN Albania October 2009 – Kigali.
GEOSS Data Sharing Principles. GEOSS 10-Year Implementation Plan 5.4 Data Sharing The societal benefits of Earth observations cannot be achieved without.
Capacity Building Mandate We, the participants…recognize the need to support: …A coordinated effort to involve and assist developing countries in improving.
1 GEOSS Monitoring & Evaluation. 2 1st GEOSS M&E WG meeting outcomes Agreed on 1.the final draft of Terms of Reference for the M&E WG 2.The plan for delivery.
Overview of the Summary Plan for the First Evaluation of GEOSS Implementation Craig Larlee (Canada) M&E WG Co-Chair Joint meeting of M&E WG and ET January.
GEOSS Monitoring and Evaluation Charles S. Baker (USA) M&E WG Co-Chair Joint meeting of M&E WG and ET January 20, 2010 Geneva, Switzerland 1.
Transparency and promotion of the CDM
Alan Edwards European Commission 5 th GEO Project Workshop London, UK 8-9 February 2011 * The views expressed in these slides may not in any circumstances.
© 2005 The Finance Project Module II: Developing a Vision and Results Orientation Oregon 21 st Century Community Learning Center Programs October 2011.
© GEO Secretariat The Group on Earth Observations – Status and Post 2015 Osamu Ochiai GEO Secretariat 41 st CGMS Tsukuba, Japan 8-12 July 2013.
GEO Strategic Plan : Implementing GEOSS
Decision Making Tools for Strategic Planning 2014 Nonprofit Capacity Conference Margo Bailey, PhD April 21, 2014 Clarify your strategic plan hierarchy.
Comprehensive M&E Systems
EEN [Canada] Forum Shelley Borys Director, Evaluation September 30, 2010 Developing Evaluation Capacity.
Slide: 1 27 th CEOS Plenary |Montréal | November 2013 Agenda Item: 15 Chu ISHIDA(JAXA) on behalf of Rick Lawford, GEO Water CoP leader GEO Water.
Student Assessment Inventory for School Districts Inventory Planning Training.
Critical Role of ICT in Parliament Fulfill legislative, oversight, and representative responsibilities Achieve the goals of transparency, openness, accessibility,
GEO Work Plan Symposium 2012 ID-05 Resource Mobilization for Capacity Building (individual, institutional & infrastructure)
Atlanta Public Schools Project Management Framework Proposed to the Atlanta Board of Education to Complete AdvancED/SACS “Required Actions” January 24,
Report of the IOC Task Force on the GEOSS Common Infrastructure (GCI) Ivan DeLoatch, U.S. Geological Survey Alan Edwards, European Commission Co-chairs.
GEO Strategic Plan : Implementing GEOSS AN UPDATE GEO WORK PLAN SYMPOSIUM 05 MAY 2015 GENEVA, SWITZERLAND.
GEOSS Midterm Evaluation User Interface Committee – March 2010 John Adamec & Yana Gevorgyan GEOSS Evaluation Team.
Do it pro bono. Strategic Scorecard Service Grant The Strategy Management Practice is presented by Wells Fargo. The design of the Strategic Scorecard Service.
Evaluation in the GEF and Training Module on Terminal Evaluations
6. Strategic Plan : Implementing GEOSS Validating the way forward: Review of feedback from Plenary.
© GEO Secretariat 5.2 Monitoring and Evaluation John Adamec Co-Chair, M&E Working Group GEO-XI Plenary November 2014 Geneva, Switzerland.
Toolkit for Mainstreaming HIV and AIDS in the Education Sector Guidelines for Development Cooperation Agencies.
THE GEO GLOBAL CAPACITY BUILDING SYMPOSIUM Seville (Spain) September 10 & 11, 2007.
© GEO Secretariat GEO Work Plan 2nd GEOSS Science and Technology Stakeholder Workshop "GEOSS: Supporting Science for the Millennium Development.
Commissioning Self Analysis and Planning Exercise activity sheets.
Harnessing a multi-stakeholder platform for improved land governance in Malawi Ivy Luhanga – Principal Secretary, Paul Jere – Land Governance Consultant,
MAINSTREAMING MONITORING AND EVALUATION IN EDUCATION Can education be effectively managed without an M & E system in place?
1. Housekeeping Items June 8 th and 9 th put on calendar for 2 nd round of Iowa Core ***Shenandoah participants*** Module 6 training on March 24 th will.
Science and Technology Committee GGOS, December 2006.
GEO Work Plan Symposium 2012 ID-03: Science and Technology in GEOSS ID-03-C1: Engaging the Science and Technology (S&T) Community in GEOSS Implementation.
Capacity Building Committee Architecture and Data Committee Meeting Seattle – July 2006.
© GEO Secretariat Work Plan Guidance document 2 nd Capacity Building Committee Meeting, Brussels September 2006.
GEO Strategic Plan : Implementing GEOSS IMPLEMENTATION PLAN WORKING GROUP GEO EXECUTIVE COMMITTEE MARCH 2015 GENEVA, SWITZERLAND.
School Improvement Partnership Programme: Summary of interim findings March 2014.
ST-09-01: Catalyzing Research and Development (R&D) Funding for GEOSS Florence Béroud, EC Jérome Bequignon, ESA Kathy Fontaine, US ST Kick-off Meeting.
Preparation of the 2010 GEO Ministerial Summit GEO Science and Technology Committee Preparation of the 2010 GEO Ministerial Summit Gilles OLLIER Head of.
Developing Consensus Principles and Standards for Evaluating Global and Regional Partnership Programs (GRPPs) Progress Report to the Fifth Meeting of the.
Tracking national portfolios and assessing results Sub-regional Workshop for GEF Focal Points in West and Central Africa June 2008, Douala, Cameroon.
Systems Accreditation Berkeley County School District School Facilitator Training October 7, 2014 Dr. Rodney Thompson Superintendent.
Introducing the Science and Technology Roadmap 1 st GEO/EGIDA Workshop Bonn, Germany, May 09 th - 11 th, 2011.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
Kathy Corbiere Service Delivery and Performance Commission
Ohio Improvement Process (OIP) Facilitating District-wide Improvement in Instructional Practices and Student Performance.
Technology Needs Assessments under GEF Enabling Activities “Top Ups” UNFCCC/UNDP Expert Meeting on Methodologies for Technology Needs Assessments
NSDI Strategic Plan Update National Geospatial Advisory Committee Meeting December 11, 2013.
The Data Sharing Working Group 24 th meeting of the GEO Executive Committee Geneva, Switzerland March 2012 Report of the Data Sharing Working Group.
Systems Accreditation Berkeley County School District Accreditation Team Chair Training October 20, 2014 Dr. Rodney Thompson Superintendent.
GEO Implementation Boards Considerations and Lessons Learned (Document 8) Max Craglia (EC) Co-chair of the Infrastructure Implementation Board (IIB) On.
Midterm Evaluation of GEOSS Highlights and Key Messages from the Evaluation Report Charles Hutchinson Chair, Evaluation Team GEO Executive Committee Meeting.
Session 2: Developing a Comprehensive M&E Work Plan.
Midterm Evaluation of GEOSS Highlights and Key Messages from the Evaluation Report Charles Hutchinson Chair, GEOSS Midterm Evaluation Team GEO Executive.
WGCapD, CEOS and the UN Sustainable Development Goals (SDGs) Committee on Earth Observation Satellites Deputy CEOS Executive Officer / CSA Marie-Josée.
QA4EO Update on the Quality Assurance Framework For Earth Observation Joint GSICS GDWG-GRWG meeting.
Report from GEO Post-2015 WG and GEO Ministerial WG D. Brent Smith, NOAA CEOS Representative CEOS SIT-28 Meeting Hampton, Virginia, USA 12 March 2013.
Developing a Monitoring & Evaluation Plan MEASURE Evaluation.
Reporting, Monitoring and Evaluation Giovanni Rum, Chao Xing GEO Secretariat GEO Work Programme Symposium Geneva, 2-4 May 2016.
Exploitation means to use and benefit from something. For Erasmus+ this means maximising the potential of the funded activities, so that the results are.
AUDIT STAFF TRAINING WORKSHOP 13 TH – 14 TH NOVEMBER 2014, HILTON HOTEL NAIROBI AUDIT PLANNING 1.
Capacity Building in: GEO Strategic Plan 2016 – 2025 and Work Programme 2016 Andiswa Mlisa GEO Secretariat Workshop on Capacity Building and Developing.
GEO Strategic Plan : Implementing GEOSS Douglas Cripe GEO Work Programme Symposium 2-4 May 2016, Geneva.
Evaluation in the GEF and Training Module on Terminal Evaluations
CEOS Response to IPWG on Draft GEO Strategy
WG Title Goes Here Name (s), Organization, CEOS Affiliation
Presentation transcript:

Midterm Evaluation of GEOSS Highlights and Key Messages from the Evaluation Report Charles Hutchinson Chair, GEOSS Midterm Evaluation Team GEO Executive Committee Meeting Geneva, Switzerland XX July 2010

Introduction Evaluation Purpose The evaluation satisfies two needs: – Allows “midcourse” adjustments to GEOSS implementation – Establishes a reference point for future evaluations 2

Methodology A wide array of data sources informed the evaluation: Key Informant Interviews GEOSS User Survey GEO Work Plan Progress Reports Work Plan/Strategic Target alignment review Documentation and Literature review Case Studies of select Tasks 3 Terms Used- “key informants” - the individuals with active participation in GEO who participated in personal interviews “survey respondents” - individuals on the provider-user spectrum who provided input via web-accessible survey “stakeholders” - a combination of key informants, survey respondents and authors of the documents consulted during the evaluation.

Key Findings 1.GEOSS represents an important new Earth observation community and network. GEOSS has raised visibility of the importance and need for integrated global Earth observations. 2.Current GEOSS implementation sufficiently reflects high-level ministerial priorities including those contained in the Cape Town Declaration. 3.Stakeholders are generally positive about the foundation that has been established and optimistic that appropriate outcomes are being realized. 4

Key Findings 5.GEOSS implementation has created a path to enable full and open data sharing and lowered discussion barriers. 6.GEOSS implementation has resulted in positive outcomes for the Earth observation community, such as Data Sharing Principles. 5 4.GEOSS implementation has brought together various organizations and governments to collaborate and support integrated global Earth observations.

Key Findings 8.Survey respondents had an overall “neutral” feeling towards the status of GEOSS development and implementation at this point. 9.Stakeholders perceive that architecture developed by GEO does not yet meet their needs for data, information, and tools. 6 7.GEO has not adequately communicated evidence of progress to show value- added results unique to the implementation of GEOSS and to unequivocally prove a positive return on investment.

In Focus: Finding 9 Stakeholders perceive that architecture developed by GEO does not yet meet their needs for data, information, and tools. 7 Specific Issues: Confusion about Portal testing. QA/QC of material. Direct access to data and information rather than links to more portals or limited products.

Key Findings 10.Stakeholders are concerned about the sustainability of GEOSS with regard to (a) the voluntary nature of GEOSS implementation which has been beneficial up to this point for engaging partners; and, (b) the lack of sufficient resources, both financial and human to sustain efforts into the future. 11.Stakeholders indicated widely varying expectations for GEO and GEOSS, particularly as a source of new funding or a competing operational entity Some stakeholders view current GEO practices as co-opting achievements of contributors and giving them limited or no acknowledgement or credit.

In Focus: Finding 12 Some stakeholders view current GEO practices as co-opting achievements of contributors and giving them limited or no acknowledgement or credit. 9 Examples: GCOS / GEO Harmonization Engagement of UN Agencies GEONETCast Task ST Specific Issues: No standard for crediting material within the GCI (Task ST 09-02). Perceived reluctance to acknowledge existing frameworks which are major pieces of GEOSS. Desire by ALL parties to highlight their own role in activities. Perceived intent to create new GEO frameworks rather than working to support and connect existing, independent systems.

Key Findings 14.The GEOSS implementation approach does not explicitly describe an end-to-end process of how the application of resources supports the overall vision and goals of GEOSS, how or why benefits are expected, or when benefits will be achieved. Without this, it may be difficult for stakeholders to make well-informed decisions about supporting GEOSS GEO has not conducted a comprehensive gap analysis of either their implementation approach (structural) or observation needs (observational).

Recommendations Based on these key findings, the Evaluation Team makes the following 11

Recommendations 1.GEO should develop a long-term strategy to ensure the sustainability of GEOSS beyond GEO must investigate alternative models for sustained resource commitments from Members and Participating Organizations which are necessary for current and future operations GEOSS implementation in the short-term should be guided by an explicit approach linking activities and outputs of the GEO Work Plan to measureable, achievable objectives and strategic targets. This can be accomplished through adopting a logic model and performance measurement strategy.

In Focus: Recommendation 2 GEO must investigate alternative models for sustained resource commitments from Members and Participating Organizations which are necessary for current and future operations. 13 Specific considerations: Means to encourage “follow-through” on pledged support and contributions. Sustaining operations of GEO (i.e. Secretariat) at an effective level. Enabling GEO/GEOSS to support broad participation in GEOSS development (e.g. travel support for developing countries). Ensuring GEOSS will have a lasting presence in Earth observations. Elevating the status of GEOSS activities within contributing program offices.

Recommendations 5.GEO must improve its efforts in communication and outreach through: a)Clarifying their purpose to the stakeholder community; b)Enhancing clarity and traceability of GEO processes; c)Providing evidence of value-added results through GEOSS, and; d)Engaging a wider audience beyond those directly involved in GEOSS implementation GEO should clarify its role as a supporting and enabling platform by facilitating and providing value through coordination among existing Earth observation systems and developing an information networks system.

Recommendations 6.GEO should act to improve its understanding, engagement, and responsiveness to the user community by: a)undertaking a detailed characterization of its current users in order to strengthen and expand the user base; and, b)increasing opportunities for dialogue with the user community to provide helpful feedback on a timely basis. 15

Recommendations 7.GEO should conduct comprehensive observational and structural gap analyses as anticipated in the 10-Year Implementation Plan and Strategic Targets document. 8.GEO should establish clear and consistent mechanisms for properly attributing contributions to eliminate the appearance of co-opting activities. 16

Thank You! We will be happy to answer your questions at this time. If you have additional inquiries about the evaluation, please contact: [Who are we putting here? Is this necessary?] 17 Closing Notes: Electronic records from the evaluation, including planning documents, source material, and working papers have been transferred to the GEO Secretariat. The Evaluation Team also produced a supplementary report on Lessons Learned during the evaluation process with recommendations for future evaluation efforts.

Supplementary Slides 18

Supplementary Slide In Focus: Finding 8 Survey respondents had an overall “neutral” feeling towards the status of GEOSS development and implementation at this point 19 But, Key Informants had a much more positive outlook. Interpretations: The users are poorly-informed. The users have greater expectations for GEOSS. The users have not been engaged in GEOSS development and have little “buy-in.”

Supplementary Slide In Focus: Recommendation 5 GEO must improve its efforts in communication and outreach through: a)Clarifying their purpose to the stakeholder community; b)Enhancing clarity and traceability of GEO processes; c)Providing evidence of value- added results through GEOSS, and; d)Engaging a wider audience beyond those directly involved in GEOSS implementation. 20 Why “must”: Limited reach of the evaluation effort. Need to curb unrealistic expectations for GEOSS (e.g. as a funding mechanism). Need to show results to maintain support. Awareness should grow to understanding, understanding should grow to active engagement.

Supplementary Slide Evaluation Timeline January Evaluation Team established. – Jan. - Team receives guidance from M&E WG. February Development of detailed evaluation framework, data collection methods, and process schedule. March Data collection including interviews, web-based survey, and available literature – Mar. - Team ‘midway’ meeting in Geneva. April Data analysis, additional data collection, report drafting. May Complete draft sent for Secretariat factual review, development of findings and recommendations. – May - Report finalization meeting in Washington. June Report presented to M&E WG. 21

Supplementary Slide Lessons Learned The Evaluation requires dedicated staff support, in this case two full- time Team Members were key to completion. Evaluation Team should include multiple “types”- scientists and evaluators, but also policy makers, GEOSS users and contributors, and other experts as appropriate. Evaluation Teams would benefit from globally equitable composition. Some combination of dedicated project budget, extended timeline, and greater preparatory work by GEO would greatly increase the efficiency and effectiveness of the process. Additional time and resources are needed for outreach and engagement of the Earth observation stakeholder communities (particularly developing countries and marginal or potential users not already deeply involved in GEO). – A particular need is support for non-web-based outreach through print and in-person mechanisms. 22