CD4.0 Measurement sub group TORs

Questions Are our current TORs still fine? Where do we stand? How can we deliver by end February?

TORs - Objective The subgroup will focus on the quantifiable aspects of measuring capacity. The sub group will look at two areas of measurement: What is being measured and how is it being measured. The sub group will seek to undertake the following: Under the title of: What is being measured, Identify the elements that require measurement Identify what elements are currently being measured by available tools that measure statistical capacity Identify any additional elements to provide for capacity measurement compatible with Capacity Development 4.0 Under the title of: How capacity is being measured, Identify the assessment tools that are currently available and develop criteria for identifying which of the tools are appropriate for measuring statistical capacity Develop an online platform which will be called the Open Assessment Repository (OAR) available for public browsing Enter the selected tools into the OAR Develop a databank of all the questions that are posed in and indicators that are developed from the current assessment/measurement tools Analyse the questions and indicators for similarities, differences, duplications etc. and derive a core set of those questions and indicators Develop and populate a metadata model for describing the available tools (i.e. similar to the Data Documentation Initiative) Address open content issues on using and re-using existing assessments

TORs - Deliverables List of elements which need to be covered to provide for capacity measurement compatible with Capacity Development 4.0 An analysis of the questions posed in and indicators derived from the current assessment/measurement tools (similarities, differences, duplications, etc.) An online platform, available for public browsing, containing the main assessment tools, their documentation available, and the metadata model for describing the available tools Report on open content issues on using and re-using existing assessments A cohesive report on the work accomplished which will be presented at the 2018 PARIS21 Board meeting

TORs – Objective The subgroup will focus on the quantifiable aspects of measuring capacity. The sub group will look at two areas of measurement: What is being measured and how is it being measured. The sub group will seek to undertake the following: Under the title of: What is being measured, Identify the elements that require measurement Identify what elements are currently being measured by available tools that measure statistical capacity Identify any additional elements to provide for capacity measurement compatible with Capacity Development 4.0 Under the title of: How capacity is being measured, Identify the assessment tools that are currently available and develop criteria for identifying which of the tools are appropriate for measuring statistical capacity Develop an online platform which will be called the Open Assessment Repository (OAR) available for public browsing Enter the selected tools into the OAR Develop a databank of all the questions that are posed in and indicators that are developed from the current assessment/measurement tools Analyse the questions and indicators for similarities, differences, duplications etc. and derive a core set of those questions and indicators Develop and populate a metadata model for describing the available tools (i.e. similar to the Data Documentation Initiative) Address open content issues on using and re-using existing assessments

TORs - Deliverables List of elements which need to be covered to provide for capacity measurement compatible with Capacity Development 4.0 {ORPHAN categories} An analysis of the questions posed in and indicators derived from the current assessment/measurement tools (similarities, differences, duplications, etc.) An online platform, available for public browsing, containing the main assessment tools, their documentation available, and the metadata model for describing the available tools Report on open content issues on using and re-using existing assessments A cohesive report on the work accomplished which will be presented at the 2018 PARIS21 Board meeting

Results of OAR review of assessments   Individual Organisational System Total 1 - Resources 0% 8% 16% 25% 2 - Skills & Knowledge 1% 36% 38% 3 - Management 17% 9% 26% 4 - Politics & Power 5 - Incentives 2% 3% 63% 35% 100%

Proposed next steps Orphan categories: propose measurement Without NSO surveying = proxies (With NSO surveying) Assessment metadata : purpose Open content, response burden Agencies to share info: when, where, non-confidential reports Drafting report: current report to be consolidated 15 Feb-28 Feb

Typology of assessment purposes Finalize available draft: purpose; terms of use – confidentiality Keep things descriptive Need for holistic CD4.0 model VS practicality of measurement VS relevance / impact on data prod/use => Tier categories Engage with other communities

Should we refer to the organization to ask about the individuals or vice versa? There are several measurement challenges for individual and system levels Also different employees require different capabilities, same as different users Middle management, is it able to perform and can the top management delegate? Governance mechanisms are relevant How does data provision work? Exhange mechanisms, IT. Probably some categories can be eliminated or merged? Better define some categories Could be time consuming to have an approach at individual level Incentives at individual should be tied to organization/system Relevance of capturing several issues at system level that are not captured in existing questionnaires (e.g. interaction between funders and agencies at national level?) Overlapping of international efforts, how to measure in assessments? How relevant are the contents of workshops to people who assist? Are they the ones who apply? WHO question?