1. Capacity development 4.0 an update
2018 PARIS21
ANNUAL MEETINGS
5 APRIL 2018

2. 1
Conceptual framework

3. Reminder of sub-group objective:
Define Capacity Development 4.0 considering three levels
Individuals
Organisations
System / Enabling environment
Identify gaps and new challenges

4. Capacity Development 4.0
“The process through which a country’s national statistical system, its organisations and individuals obtain, strengthen and maintain their abilities to collect, produce, analyse and disseminate high quality and reliable data to meet users’ needs”

5. Gaps and overlooked aspects
Political economy of statistical capacity
Soft skills
National ownership

6. New challenges Produce data on new domains
Increase public trust in statistics
Educate citizens in distinguishing reliable from unreliable data

7. CD 4.0 in a nutshell Resources Individual Skills and Knowledge
Organization
Management
Politics and power
System
Incentives
Resources: Means (human, physical, institutional) to produce an output.
Skills and knowledge: Cognitive and non-cognitive abilities (e.g. information processing, teamwork) required by workers to perform their jobs
Management: Combination of skills & knowledge and resources to produce an output. 
Politics & Power: Interactions and relationships between the units (how are they interacting?) 
Incentives: The motives guiding individuals and organisations (why are they interacting?)
The framework is (currently) composed of 45 dimensions depicting capacity.
From: Denney, L. and Mallett, R. with Benson, M. S. (2017) Service delivery and state capacity: findings from the Secure Livelihoods Research Consortium. London: Secure Livelihoods Research Consortium.

8. 2
operationalisation

9. Reminder of objectives:
 Describing current practices in CD  Identifying “success criteria” for CD  Develop guidelines for CD 4.0
The operationalisation subgroup had the task of focusing on the implementation aspects of capacity development 4.0, such as providing recommendations on how to improve data literacy or fostering soft skills. However, we first needed to get a clear picture of what is currently being done and identify what would constitute a successful CD programme.
We developed the Capacity Development 4.0 group together with the HLG-PCCB and reviewed secondary sources that described current practices in capacity development. As a group we gathered examples of our own programmes and mapped them to the Framework that Shaida has described.
We identified success criteria from the various organisations that worked in capacity development and we defined a preliminary list.
We were not able to develop guidelines for CD 4.0 because we first needed to come to an agreement between the three groups of what CD 4.0 is and how to measure capacity deficiencies in order to design programmes that tackle them. The idea of today’s breakout session is to start working towards them.

10. Current practices Mainly technical support
training on technical skills
data collection exercises
alignment with international standards
Usually no clear objectives for capacity development
Little attention to data use
We identified the characteristics of current practices through the survey, literature review and our own collective thinking in the Workshop (December).
The main three types of capacity development are:
-Training on technical skills (as you can see in the picture, the individual skills and knowledge category had many post-its).
-Data collection is another privileged area, many times CD focuses on providing budgetary support for finishing surveys or censuses, and there are various international survey programmes, such as IHSN. This is the organisational resources category.
-Finally, alignment with international standards mainly takes the form of international workshops. Staff are trained in how to calculate supply and use tables following SNA2008, for example. This can also be observed in the picture, in organisational skills and knowledge.
Overall, the type of support given to countries is mainly technical (or budgetary).
In general, not enough attention is given to the expected outcomes of capacity development (e.g. greater monitoring and evaluation).

11. Current practices (II)
Funding differs between regions
Percentage of NSO employees trained in 2017
and
others
Proportion of trained employees varies between regions
The survey on capacity development 4.0 allowed us to understand how training is implemented in the various regions. We find that while Western Europe and others receive 6% of international support, African countries depend largely on international funds for training their staff. Still, less than a half of the respondents replied to this question. Several were not able to report on international funds.
We also found that the proportion of trained employees differs largely across regions. While Western Europe and others have 77% of their staff trained, only 16% of NSO staff from Asia and Pacific were trained.
Sources of budget for training in 2017 (Relative frequency)

12. Success criteria Accountability for project management
Effective absorption of new knowledge and/or processes
Comprehensive approach
Adaptability to the local environment
Project sustainability (outcomes and financing)
These success criteria were identified by the Task Team members at the workshop in December.
 Accountability practices for project management and the incorporation of results-based monitoring.
 Effective absorption of new knowledge and/or processes, which can be improved by targeting a country’s innovation capability.
 A comprehensive approach to capacity development, which goes beyond technical competencies to include soft skills, changes to organisational structures and to address institutional constraints where needed.
 Adaptability to the local environment, respecting organisational practices, culture and the work environment.
 Project sustainability, both in terms of outcomes (where tangible and measurable results can be defined for the medium term) and of financing.

13. 3
measurement

14. Reminder of objectives:
 Identify how capacity is currently measured
Detect overlaps
Detect gaps in measurement
 Propose ideas for solving gaps
 Improve current questions and reduce country burden
The measurement subgroup had the task of identifying what is currently being measured as statistical capacity. This task had two main objectives, first to detect overlaps between existing assessments, in terms of what duplications can be found in existing questionnaires (e.g. what questions are asked over and over again) and second, to find what areas require measurement but are not being fairly covered by existing questionnaires.
In order to start the discussion on how to measure the new areas, we proposed new questions and highlighted those from existing assessments that do enquire about them.
Since most of the results of existing assessments are confidential, and several of them are voluntary, we do not know what countries responded to them and when. Reducing country burden proves a difficult task.
Improving current questions is a task that we were not able to accomplish because of the complexity of comparing a vast number of questions and testing them against each other.

15. Current measurement
Mostly focused on organisational skills and knowledge and systemic resources
Gaps in measurement at the individual level and incentives, as well as skills and knowledge at the systemic level
Individual
Organisational
System
Total
1 - Resources 0% 7%
16%
23%
2 - Skills and knowledge 1%
38%
40%
3 - Management 9%
18%
4 - Politics and power
5 - Incentives 3% 2%
62%
36%
100%
Regarding how capacity is being measured, we compared the questions of 14 existing assessments to our CD 4.0 framework (in the form of this matrix). This allowed to see the gaps and the overlaps between tools.
We have copies of the paper that we drafted as a result of this evaluation. Summarising the results, as a group the assessment tools characterise statistical capacity as predominantly composed of skills and knowledge at the organisational level (specifically, statistical production processes, quality assurance and codes of conduct) and of resources at the systemic level (legislation, principles and institutional setting as well as existing data). There is very little coverage of incentives at any level; of skills and knowledge at the systemic level (data literacy and knowledge transfer); or of the capacity at the individual level in general. This reveals that the understanding of statistical capacity is only partial. While the assessment tools vary according to type, method of administration, and purpose, nevertheless they all portray a similar definition of statistical capacity.

16. Proposals for solving gaps
Existing indicators for “orphan” categories highlighted
E.g. Data literacy: Does the NSO work closely with the media in their training and communication programs? (StatDI)
Ideas for new indicators
E.g. work ethics and motivation: Number of sanctions (following the organisations’ code of conduct) in the past year
These two are examples of ideas for covering the categories that we call orphans. Orphans are those categories that receive less than 1% of the questions. For some of these, existing questionnaires provide a starting point, such as for Data Literacy.
However, there are less questions in the case of the individual level. This is interesting, given that most of the capacity development programmes focus on training employees, still they are left out of the definition of statistical capacity. We proposed indicators to start measuring the capacity at this level.