Assessing the effectiveness of national statistical systems Some issues and challenges Mohamed TAAMOUTI Director of Statistics, Morocco 1

Outline Effectiveness, what does it mean? How to asses effectiveness ? Assessing conditions of effectiveness Assessing indicators of effectiveness Concluding remarks 2

Effectiveness, what does it mean? Many references : Handbook of Statistical Organization; fundamental principles of official statistics, many papers and reports; … Ivan Fellegi: “ An effective statistical system is characterized by its ability to illuminate issues, not just to monitor them; by its ability to evolve in response to needs; indeed, by its great ability to be aware of priority information needs and by its capability to set priorities ». 3

How to assess effectiveness ? Conditions of sustainable effectiveness : legal framework, independence, formal coordination, etc. Indicators of effectiveness: relevance, coverage, sound methodologies, quality, timeliness, cost effectiveness, etc. 4

How to assess effectiveness ? On the basis of conditions of its sustainability? – Difficult in the context of developing countries. On the basis of the indicators of effectiveness? – Relatively appropriate. 5

Legal framework An appropriate legal framework is a necessary condition. Most developing countries (all Arab countries) have a statistical law. However, What about law enforcement In many DC laws are not fully applicable : – WB “rule of law” indicator : The percentile rank for MENA region for example is 49,9 (over 100) with a score of -0,02 (the percentile rank for OECD : 90,2 with a score of 1,5). 6

Legal framework: Coordination In general coordination is efficient where there is a National Statistics Council allowing the NSS to act as a coherent system. Following a study by the AIRTS, 9 among 19 countries in the Arab region have such a body. Is this always good for effectiveness: It depends. – In some cases an NSC outside the authority of the NSO (which should be the case) may become a source of conflict with the NSO. 7

Legal framework: Coordination When coordination is weak and a large proportion of information is produced outside the NSO we may have a system with two tiers. Effectiveness of the NSS ≠ the effectiveness of the NSO and the NSS may be poor despite an effective NSO : – Implementing some methodologies (ex. Classifications) may become a too difficult and time costly task. 8

Legal framework: Coordination In many countries NSO do not have access to information produced by other components of the NSS (Income tax registers, business registers, etc.). Usually, when there is no institutional mechanism coordination is done informally (bilateral partnerships, personal relationships, NSO as focal points for international programs, etc.). 9

Coordination: NSDS Following WB data, Most DC have an NSDS at different stages (23 planned, 31 being designed, 70 implementation). If real coordination lacks NSDS become difficult to implement. Many countries planned their NSDS without any real commitment and without securing necessary human and financial resources. 10

Legal framework: Independence Following a study by the AITRS (2008, page 40), Independence of the NSO is, in general, not clearly emphasized in statistical laws in Arab countries. This assessment is not, however clearly evidenced in the report. A necessary condition for independence is a clear emphasis is the statistical law, an enforced law. 11

Legal framework : Independence Independence is difficult to assess. Very few can verify the reliability of data produced. In some cases, media play a major role in the perception of this independence and hence the credibility of the NSO or NSS. Independence matters for credibility of the whole system. 12

Independence and credibility Credibility encourages cooperation between users and producers of statistics. – Data produced is trusted and used ; – Assures the quality through the cooperation of respondents (citizens, enterprises, …) by providing reliable information. In Developing Countries, the discrepancy between International agencies estimates and national data is very harmful to the credibility of the NSS. 13

Legal framework: Confidentiality Confidentiality and data (micro data) accessibility, two key elements of a statistical law. Few developing countries (the Arab region) have a clear dissemination policy of micro data. Confidentiality is not well defined even at the international level. Sometimes it is used against accessibility of micro data. Restricted accessibility usually interpreted as “some thing to hide”. 14

Indicators of Effectiveness: coverage Coverage depends on the needs of the country. Basic data is in general produced. The main challenges in this way are related to : – The lack of administrative data makes it difficult and very costly for the NSS to provide data at sub national level. 1 example: Among the 12 DC countries on the STATCAP of WB, only one has a complete vital registration system coverage. – In some Arab countries, it is difficult to extend coverage to some important issues (HIV prevalence, violence against women). 15

Methodology and Timeliness Statistical Capacity indicators (WB) Subscribers to SDDS: 4 Arab countries, 4 in Africa. 6 Arab countries implemented SNA93. No clear quality policy in most developing countries. 16 MethodologyPeriodicity & Timeliness MENA Developing countries

Concluding remarks Quantitative indicators of effectiveness show some improvement globally however a complete assessment is difficult in DC. Many initiatives for NSS assessment, but there is a need for the development of a more complete unified assessment framework. This will promote quality and credibility of NSS and contribute to the development of NSS. 17