1. Towards a Research Agenda for Official Statistics
Lars Lyberg
Statistics Sweden
ETTS 2009 Brussels
February 18-20, 2009

2. The current scenario An overwhelming number of initiatives
150+ ESS R&D groups (advisory, coordination, executive boards, expert, working, organizing committees, round table, steering, technical, task forces)
Problems with priorities, finishing off, and implementation

3. Examples of initiatives
A mixture of traditional methodological areas and more visionary ideas
Traditional areas: confidentiality, nonresponse, small area estimation, harmonization, response burden
More visionary ideas: model-based statistics, combining data from different sources, more efficient collaboration models such as ESSnet, standardization and evaluation efforts such as rolling reviews, Code of Practice, LEG on Quality, the new Regulation

4. So what is lacking? A concerted effort
A project prioritization process based on strategic goals for the ESS
Recognizing risks
Involving the stakeholders even more
Taking care of all links in the statistics production process (i.e. more on conceptualization, response process, measurement errors, quality control)
Systematic implementation and evaluation of best practices

5. Anything more that is lacking?
The systematic division of labour
NSIs have similar challenges
NIPS, CENEX, ESSnet, sharing results
Capacity building (ESS competence development)
Cooperation with universities and agencies world-wide
Standards

6. A dog’s breakfast

7. Having said that…..
Admittedly the general conditions are difficult and vary a lot
The total openness about ESS problems is admirable
Considerable persistence and success in some work areas
Lots of work laid down on
Subject-matter
Methodology
Strengthening the role of official statistics
Quality
Stakeholder dialogue
Lots of recommendations provided

8. Let’s go back to the ESS basics
ESS shall provide reliable, comparable statistics for EU decision making, monitoring and evaluation
Eurostat shall ensure that methods, criteria and practices within ESS are as harmonized as possible and statistics as comparable as possible
NSIs and MSs shall provide national data and technical expertise and advice
Implicit is that production should be efficient

9. How central are these dimensions in ESS R&D work?
“Reliable” should probably read “accurate”
Small errors
Stable processes that deliver what they are supposed to
Important error sources left behind
Comparative
Hidden under the harmonization label
Generic to ESS but comes out as an “add on”
Efficient production
Central in many R&D projects

10. Accurate data Do not leave any key processes behind
Develop and/or apply standard operating procedures (SOP) (quality assurance)
Validate process performance (quality control)
Allocate assurance and control resources with a risk management perspective
Quantify total survey error (TSE)
How should TSE be interpreted in a comparative setting?
Resources outside ESS: TSE workshop, other specialized networks and conferences, SOPs and guidelines

11. Comparative issues What could and should be standardized (harmonized)?
How to achieve procedural equivalence?
How to adjust concepts so that we get a uniform meaning across countries?
How should differences in funding and methodology resources be handled?
How should national pride and conflicts of interest be handled?
Resources outside ESS: CSDI network, guidelines for comparative research published by U o Michigan and CSDI

12. Controlling Quality Quality Level Main stake-holders
Control instrument
Measures Indicators
Product
User, client
Product specs, SLA
Frameworks, compliance, MSE, user surveys
Process
Survey designer
Process variables, CBM, SOP, standards, checklists, verification
Variation via control charts, other paradata analysis
Organization
NSI, owner, society
Excellence models, ISO, CoP, reviews, audits, self-assessments
Scores, strong and weak points, user surveys, staff surveys

13. Hypothetical project priority matrix
Selection criteria
Significance ratings
Project scoring
Links to ESS’s strategic goals 10 x1
Relates to key process x2
Risk of no useful result x3
Customer satisfaction impact x4
Data availability 9 x5
Measurability x6
Cost vs revenue 8 x7
Time until useable results are achieved 7 x8
Resource availability 5 x9

14. Matrix use Project ideas submitted
Each project suggestion scored by 10x1+10x2+10x3+9x4+……..5x9
Scoring is done by project management group
Rolling R&D plan
Projects executed in ranking order

15. Endnote More search for and use of what is already available
Continue the work on simplified coordination
Collaborate globally
The new Regulation should be operationalized such as Article 10: European statistics shall be developed, produced and disseminated on the basis of uniform standards and of harmonized methods
Emphasize the comparative aspect of ESS
Implementing best practices