Does compliance with the (UK’s) Code of Practice mean that the figures are ‘right’? Q2010, Helsinki Special Session 35, Tuesday 4 May 2010 Richard Laux, UK Statistics Authority

Coverage of presentation The Code of Practice for Official Statistics How we assess compliance with the Code What compliance with the Code means What compliance with the Code does not mean, and how we communicate this.

Assessment - the Authority’s statutory objectives The Authority must: prepare, adopt and publish a Code of Practice for Statistics assess and determine whether the Code of Practice for Statistics has been complied with designate the statistics as ‘National Statistics’, if it is determined that the Code has been complied with (Statistics and Registration Service Act 2007, sections 10, and 12/14)

Code of Practice for Official Statistics (Version 1) Published January 2009 A Preamble explaining the context within which the Code applies Eight Principles Three Protocols Each Principle and Protocol is supported by a statement of related practices (74 in total) Consistent with UN Fundamental Principles, and European Statistics Code of Practice

Assessment – some process issues Desk research on the basis of evidence supplied by the producer body regarding their view of their compliance with the practices of the Code Supplemented by feedback from users (and suppliers) Complemented by our own analysis of the statistical reports, and web-accessibility We don’t examine in detail the nuts and bolts of the production process Designation in all cases so far has been conditional on the implementation of certain Requirements to enhance compliance with specific practices of the Code

What Code compliance means The Code seeks to ensure that: The right statistics are produced (i.e. statistics about the right things) The right methods are used (i.e. statistics are produced in the right ways), and; The right explanation is given (i.e. statistics are presented in the right way). The Code requires statistics to be useful and useable – “produced to a level of quality that meets users’ needs” - it does not specify absolute levels of quality.

What Code compliance does not mean… That we guarantee that the underlying processing systems are robust or resilient That we guarantee that there’s no scope for human error That we can be sure that producers have told us all we need to know That we guarantee that the published figures have been calculated correctly

Why is it important to be clear what Code compliance does and does not mean? Risk to the reputation of the Authority, and the assessment process – for example, if a ‘problem’ becomes apparent after a set of statistics is designated as National Statistics (i.e. is Code compliant) Risk to users – that they might place greater confidence in the statistics than is merited Risk that producers might misunderstand what we have assessed, and might not undertake enough quality assurance, systems development, and so on.

Communicating what Code compliance doesn’t mean (1) (Robustness etc of processing systems) (Scope for human error) If we have concerns or suspicions, we would note these in the Assessment report, neutrally, saying for example “The Assessment team felt that …” As well as Requirements (to enhance compliance) we also make Suggestions (to improve confidence in the statistics)

Communicating what Code compliance doesn’t mean (2) (Not told all we need to know) As a ‘defence’, we use the following form of words: “The producer told us that …” Also, in relation to matters of integrity, we typically say: “No incidents of political pressure or abuse of trust were reported to or identified by the Assessment team in the course of this assessment”

Communicating what Code compliance doesn’t mean (3) (figures calculated correctly) We propose to add to Assessment reports the following text: “Designation as National Statistics should not be taken to mean that the statistics are always correct. For example, whilst the Code requires statistics to be produced to a level of accuracy that meets users’ needs, it also recognises that errors can occur – in which case it requires them to be corrected and publicised”.

Questions Are we taking sufficient steps to communicate what Code compliance does and doesn’t mean? Is it sufficient for us to be transparent? How can we manage better the overlaps between the process of assessment against the Code, and quality management?