Presentation is loading. Please wait.

Presentation is loading. Please wait.

“Quality Measurement at Statistics Austria”

Similar presentations


Presentation on theme: "“Quality Measurement at Statistics Austria”"— Presentation transcript:

1 “Quality Measurement at Statistics Austria”
Tacis High Level-Seminar Vienna, August 31 – September 2, 2005 Session 5: Assuring High Quality Products in Official Statistics by Quality Management Doc. TACIS Statistics Austria Thomas Burg

2 Quality Measurement at Statistics Austria
Thomas Burg

3 Outline of Presentation
Definition of Quality Instruments for Quality Measurement Quality Reports Consequences and user oriented purposes

4 => Different Answers Quality?
What is Quality of a statistical product (I)? When do you think a statistical product is good? This question can be asked to a user citizen customer politician scientist This question can be asked to a producer => Different Answers Quality?

5 Possible Answers What is Quality of a Statistical Product (II)?
It is good, when the source is reliable It has to be accurate It has to be up to date The sample must be representative It must be well documented The variance should be low The concepts must be clear I have no idea

6 => A Statistical Product is a complex system
What is Quality of a Statistical Product (III)? The answers are fuzzy but most of the aspects are justified One of the most important consequences is that the approach must be multidimensional What do we need to evaluate the product Metadata Experience Agreed standards and definitions => A Statistical Product is a complex system

7 What is important for the definition?
Multidimensionality Understandability for user Transparency Process vs. product quality Measurability

8 Definition of Quality in general
ISO /1994 Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.

9 What is specific for official statistics
Large data sets Mandatory surveys (Sometimes) Official character of figures Information provider for policy makers Constraints by public budget

10 Quality of Statistics is defined with reference to the six criteria:
EUROSTAT Definition of Quality of Statistics(I) Quality of Statistics is defined with reference to the six criteria: Relevance Accuracy Timeliness and Punctuality Accessibility and Clarity Comparability Coherence

11 EUROSTAT Definition of Quality of Statistics(I)
These criteria are covering a good part of quality of a statistical product but there remains a residual which seems to be unexplainable (“White Noise”). The criteria are not always equal in their ranking or their weights. => Q= W1*Rel+W2*Acc+W3*Tim+W4*Acl+W5*Comp+W6*Coh+є

12 RELEVANCE Relevance is the degree to which statistics meet current and potential users’ needs. It refers to whether all statistics that are needed are produced and the extent to which concepts used (definitions, classifications etc.) reflects user needs.

13 ACCURACY Accuracy in the general statistical sense denotes the closeness of computations or estimates to the exact or true values.

14 TIMELINESS AND PUNCTUALITY
Timeliness of information reflects the length of time between its availability and the event or phenomenon it describes Punctuality refers to the time lag between the release date of data and the target date when it should have been delivered, for instance, with reference to dates announced in some official release calendar, laid down by Regulations or previously agreed among partners.

15 ACCESSIBILITY AND CLARITY
Accessibility refers to the physical conditions in which users can obtain data: where to go, how to order, delivery time, clear pricing policy, convenient marketing conditions (copyright, etc.), availability of micro or macro data, various formats (paper, files, CD-ROM, Internet…), etc. Clarity refers to the data’s information environment whether data are accompanied with appropriate metadata, illustrations such as graphs and maps, whether information on their quality also available (including limitation in use…) and the extend to which additional assistance is provided by the NSI.

16 COMPARABILITY Comparability aims at measuring the impact of differences in applied statistical concepts and measurement tools/procedures when statistics are compared between geographical areas, non-geographical domains, or over time.

17 COHERENCE The coherence between statistics is orientated towards the comparison of different statistics, which are generally produced in different way and for different primary uses. When originating from a single source, statistics are coherent in that elementary concepts can be combined reliably in more complex ways. When originating from different sources, and in particular from statistical surveys of different frequencies, statistics are coherent insofar as they are based on common definitions, classifications and methodological standards. The messages that statistics convey to users will then clearly relate to each other, or at least will not contradict each other.

18 General aspects of the criteria
They are not measurable by a certain figure Multidimensional for themselves Accuracy: Sampling + nonsampling Offers a wide variety for indicators Measurement sometimes a problem Not all criteria really fit exactly for all products

19 Tradeoffs If you loose quality in one criteria when deliberately improving another criteria we call it a tradeoff between two quality criteria. Timeliness <-> Accuracy Relevance <-> Comparability Timeliness <-> Coherence Accuracy <-> Coherence Etc.

20 Frame for quality reporting
Austrian Federal Statistical Law 2000 Obligations Against EUROSTAT TQM Board was established Desire of our Statistical Council to learn more about our products

21 Historical background
Quality Reports Historical background In 1998 it became clear that the legal status of official statistics will change Innovative projects: SYSQUAST Team of well-proofed statisticians discussed concepts of EUROSTAT Detailed quality reports were requested for some products by EUROSTAT First prototype of quality report was developed (Set of indicators)

22 Indicators for a statistical product
Numerical Binary Textual Restrictions in MS-Access Links to more detailed descriptions

23 Detailed Quality Report (I) Historical aspects and motivation
Requirements by EUROSTAT for some products Desire of Statistical Council to evaluate some key products Fulfill principles of TQM sufficiently Compendium of metadata and description of used methods Achieving a documentation of the whole production process

24 Detailed Quality Reports (II)
Problems Who is paying for it Time consuming Intensive methodological support Discussion which products should be involved

25 Detailed Quality Reports (III)
Publication Strategy Which information should be transferred to the user. Maybe different user classes Standard documentation will contain several quality aspects Standard documentation should be placed on the internet

26 Detailed Quality Reports (IV)
Management Model

27 Relation Standard documentation <-> Quality Report
contains: Detailed Quality Report Metadata Feedback talks Quality Issues Internet, publication

28 Detailed Quality Reports so Far
Plan of establishing quality reports for every relevant product in the next 5 years (since 2003) Structured after subject matters (Directorates) Detailed schedule for receiving the reports Feedback Talks

29 Feedback Talks (I) Every Quarter of a year is planned 2 month in advance Set of possible participants (suggested by subject matter division) One product per month is selected

30 Feedback Talks (II) Quality report is delivered by subject matter division Quality report is discussed by quality management, methods, responsible statisticians Quality Report is embedded in a fixed frame Should contain Important hints Description of methods Quality issues

31 Feedback Talks(III) Fixed procedure Stepping through the report Improvement of the report Improvement of the product Minutes containing all proposals „To do‘s“

32 Thank you for your attention!


Download ppt "“Quality Measurement at Statistics Austria”"

Similar presentations


Ads by Google