1. Data, Metadata and Quality Management Framework ( Quality and Information Management Framework at the Vietnamese Ministry of Planning and Investment ) Michael Colledge and Bryan Fitzpatrick, Consultants European Technical Assistance Programme for Vietnam (ETV2)

2. 2 Content of Presentation 1.Context – Vietnamese Ministry of Planning and Investment 2.Problems 3.Solutions 4.Quality Concepts and Management 5.Information, Data and Metadata Concepts and Mgt 6.Quality and Information Management Framework 7.Conclusions

3. 3 1. Context: Vietnamese Ministry of Planning and Investment (VMPI) Core business functions Organisation and working environment Inputs and outputs

4. 4 Core business functions as defined in regulations Developing strategies and plans for national socio- economic development − includes analyzing and forecasting economic performance Developing mechanisms and policies for economic management Issuing decisions, instructions, and circulars on planning and investment Managing official development assistance − including developing strategies for attracting, coordinating and reporting

5. 5 Core business functions as defined in regulations (continued) Managing procurement and tendering Managing enterprise and business registration Developing plans for renewal and development of state enterprises Promotion of small and medium size businesses Appraising and monitoring individual investments

6. 6 Core business functions Briefly summarised − planning, monitoring, analysis, forecasting, decision making

7. 7 Organisation and Environment 28 Departments and Centres − fragmented, overlapping responsibilities Rapid evolution − Transition for market economy − No such ministry in western developed countries − Move to role as Ministry of Economy General Statistics Office recently added

8. 8 Information and Data Inputs and Outputs Characteristics of inputs − diverse range − relatively low volume Characteristics of outputs − mostly information rather than data

9. 9 2. Quality Problems and Issues Information and data acquired are poor or unknown quality and/or lacking in timeliness Procedures and facilities for acquiring, processing, analysing and sharing information and data are poor Information and data often acquired and maintained on paper Data sources not fully exploited – duplication of data collected by GSO There are differing versions of what are nominally the same data

10. 10 Quality Problems and Issues (continued) Limited metadata accompanying information and data − data are not well understood Departments/centres receive funding directly from donors and undertake uncoordinated developments − No organisational unit with mandate or sufficient resources to take coordination or leadership role. There is no definitive repository of information and data MPI website is poorly populated and data are error prone

11. 11 Underlying Problems Division of functions amongst departments has resulted in semi-autonomous fiefdoms with little motivation to collaborate No organizational unit responsible for developing, facilitating and monitoring general quality management policies, guidelines and procedures for the MPI as a whole No organizational unit responsible for developing, facilitating and monitoring general information, data and metadata management policies, guidelines and procedures

12. 12 3. Solutions Introduction of the Quality and Information Management Framework Development principles − Simplicity − Use of Standards, Guidelines and Recommended Practices − Harmonisation and Integration

13. 13 Harmonisation and Integration

14. 14 Transformation Process InputsOutputs Figure 1:

15. 15 Transformation Process Inputs Outputs Acquisition Process Distribution Process Repository Figure 2

16. 16 Figure 3

17. 17 Transformation Processes InpuInpu Acquisition Process Distribution Process Repository Figure 4

18. 18 Transformation Processes InpuInpu Acquisition Processes Distribution Processes Repository 1 Figure 5 Repository 2

19. 19 4. Quality Concepts and Management Evolution of quality concepts − Inspection − Quality control − Quality assurance − Total quality management International quality standards − ISO 9000 series − European Foundation for Quality Management (EFQM) Excellence Model

20. 20 ISO 9000 Series ISO 9000: 2005 Quality Management Systems - Fundamentals and Vocabulary − To provide concepts and terminology ISO 9001:2000 Quality Management Systems - Requirements − Basis for certification ISO 9004:2000 Quality Management Systems – Guidelines − for performance improvements in mature system

21. 21 ISO 9000 Quality Concepts Customer focus Leadership Involvement of people Process approach System approach to management Continuous improvement Fact based decision making Mutually beneficial supplier relationships

22. 22 ISO 9001: 2000 Quality Management Systems Basis for certification Comprises five parts: − Quality Management System − Management Responsibility − Resource Management − Product Realization − Measurement, Analysis and Improvement Does not provide implementation guidelines Needs to be interpreted/adapted to particular circumstances

23. 23 Interpretation/Adaptation to VMPI Context ISO Standards primarily designed in context of enterprises selling to market Need to be interpreted/adapted to particular situation Government organisation − not profit based − “user” rather than “customer” − mostly internal users − user cannot influence quality through purchase decisions Whose products are information and data − with structural and reference metadata structural – needed to access product reference – needed to understand its quality

24. 24 Interpretation/Adaptation to VMPI Context (cont) Statistical offices are specialists in information/data Have developed several standards − Eurostat Quality Framework − OECD Quality Framework − IMF Data Quality Assessment Framework − Statistics Canada Quality Framework − Statistics Sweden… Frameworks differ only slightly

25. 25 General Definition of Product Quality Starting point for quality management From customer/user perspective − “Totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs” (ISO 8402 – 1986) − “Degree to which set of inherent characteristics (of product) fulfill requirements (ISO 9000) − “Fitness for Use” Need to specialise to government information agency − by identifying and defining characteristics of quality

26. 26 Characteristics of Information/Data Product Quality Relevance − Degree to which products meet current and potential user needs − Are required information/data available? − Are information/data available required? Accuracy/Reliability − Degree to which information/data correctly describe event/phenomenon/situation they are designed to measure/represent − Many aspects, no single overall measure Timeliness − Length of time between information/data availability and events/ phenomena they describes − Measured relative to time period for which product is likely to be useful

27. 27 Characteristics of Information/Data Product Quality Accessibility − Extent to which users informed of information/data availability − Suitability of format and medium by which product accessed − Cost of access Interpretability − Ease with which user may understand and use informatin/data − Definitions of concepts, coverage and data elements Coherence − Degree to which information/data products are logically consistent and complete

28. 28 Other Quality Considerations Quality dimensions are overlapping and interrelated − cannot be combined into a single indicator Achieving acceptable level of quality is matter of trade-off − for example accuracy against timeliness Cost must also be considered as constraint on quality − quality cost trade-off Product quality achieved through process quality and performance management

29. 29 Process Quality and Performance Management Two aspects to management of processes − Effectiveness – degree to which processes generates products of high quality − Efficiency – minimal use of resources in generating products

30. 30 Figure 6. Process Quality and Performance Management Quality and Performance Characteristics Methodology Management Information /Data Management Technology Management Human Resource Management Effectiveness (Product Quality) Relevance*** Accuracy*** Timeliness**** Accessibility**** Interpretability*** Coherence*** Efficiency (performance) Resource Usage ** * Key: ** primary responsibility; * shared responsibility

31. 31 Quality Concepts - Conclusion In organisation whose primary inputs and outputs and information/data information, data and metadata management is key to quality and performance management

32. 32 5. Information, Data, Metadata Concepts and Management

33. 33 Information and data management concepts and management Terminology Data refers to numerically structured information − typically tables of numbers, graphs and charts Data has to be accompanied by metadata − describing to what the numbers refer and how they have been produced Used in narrow sense information refers material that are not structured − reports, correspondence, pictures, images, etc. Used in broad sense information includes data and metadata

34. 34 Information Metadata Standards Dublin Core ISO 1588-2003 Comprises set of descriptors for any information resource: − title, creator, subject − description, publisher, contributor − date, type, format, identifier − source, language, relation − coverage, and rights Others?

35. 35 (Data) Metadata Standard ISO/IEC 11179: Information Technology Metadata Registries Standard aims to: provide a common understanding of data elements across organizational units and between organizations support re-use and integration of data over time, space, and applications harmonise and standardise data within an organization and across organizations

36. 36 (Data) Metadata Standard ISO/IEC 11179: Information Technology Metadata Registries (continued) Comments: Part 3: Registry metamodel and basic attributes – generalised, complex difficult to understand Part 6: Registration – good starting point, will use simplified version

37. 37 Data and Metadata Standards ISO/TS 17369:2005 SDMX V1.0 To support business practices by enabling efficient exchange and storage of data and metadata Provides format for structuring and reporting metadata, including methodology Provides an agreed structure fordata/metadata flows SDMX V2.0 (not yet ISO standard) Being reviewed by WG2 of ISO TC 154 Enhanced treatment of metadata

38. 38 Figure 7: SDMX Top Level Model

39. 39 SDMX Top Level Model This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources)

40. 40 SDMX Top Level Model This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) This is an instance data or metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period

41. 41 SDMX Top Level Model This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) This is an instance data or metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period Provision Agreements indicate what Providers will provide what subset, when, how often, and how

42. 42 SDMX Top Level Model This is a description of an data or metadata “flow” – an abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by MPI from various sources) This is an instance data or metadata set from a particular provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period Provision Agreements indicate what Providers will provide what subset, when, how often, and how This identifies the Data Providers, giving indicative and contact information and linking to Provision Agreements and actual data and metadata sets

43. 43 SDMX Top Level Model This describes the structure of the data or metadata flow – all the metadata needed to request and understand an instance of the flow (an actual data or metadata set). Links to all other structural metadata. This categorises all the defined data and metadata flows, providing a structuring framework and a basis for searching. Links to other structural metadata.

44. 44 SDMX Top Level Model

45. 45 SDMX Application at VMPI Envisage SDMX Registry/Repository with: Code sets and classifications − environment for standardising and harmonising Data structure definitions − for all regular data sets, i.e., data flows Categorisation schemes − to index data flows Data storage environment for data sets − initially simple file store − possibly a database store − possibly a star-schema store with star schema, design generated automatically from structural metadata provides options for different “cuts” through data flows

46. 46 6. Elements of Quality and Information Management Framework at VMPI A: More Strategic and Integrated Perspective of VMPI Role, Functions and Information: − A1: Review and Revision of Role and Functions − A2: Comprehensive Understanding of Inputs, Processes and Outputs B: Fully Functional Quality and Information Management Programmes: − B1: Establish Quality Management System − B2: Establish Information, Data and Metadata Management Programme

47. 47 Elements of Quality and information management framework (continued) C: Improved Quality and Information Management Infrastructure: − C1: Develop Corporate Quality Management Facilities − C2: Develop Corporate Metadata Management Facilities − C3: Develop Corporate Facilities for Acquisition, Capture, Storage, Access and Dissemination of Data − C4: Develop Corporate Facilities for Acquisition, Storage, Access and Dissemination of Information − C5: Enhance Corporate Planning, Monitoring, Analysis and Forecasting Facilities

48. 48 Elements of Quality and information management framework (continued) D: Continuous Improvement and Reengineering of Business Processes: − D1: Continuous Improvement of Core Business Processes − D2: Reengineering of Core Business Processes E: Comprehensive Quality and Information Management Training Programme: − E1: Develop and Conduct Quality Awareness and Management Training − E2: Develop and Conduct Information, Data and Metadata Management Training − E3. Develop and Conduct Training in Planning, Monitoring, Analysis and Forecasting

49. 49 Conclusions In an organisation whose main inputs and outputs are information and data − Quality management and information/data management go hand in hand − Information/data management can be viewed as component of quality management − Quality management is good umbrella for achieving information/data management Need for recipe describing: − (parts of) international standards that are readily applicable and useful − additional best practices that should be considered