Metadata for a Statistical Data Warehouse Lars-Göran Lundell Statistics Sweden Luxembourg 22 September 2011.

Slides:

Advertisements

Similar presentations

Rgu: Information management Management of target student numbers Working group.

Advertisements

SDMX in the Vietnam Ministry of Planning and Investment - A Data Model to Manage Metadata and Data ETV2 Component 5 – Facilitating better decision-making.

Metadata to Support the Survey Life Cycle Alice Born, Statistics Canada Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (METIS) Geneva,

IS 4420 Database Fundamentals Leon Chen. 2 Agenda About yourself About yourself  Name  Major About the instructor About the instructor Syllabus Syllabus.

WJEC Applied ICT Databases – Data Dictionary and Data Types Data Dictionary According to Wikipedia: A data dictionary, as defined in the IBM Dictionary.

Data Management I DBMS Relational Systems. Overview u Introduction u DBMS –components –types u Relational Model –characteristics –implementation u Physical.

1 Chapter 2 Database Environment. 2 Objectives of Three-Level Architecture u All users should be able to access same data u User’s view immune to changes.

Metadata for the S-DWH ‒ an overview Lars-Göran Lundell Statistics Sweden.

Database Administration Chapter 16. Need for Databases  Data is used by different people, in different departments, for different reasons  Interpretation.

Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata.

Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.

MDC Open Information Model West Virginia University CS486 Presentation Feb 18, 2000 Lijian Liu (OIM:

Database Systems: Design, Implementation, and Management Ninth Edition

Data Governance Data & Metadata Standards Antonio Amorin © 2011.

WP.5 - DDI-SDMX Integration

WP.5 - DDI-SDMX Integration E.S.S. cross-cutting project on Information Models and Standards Marco Pellegrino, Denis Grofils Eurostat METIS Work Session6-8.

Case Studies: Statistics Canada (WP 11) Alice Born Statistics UNECE Workshop on Statistical Metadata.

Metadata management and statistical business process at Statistics Estonia Work Session on Statistical Metadata (Geneva, Switzerland 8-10 May 2013) Kaja.

Using ISO/IEC to Help with Metadata Management Problems Graeme Oakley Australian Bureau of Statistics.

SDMX and DDI Working Together Technical Workshop 5-7 June 2013

Statistics Sweden Results from operations in 2006: 146 publications 356 press releases commissions 3,7 million visitors at

Metadata Models in Survey Computing Some Results of MetaNet – WG 2 METIS 2004, Geneva W. Grossmann University of Vienna.

New ways of working at Statistics Sweden – a description with emphasis … on preparatory sub-processes Eva Elvers Statistics Sweden

ESS-net DWH ESSnet DWH - Metadata in the S-DWH Harry Goossens – Statistics Netherlands Head Data Service Centre / ESSnet Coordinator

Metadata Architecture at StatCan MSIS 2008 Luxembourg, April 7-9, 2008 Karen Doherty Director General Informatics Branch Statistics Canada.

Explaining the statistical data warehouse (S-DWH)

What is a schema ? Schema is a collection of Database Objects. Schema Objects are logical structures created by users to contain, or reference, their data.

Chapter(1) Introduction and conceptual modeling. Basic definitions Data : know facts that can be recorded and have an implicit. Database: a collection.

Case Study Statistics Netherlands Max Booleman Statistics Netherlands METIS, 2010.

ESSnet on microdata linking and data warehousing in statistical production: Metadata Quality in the Statistical Data Warehouse.

Identify a Health Problem Qualitative Quantitative Develop Program -theory -objectives -format -content Determine Evaluation -design -sampling -measures.

Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.

Implementation Experiences METIS – April 2006 Russell Penlington & Lars Thygesen - OECD v 1.0.

Database Environment Chapter 2. Data Independence Sometimes the way data are physically organized depends on the requirements of the application. Result:

Database Environment Session 2 Course Name: Database System Year : 2013.

Database Administration

EXPERIENCES FROM DISTRIBUTED REGISTERING OF METADATA IN METAPLUS Klas Blomqvist and Lars-Göran Lundell Statistics Sweden.

Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,

SDMX IT Tools Introduction

2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.

ESS-net DWH ESSnet on microdata linking and data warehousing in statistical production Harry Goossens – Statistics Netherlands Head Data Service Centre.

STRATEGY FOR DEVELOPMENT OF ISIS AND IT STRATEGY IN THE NSI-BULGARIA Main principles, components, requirements.

MetaPlus Klas Blomqvist Statistics Sweden Research and Development – Central Methods

Joseph Lukhwareni Statistics South Africa Reengineering projects focusing on metadata and the statistical cycle Statistics South Africa, South Africa 3-5.

Metadata Framework for a Statistical Data Warehouse

Two-Tier DW Architecture. Three-Tier DW Architecture.

WEB-SUPPORTED STATISTICAL DISSEMINATION PROCESS SERVING STATISTICAL DATA USERS Matjaž Jug, M.Sc.

ESS-net DWH ESSnet on microdata linking and data warehousing in statistical production.

Page 1 Development of Metadata System at Croatian Bureau of Statistics Development of Metadata System at Croatian Bureau of Statistics Presented by Maja.

1 Chapter 2 Database Environment Pearson Education © 2009.

3-1 Modeling Basic Entities DBMS Create Sort Search Addition Deletion Modification Create Sort Search Addition Deletion Modification DBMS is a Software.

Harry Goossens Centre of Competence on Data Warehousing.

Statistical process model Workshop in Ukraine October 2015 Karin Blix Quality coordinator

METADATA MANAGEMENT AT ISTAT: CONCEPTUAL FOUNDATIONS AND TOOLS Istituto Nazionale di Statistica ITALY.

SQA project process standards IEEE software engineering standards

Topic 2 (ii) Metadata concepts, standards, models and registries

SQA project process standards IEEE software engineering standards

THE BNSI EXPERIENCE IN METADATA COLLECTION AND ORGANIZATION

Tomaž Špeh, Rudi Seljak Statistical Office of the Republic of Slovenia

ESSnet on Data Warehousing 4th Workshop Maia Ennok 20th. of March 2013

2. An overview of SDMX (What is SDMX? Part I)

2. An overview of SDMX (What is SDMX? Part I)

Chapter 1: The Database Environment

The Database Environment

ESS VIP ICT Project Task Force Meeting 5-6 March 2013.

Chapter 2 Database Environment Pearson Education © 2014.

Presentation to SISAI Luxembourg, 12 June 2012

Business architecture

Work Session on Statistical Metadata (Geneva, Switzerland May 2013)

7. Introduction to the main SDMX objects for metadata exchange

Presentation transcript:

Metadata for a Statistical Data Warehouse Lars-Göran Lundell Statistics Sweden Luxembourg 22 September 2011

Metadata and Data Metadata are data about data Data are qualitative or quantitative information collected through observation Sources: (derived from) Wikipedia, ISO, METIS

Statistical Metadata and Data Statistical metadata are data about statistical data Statistical data are data from a survey or administrative source used to produce statistics Source: METIS

Metadata for a Data Warehouse Technical metadata Structural information How to physically find and use logical data Process descriptions How data flows in the DW Authentication rules Who may do what? Business metadata Definitions and descriptions Help the end-user interpret and evaluate the data Sources: Kimball, Inmon, others

Metadata for statistics production Structural metadata Act as identifiers and descriptors of the data Identify, use, and process data matrixes and data cubes Names of databases, columns, dimensions Reference metadata Describe the contents and the quality of the data Include conceptual, methodological and quality metadata Source: METIS

Metadata categories A metadata item is either Structural (technical) or reference (business) Other mutually exclusive categories include Active  passive Structured  free-form Standardised  non-standard Centralised  local

The Statistical Data Warehouse A central “statistical data” store for managing all available data of interest, regardless of source, enabling the NSI to: perform reporting; execute analysis; produce the necessary information; (re)use data to create new data/new outputs. Data Warehouse Statistics production Statistical Data Warehouse

Metadata for a Statistical Data Warehouse Emphasis on Active metadata Structured metadata Structural metadata And Process metadata Describe expected or actual outcome of one or more processes using evaluable and operational metrics Quality metadata Source quality, methods used, usability/restrictions Tracing information Which surveys/registers contributed to a specific output? Plus metadata common to all statistics production Reference metadata

Metadata standards in a Statistical Data Warehouse What should be standardised? Contents, formats, repository, software Which level of standards should be used? International/Eurostat, National/NSI, DW internal How should a standard be interpreted? Complete adherence, compatible How strict adherence should be required? Mandatory, recommended Should some components be prioritised? Big bang, evolution

Metadata Quality The more data, the more need for metadata The Statistical Data Warehouse contains lots of data, making it dependent on its metadata Correct, high-quality metadata are vital for its use and governance No metadata  useless data Bad metadata  misused data Good metadata  useful data

Metadata for a Statistical Data Warehouse – what’s next? More detailed descriptions Standards Collection and usage Storage... more