General Data Quality requirements (and metrics)

Slides:



Advertisements
Similar presentations
Banking Supervision: Asia-Pacific, America and Europe Perspective Data of General Identification (GCD-like) Taxonomy a help for filling Iñaki Vázquez June.
Advertisements

Professional Behaviour
DAVID M. KROENKE’S DATABASE PROCESSING, 10th Edition © 2006 Pearson Prentice Hall 13-1 COS 346 Day 25.
Data Quality Class 5. Goals Project Data Quality Rules (Continued) Example Use of Data Quality Rules.
Data Quality Class 3. Goals Dimensions of Data Quality Enterprise Reference Data Data Parsing.
Modeling and Evaluation. Evaluation Existing systems Meeting with users Merging views.
Foundation Data Workshop
Employee Central Presentation
App-ID Ad-Hoc Technical Issues TP AppID R02 Group Name: App-ID Ad-Hoc Group Source: Darold Hemphill, iconectiv,
Chapter 4: Managing Information Resources with Databases Copyright © 2013 Pearson Education, Inc. publishing as Prentice Hall Chapter
Mrs. Maninder Kaur 1Maninder Kaur
DATA GOVERNANCE: Managing Access Jeremy Singer Suneetha Vaitheswaran.
Microsoft Access 2000 Creating Tables and Relationships.
S New Security Developments in DICOM Lawrence Tarbox, Ph.D Chair, DICOM WG 14 (Security) Siemens Corporate Research.
1 1 Definitions and basic concepts Statistical Training Course Use of Administrative Registers in Production of Statistics Warzaw 14 – 17 October 2014.
Chapter 1 In-lab Quiz Next week
Normalization (Codd, 1972) Practical Information For Real World Database Design.
Databases and Statistical Databases Session 4 Mark Viney Australian Bureau of Statistics 5 June 2007.
Massella Ducci Teri Italian approach to long-term digital preservation Policies for Digital Preservation ERPANET Training Seminar.
Slide Chapter 5 The Relational Data Model and Relational Database Constraints.
Chapter 4: Managing Information Resources with Databases Copyright © 2013 Pearson Education, Inc. publishing as Prentice Hall Chapter
Data Quality Class 3. Goals Dimensions of Data Quality Data Extraction, Transformation, and Loading Data Cleansing Project.
Use of Administrative Data Seminar on Developing a Programme on Integrated Statistics in support of the Implementation of the SNA for CARICOM countries.
Prepared By Prepared By : VINAY ALEXANDER ( विनय अलेक्सजेंड़र ) PGT(CS),KV JHAGRAKHAND.
Ontology – the benefits trail Matthew West. Why bother with Ontology? 2 Reduced Risk Identify Business Opportunities Responsive to change Increased effectiveness.
Geneva – 18/19 June 2007 EuroGroups Register A pilot study for Eurostat to create a register of multinational enterprise groups John Perry – UK ONS Barry.
Data sources of the EuroGroups Register Presentation by Eurostat
United Nations Oslo City Group on Energy Statistics OG7, Helsinki, Finland October 2012 ESCM Chapter 8: Data Quality and Meta Data 1.
XML DTD. XML Validation XML with correct syntax is "Well Formed" XML. XML validated against a DTD is "Valid" XML.
1 ESSnet on EuroGroups Register 1 EuroGroups Register (EGR) Future technical development Description of technical improvement of the EGR Based on needs.
Framework service contract Lot 4 Project: CDDA in conformity with INSPIRE CDDA – INSPIRE PS mapping conclusions Meeting Darja Lihteneger.
Chapter 3: Relational Databases
CONCEPT DICTIONARY MANAGEMENT Concept Dictionary -This is the fundamental building block of Open MRS. -It contains concepts which are questions and answers/responses.
ASET 1 Amity School of Engineering & Technology B. Tech. (CSE/IT), III Semester Database Management Systems Jitendra Rajpurohit.
14-Sept-11 The EGR version 2: an improved way of sharing information on multinational enterprise groups.
Core elements of GIS Guidance and practical steps toward harmonisation By Albrecht Wirthmann, GISCO, Eurostat 2 nd.
Quality control of the statistical register in the Republic of Belarus Svetlana Nichiporuk Head of Statistical Register Department of National Statistical.
IFS310: Module 10 Database Design - Physical design of files and databases.
PRACE user authentication and vetting Vincent RIBAILLIER, 29 th EUGridPMA meeting, Bucharest, September 9 th, 2013.
Introduction To DBMS.
App-ID Ad-Hoc Technical Issues TP AppID R02
Figure 9.8 User Evaluation Form
Task Force Group For SMIC
ACHIEVING k-ANONYMITY PRIVACY PROTECTION USING GENERALIZATION AND SUPPRESSION International Journal on Uncertainty, Fuzziness and Knowledge-based Systems,
Using Item Attributes in a Make-to-Order Environment
Chapter 6 - Database Implementation and Use
Objective ITY-ADQ ESSIP Plan 2015 Ana Paula FRANGOLHO DPS/PEPR
Use of population registers for vital statistics purposes
Chapter 5: The Art of Ensuring Integrity
Instructor Materials Chapter 5: The Art of Ensuring Integrity
Introduction to MS Access: creating tables, keys, and relationships
IT Data Modeling Basics 2016 Alar Krist Swedabnk Group, Strategy and Architecture, Enterprise Information Architect IT Kolledz, Süsteemianalüüsi.
Entity Based Staging SQL Server 2012 Tyler Graham
COS 346 Day 8.
SDMX Information Model
Chapter 2 Database Environment.
1. Residence; 2. Definition of Ownership.
Instructor Materials Chapter 5: The Art of Ensuring Integrity
ESS.VIP VALIDATION An ESS.VIP project for mutual benefits
Metadata The metadata contains
Chapter 1: Introduction
Data Validation Methods WFD GIS Working Group
Data Warehousing Concepts
«Data exchange in bovine identification database BOVEX»
Instructor Materials Chapter 5: Ensuring Integrity
Accrual accounting practices from a statistical viewpoint
Information for Investment Funds Users (1)
CMPE/SE 131 Software Engineering March 7 Class Meeting
EEO MIS Reporting Equal Employment Opportunity (EEO)
Presentation transcript:

General Data Quality requirements (and metrics) CLARITY (understandable) All data elements have clear business meaning (name and definition) COMPLETENESS (mandatory attributes) All required business attribute values exist in record level FORMAT CONFORMANCY (right business format of attributes) Each attribute must have Right Data type, lenght, min max value, ... Unique IDENTIFIABILITY (non duplicate in technical and in real world level) Every record represents one unique real world event or person or thing REFERENTIAL INTEGRITY (relationships inside one DB, enterprise, world) All important relationships exist between related data records CONSISTENCY (non conflict on record level or cross record level) all similar attribute values are not conflict with each other ACCURACY TO SOURCE All loaded and transformed data is correct according to a source TIMELINESS All data is available on the agreed time

Example of Party DQ Rules and Metrics (set of rules) DQ rule code name Rule description Identify ability Complete ness Referential integrity Consis tency Right Format System ID Party must have unique and non-changing system assigned ID which must identify party internaly inside one data base x   Name Party must have country based name in original language Type Party must have main type from list (Privat, Legal, goverment, financial institution) Reg Code Privat Privat Party must have register code / social security number Reg Code Privat Format Privat party social security number format must be according the country based rules if Reg Country is EE, LT, LV x   Reg Code Legal Legal Party must have register code Reg Country Party must have register country code from list (EE, LV, LT, ...) Reg Code Country code Register code + register country code + party type must be unique and must identify party globaly / externaly Gender Reg code Gender and register code of private party must be consistent if Reg Country is EE, LT, LV  x