1. 1 DATA QUALITY The general method Data model Non-conform data Corrected data / improved IS Corrected programs Exceptions management measure correct prevent

2. 2 MEASURE DATA QUALITY DB Data acquisition schema ? ? ? ? Treatment ? Extraction system ? ? ?? The data model is the central point for all actions objectivesquestionswhat to measure The data contained in databases are the result of a processing Does the processes (collection, calculation, extraction) respect the structures, relations and data rules? Data compliance with the data model The data must allow the users to process tasks Does the application meet the users requirements ? Compliance of data model with users requirements

3. 3 MEASURE DATA QUALITY data programs data quality model A DB data programs data quality model A DB application B information system quality the organisation model (A+B+ functional links) organistion information system quality application A information system quality real world

4. 4 TO MEASURE DATA QUALITY

5. 5

6. 6

7. 7 DATA QUALITY The general method Data model Non-conform data Corrected data / improved IS Corrected programs Exceptions management measure correct prevent

8. 8 TO CORRECT  For the data  Concept inadequacy  Fields segmentation and normalization  Fields value cleaning  orphan data detection  Occurrences deduplication  For the Information system  Data model and application improvements

9. 9 TO CORRECT

10. 10 TO PREVENT The deployment of the data quality process must allow :  To clean up the bottom of the river punctually  To dam up the arrival of new information flows of doubtful quality

11. 11 DATA QUALITY The general method Data model Non-conform data Corrected data / improved IS Corrected programs Exceptions management measure correct prevent

12. 12 TO PREVENT  Objective : to (re)organize the data flows in order to guarantee a given quality level, so to minimize the corrective process.  Principle : data are products coming from a production line. For this reason, one should apply the quality control principles applied in the industry.  measure at different spots  validation referenced with external world measures  …  Involved the organization (management, administrative process) as well as technology  People and organisation resistance are important to consider

13. 13 TO PREVENT  Technical issue  Program correctionCorrection des programmes  Data dictionary consolidation (complete méta-data)  DB re-engineering  Organizational issue  Identification of the processes and data flows  Identification of the critical points and the responsabilities  Users training  Organizational restructuring : flow

14. 14 SYNTHESIS The added value of the proposed approach Data profiling Reverse- engineering Rules definition Data merge Programs correction Model evolution Data dictionary Exceptions management Concepts precision The data quality steps according to Gartner data profiling standar- disation de duplication cleaningfollow upenrichment measurecorrectprevent Logical Data extraction « Orphan » data detection

15. 15 Synthesis What needs to be done 1 measurecorrectprevent 1How to guarantee the conformity of the data after an IS merge ? 2How to manage « old IS data » with respect with the new data management rules ? 3How to manage the quality of the data flow entering or leaving the IS ? 4How to manage the data rules with respect to the applications ? Data profiling Reverse- engineering To specify and complete the rules To manage the data dictionary To specify and complete the concepts To correct the data 2 Data profiling Reverse- engineering To specify and complete the rules To specify and complete the concepts To correct the data To manage the data dictionary 3 To manage the exceptions To correct the programs Reverse- engineering To specify and complete the rules To specify and complete the concepts 4 Reverse- engineering To specify and complete the rules To specify and complete the concepts To manage the exceptions To correct the programs