1. Universal tool of data validation for support of implementation projects of ERP systems
Yuri Byteryakov
July 31, 2017

2. Typical problems that arise during migration
Data from different sources can contradict each other
Manually prepared data requires reconciliation for purity and consistency
Presence of data duplicates
Lack of required analytics in legacy systems
Necessity to monitor changes in legacy systems that arise during the period of data migration
Data migration when implementing new systems requires preparation and data reconciliation

3. Tool to simplify data migration
MAGINFO has developed a tool that simplifies the data reconciliation process and data migration. The tool allows to perform test migrations on real data stored in existing systems at the time of testing, to test migration results and perform adjustment procedures for cleaning and normalizing the data according to migration results. A significant advantage of the tool is a possibility of rapid data preparation ensuring execution of test migrations on real data without stopping the operation of existing legacy systems.

4. Tool to simplify data migration
This tool allows to perform:
Loading and consolidating of data from external sources into your own repository
Validation of prepared data and their consistency based on configurable rules of any complexity
Uploading of verified data in the required format (txt, xls, xlsx, etc.)
Uploading of custom reports on data test results
Save and edit validation rules and report templates for further use
Providing analysts with a tool to verify and prepare data for loading into the target systems

5. Preconfigured SAP validation templates
Currently, there are preconfigured templates (can be set up taking into account the specifics of the Customer) to prepare data when migration to SAP ERP and Oracle EBS:
Material Master Record (MMR, including most angles)
Customer/Vendor Master Record
Sales contract
Procurement contract
Handbook of the parties (model TCA Oracle EBS)

6. Approach to data preparation implemented in the tool
Existing Legacy Systems
Migration Tool
Data Warehouse
Normalization, Data Cleaning, and
Conversion
Target Tables
(Stored in the Migration Tool)
Data Uploading
System 1
Table 1
Target Table 1
System 2
Table 2
Target Table 2
System N
Table N
Normalized Data
Data Enrichment Operator Workplace
Data Quality Specialist Workplace
Target System
Enriched Data
Export
The first stage is data uploading to migrate from legacy systems. Unloading is performed in any format. It is assumed that the format of uploading tables for re-migration will not change.
Uploaded data are stored in the data warehouse of migration tool.
Using custom rules, data cleaning, normalization and conversion to the target format (target tables of a new system) is carried out. The configured rules are stored for use in repeated migrations.
If necessary, manual enrichment of data with missing analysis is carried out. To do this, a workplace of a data enrichment operator is provided in the system. The enriched data will be used in all subsequent cycles of migration, without re-updates.
A data preparation specialist assesses their completeness, quality and consistency. According to evaluation results, adjustments to procedures for cleaning and normalization are carried out, and then a new cycle of testing or data uploading for subsequent import into the target system is triggered.
Data Entry Forms
Data Quality Reports
Data Correction Forms
Interface Tables/API
New Testing Cycle and Data Uploading Triggered if Necessary

7. Screen forms (1/7): Start-up application screen

8. Screen forms (2/7): Form «Data domains»
Block “1” is responsible for navigating along a list of records, in this case, the list of data domains
Block “2” allows to search by various fields including advanced search “Advanced Search”
Block “3” allows to edit the data domain name.
After changing data you should press button “Save”.
To navigate to the templates, press button “Migr.Templates“

9. Screen forms (3/7): Form «Data Preparation Templates»
All forms, including “Data Preparation Templates”, are organized in a uniform manner and have the same navigation area, save and cancel buttons, search area as on the Data domains Form described above. The features:
Button “Process Template” allows to run a data update procedure. In this case, rules for each data source then rules for each of targets will be consistently applied. Sources, target objects and rules will be selected sequentially according to the field “OrderNum”
Button “Migr. Sources” will open the form for editing the data sources bound to this template.
Button “Migr. Targets” will open the form for editing the target tables bound to this template.

10. Form «Data preparation Templates» Process Template Launch
Screen forms (4/7):
Form «Data preparation Templates» Process Template Launch
Template processing starts with button Process Template.
Clicking the button opens the confirmation form.
In the window that appears, it is necessary to confirm start of processing by clicking “Process Template”.
The window can be closed without causing processing by pressing “x” in the upper right corner or button “OK”.

11. Screen forms (5/7): Form «Sources»
Block “1” allows to edit data source settings, where it is possible to create new data sources, delete, change the processing order (Order Num), to include/exclude from the data processing procedure (field "Active Flag")
Block "2" is responsible for the rules of data processing

12. Screen forms (6/7): Form «Sources»
Button «Detach» allows to expand the rule window on full screen of the browser

13. Screen forms (7/7): Form «Targets»
It allows to create, edit, delete target tables associated with the template, and also to add, delete, edit rules bound to the target table.
The result of the work must be completed target tables that can be exported to EXCEL and other formats using rules of data or third-party application uploading.

14. Technical implementation of the tool
J2EE
Private data storage/model
Support for commercial and Open Source DBMS (Database Management System)

15. Variants of cooperation
Providing the tool for data migration (license or SAAS)
Involvement in data migration tasks in projects of enterprise information systems implementation as a subcontractor
Services:
Developing of migration strategy
Configuring of rule sets for data loading from sources
Configuring of rule sets for data transform
Configuring of workplace for an analyst on data quality
Services for data analysis: checking consistency and purity of provided data using pre-configured rules
Providing final data for loading into the target system

16. Why MAGINFO?
We have a professional team with extensive experience in ERP systems implementation
Availability of pre-configured templates
Presence of a specially designed tool to test purity and consistency of the data allows to:
Minimize the number of problems arising from incorrect downloaded data
it increases likelihood of successful launch of the system into industrial operation
Minimize the time for data migration
Improve the quality of data loaded into the new system

17. Maginfo, LLC