1. Conquering Data Conversion Projects

2. Who is that furry guy anyway? Austin Zellner = presenter 15+ years Information Technology Multiple large data migration projects Recognized patterns to these projects

3. Have you ever read Dune? Data = life force of information systems Data migration least well understood projects by management. High Risk / Low tolerance for error

4. Phases of Data Conversion Discovery Mapping Programming Conversion Post Conversion

5. Discovery Phase Define Success = Guiding Principle Need to know “When” Need to understand options and Scope

6. Discovery Phase – Gathering your forces Get an inventory of your Knowledge Knowledge: Documentation Experts System Self Documentation Source Code

7. Discovery Phase – Intelligence Gathering Based on Guiding Principle: Generate transactions in source system Generate transactions in target system Identify tables / data modified in source Identify tables / data modified in target This will give a rough estimate of what needs to be touched

8. Discovery Phase – Siege Points Identify access to target system Direct DB Access = Flexible / High Risk Import Tools = Consistent / Lower Risk Data Entry = Slow / Least Risk

9. Discovery Phase – Changing Business Practices May need to change business practices Change code Change business process Convert data to support

10. Discovery Phase – Going from Apples to Oranges Differing data models adds Risk Structural differences Conceptual differences Must be accounted for in timeline

11. Discovery Phase – Watch out for Fuzziness Fuzzy data = values relative to the user Fields overridden Value means different things in time Fields co-opted by departments

12. Discovery Phase – Transaction Types Treat Transaction Types as distinct May require specific mapping Source valuable for understanding

13. Discovery Phase – Calculating Mapping Phase Calculating Mapping Phase ( in hours ) Each field in source and target =.25 Knowledge Modifiers: No documentation / source = x2 Apples to Oranges = x3

14. Discovery Phase – Calculating Programming Phase Calculating Programming Phase ( in hrs ) Each table / view = 5 Modifiers: New to any tools = x2 Direct to DB with Triggers = x3 QA = Total x2

15. Discovery Phase – More stuff you have no control over Buck stops here designated Provide pros / cons for decisions May require secrecy

16. Discovery Phase – End of the Beginning What you should have: -Agreement on what “success” is -Delivery date based on calculations -Buck stops here designated -Test system matching source / target -Real test data -QA Team identified

17. Mapping Phase - Overview Mapping is getting the data from the source system to the target so that the “information” is preserved Automation <> Understanding

18. Mapping Phase – Two types of data Business data = describes record System data = record’s state in system

19. Mapping Phase – Practical Approach From Target Perspective A field level listing of tables For each field: Identify source value that fills Business Identify logic that fills System

20. Mapping Phase – Duplicate Data If merging two datasets, watch for dupes 2 types: Duplicate Keys = renumbering Duplicate Values = merging In both cases, requires creating XRef

21. Mapping Phase – System Maintenance Screens Systems often have “global” configuration screens Often have explanations of what obscure codes mean Easier to just manually set in target system than to convert

22. Mapping Phase – When you are done Should have the following at end: -Master document showing field mappings from source to target -Xref for converted keys, values -Notes on logical conversion of system data from source to target

23. Programming Phase - Overview Bringing records from source to target Simplicity is key Build safeguards to protect accidental launch

24. Programming Phase – Example Code One program per table Establish connection to source and target Read in from source -For each column is there anything to be modified? -> Yes -> log old / new value -Write to target Loop until done

25. Conversion Phase - Preparing Plan as if major outage -Coordinate with departments affected -Have black out period for system use -No new records into system after cutoff Central Gatekeeper designated

26. Conversion Phase – Conversion Time Gatekeeper verifies backups in place -All batch jobs / sql tasks / etc. stopped -Initiate the conversion process -Monitor logs for success -Once complete, begin test transactions -Once signed off, start processes -Allow users back in

27. Post Conversion – Work left to do Watch for broken records -Spot fix individual, batch groups Take care around special events -Month end, year end -New transactions, closing transactions The Spice Must Flow