1. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 1 Data Bridging An Overview Prepared for DIGIT 07-11- 2001 By Advanced Strategies, Inc.

2. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 2 What is Data Bridging? A process that connects two or more 4 fully specified data stores 4 for a limited time or on an ongoing basis

3. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 3 What is Your Data Bridging Experience?

4. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 4 Foundation Concepts

5. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 5 Why Do We Say Data Stores and Not Data Bases? 4 A Data Store is any facility that is capable of retaining data over time 4 The data stores can be of any type: –Flat files, Relational Databases, … –Manual or automated 4 Bridges can be theoretically built among any collection of data stores

6. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 6 Bridges are used: 4 In Data Conversions & Initial Data Loading 4 As Ongoing Interfaces 4 For sourcing Data Warehouses or Data Marts 4...

7. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 7 Bridging is Not Simple and Is Often Underestimated.

8. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 8 Key Concepts 4 Bridges Can Be Batch 4 Bridges involve: Occurrences of Data Components 4 Bridges Can Be Real-Time

9. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 9 Key Concepts Continued 4 The target of one bridge can be the source of another

10. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 10 The Bridging Process

11. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 11 The Data Bridging Process 4 A bridge involves an integrated collection of activities 4 Each activity performs a specific function

12. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 12 4 The functions can be performed singularly across an entire set of data occurrences; 4 or collectively on a given set of data occurrences The Data Bridging Process

13. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 13 What are the Minimum Processes in a Bridge? Suppose there were multiple choices for possible source stores

14. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 14 Adding Source Selection Suppose we are not confident that the Source Data is pristine (enough)

15. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 15 Adding Source Data Audit Suppose the Source Data is predictably flawed.

16. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 16 Adding Source Data Scrubbing Suppose the Source Data is not in the units, size or format we need Suppose the Target Data needs to be assembled from multiple Sources,...

17. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 17 Adding Transformation Suppose we need to establish an audit trail for the data; or be able to provide the user with information about the source and/or quality of the Target Data.

18. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 18 Adding Logging Suppose we need to assure ourselves and our users that the process has gone as expected and that the Target Data can be trusted

19. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 19 Adding Validation

20. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 20 4 Sourcing 4 Auditing 4 Unloading 4 Loading 4 Scrubbing 4 Transforming 4 Logging 4 Validating Here are the possible activities in our bridging process; Not all are always needed, but all should be carefully considered

21. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 21 The Data Bridging Process - The Framework

22. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 22 A Bridging Methodology A Few Notes

23. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 23 How to Approach a Bridging Effort? 4 As a Fully Formed Subproject: think “Path To A Solution” 4 Ideally, initiated after the target data stores have been designed

24. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 24 Abbreviated Path To a Solution for Bridges 4 Define the bridge, especially the values 4 Analyze the Source, Target and Specify Mapping, Bridging Rules,... 4 Design the Bridge 4 Realize the Bridge 4 Implement & QA the Bridge 4 Use & Maintain the Bridge

25. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 25 Bridges Involve All Aspects of a Business 4 Object/Data - what are the data and relationship occurrences we are moving 4 Process - what are the bridging activities 4 Event - what triggers the bridging activities 4 Location - where is the data and how do we transport it

26. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 26 4 Socio-Political - do we need permission to access the data, are there data practices mandates we must consider, can the data be misinterpreted or misused 4 Enabling-Medium - are there platform conversions required; technology bridging Bridges Involve All Aspects -- Continued:

27. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 27 Summary

28. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 28 The Big Picture: A Summary

29. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 29 4 Data Bridging is not normally trivial or cheap 4 Bridging can make or break many types of projects; it should never be an after thought 4 A rigorous process can help make it manageable 4 Data Bridging can be treated as a series of subprojects Conclusion

30. Copyright 2001 Advanced Strategies, Inc. www.advstr.com 30 Questions?