1. Philadelphia, May 2–4, 2005 www.locationintelligence.net Effective use of spatial databases for Enterprise data integration Dr Paul Watson Laser-Scan

2. Philadelphia, May 2–4, 2005 www.locationintelligence.net Overview Motivation - review key drivers for Data Integration Obstacles to Data Integration Data Quality Management Principles –Data Stewardship –Knowledge Management –Enterprise metadata –Rules based processing Methdology – Data Quality Improvement cycle Spatial Business Rules Spatial Knowledge Management – Radius Studio –Rules browser/ rules repository –Conformance check/ report –Reconciliation –Certification

3. Philadelphia, May 2–4, 2005 www.locationintelligence.net Drivers for Data Integration Agile Competitors and Virtual Organisations - Goldman, Nagel and Preiss

4. Philadelphia, May 2–4, 2005 www.locationintelligence.net Obstacles to Data Integration Proprietary Data Silos - data cul de sac Monolithic Information Systems – embedded logic Built-in, Private Data Models – structure/lifecycle Unknown/Unproven Data Quality – KR/KM

5. Philadelphia, May 2–4, 2005 www.locationintelligence.net Enablers For Enterprise Information Architectures Electronic Data Interchange Straight-through Processing Service Oriented Architectures Rules-based Processing Knowledge Representation Standards Data & service integration is the goal

6. Philadelphia, May 2–4, 2005 www.locationintelligence.net Technological Building Blocks Open data access standards Extensible, interoperable platforms Metadata publication/retrieval Data stewardship

7. Philadelphia, May 2–4, 2005 www.locationintelligence.net Data Quality Impact One third of companies have been forced to delay or scrap new systems because of faulty data, and a full 75% have experienced significant problems resulting from data quality issues - PwC 55-70% of CRM and 70% of Data Warehouse project failures are due to data quality issues - Gartner, Meta US business loses $600 billion each year due to data quality problems - Data Warehousing Institute When is data “fit for purpose”?

8. Philadelphia, May 2–4, 2005 www.locationintelligence.net Principles, Tools & Method for DQ Improvement Knowledge Management –Describe the domain/problem independently (from data and systems) Rules-based paradigm –Decouple the problem definition from problem discovery & resolution (what to do about it) Data Quality Improvement Cycle –Employ a process of continuous monitoring & improvement - sustainable interoperability ™

9. Philadelphia, May 2–4, 2005 www.locationintelligence.net Knowledge Management Where is knowledge now? –Embedded in data –Hidden in point applications –Inside people’s heads Store the knowledge/expertise of the organisation where everyone can contribute to it and share it - as enterprise metadata in a database

10. Philadelphia, May 2–4, 2005 www.locationintelligence.net Benefits of a knowledge management approach Explicit/unambiguous – not embedded Inclusive - accommodates domain experts Non-technical – not just for developers Open, distributable – location/applications Auditable – regulatory/governance issues Evolutionary – incremental acquisition, knowledge is refined/grows over time Structured – machine-readable

11. Philadelphia, May 2–4, 2005 www.locationintelligence.net Knowledge Representation RDF OWL SWRL

12. Philadelphia, May 2–4, 2005 www.locationintelligence.net Data Quality as Knowledge Express data requirements as rules (e.g. SWRL) Data quality rules – enterprise metadata - DB Rules metadata can be shared (interpreted and enforced) by many different applications Rules can be used to measure Data Quality - % conforming data instances Rules guide data reconciliation - prioritise Rules can be used to measure quality improvement reliably “Fit for Purpose” = satisfies the DQ rules

13. Philadelphia, May 2–4, 2005 www.locationintelligence.net Rules-based Processing Paradigm Fact – Pattern – Action Given some facts, if they meet any of the patterns/rules, perform the defined action –Declarative – rule separated from processing –Pluggable actions – reporting/ reconciliation DataRule A Report Action A Reconcile Action A

14. Philadelphia, May 2–4, 2005 www.locationintelligence.net Philadelphia, May 2–4, 2005 www.locationintelligence.net make baseline assessment refine rule metadata check data conformance perform data reconciliation data certification data publication define rule metadata define quality mission Data Quality Improvement Cycle SP ☺ e.g. address cleaning

15. Philadelphia, May 2–4, 2005 www.locationintelligence.net Enterprise Integration Gap for Location Spatial data - complex, brittle and surprising Maintenance - manual, expensive, error-prone Adherence to a spatial model is often business critical (e.g. land & property management, utilities) Tools - significant bespoke development, inflexible, built by developers not domain experts Location data - key for BI, data quality chasm, mining Rudimentary IT standards for spatial data Spatial semantics never explicit – KR/KM

16. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial Knowledge Management Same methodology –Spatial business rules as metadata –Conformance checking/reporting –Data Reconciliation – rules driven –Certification/publication –Only detailed data operations change Business Rule - Percentage: 0 ≤ x ≤ 100 Location Rule Spatial (a,b) : coveredBy, contains, withinDistance etc. Empower the domain expert – short “dev.” cycles, stay agile

17. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial Data Quality Rules Intra-feature Constraints physical geometric Data Specifications Ad hoc Rules Inter-feature Constraints proximity topological directional Spatial Rules Discovery

18. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial Rule Types

19. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial Rules Authoring curbline building

20. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial Rule Builder

21. Philadelphia, May 2–4, 2005 www.locationintelligence.net Add Rule Clauses building zoning

22. Philadelphia, May 2–4, 2005 www.locationintelligence.net Spatial & Non-spatial Conditions Fire station building Street centreline “There’s no such thing as a spatial rule”

23. Philadelphia, May 2–4, 2005 www.locationintelligence.net r r g r n n j j a g a High St. Road Road Segment For each Road, the set of Road Segments having the same name as the Road must have the same aggregate geometry as the Road Rules as Knowledge

24. Philadelphia, May 2–4, 2005 www.locationintelligence.net Conformance Checking Spatial Rules Engine Enterprise Messaging Enterprise Spatial DB Enterprise Metadata - Spatial Rules Web Services Clients Asynchronous Messaging Client Browser Client Reporting Solution

25. Philadelphia, May 2–4, 2005 www.locationintelligence.net Data Reconciliation Spatial Rules Engine Enterprise Spatial DB Enterprise Metadata - Spatial Rules Web Services Clients Update A-B-A Clone A-B-A’ Schema Map A-B-C

26. Philadelphia, May 2–4, 2005 www.locationintelligence.net Warehouse Certification Spatial Rules Engine Enterprise Spatial DB Enterprise Metadata - Spatial Rules Web Services Clients

27. Philadelphia, May 2–4, 2005 www.locationintelligence.net Summary Data quality issues in integration are best addressed using knowledge management/rules-based approaches Spatial data quality is no different to any other data quality – standard, interoperable rules are key Enterprise spatial rules engines form a secure base from which to develop open, distributable location enabled applications

28. Philadelphia, May 2–4, 2005 www.locationintelligence.net