Download presentation
Presentation is loading. Please wait.
Published byMarlene Juliet Bennett Modified over 9 years ago
1
David Besemer, CTO On Demand Data Integration with Data Virtualization
2
2 Agenda State of Enterprise Information The Case for Data Virtualization How Data Virtualization Works Data Virtualization Adoption Patterns © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
3
3 The State Of Enterprise Information More demanding business users Competition drives faster time-to-information Younger staff want more “do-it-yourself” “IT’s challenges are not my problem.” Information overload Exponential data volume growth Omnipresent delivery “Over the top” IT complexity New sources, uses, and enabling technology Layered on byzantine IT infrastructures © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
4
4 Data Management Trends Changing role of the Data Warehouse Data warehouse no longer viewed as only focal point for all data integration Lower latencies required Information needs moving toward real time Rising “fit-for-purpose” storage and processing Appliances, MPP, NoSQL Data Quality being addressed at every layer Source, Consolidation, Virtual, and Visualization Clouds are approaching… Most enterprises looking to leverage cloud computing © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
5
5 Agenda State of Enterprise Information The Case for Data Virtualization How Data Virtualization Works Data Virtualization Adoption Patterns © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
6
6 The Challenge Big DataFilesPackaged Applications Web Services RDBMS BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives Source Data Siloed & Rigid Constant Change Business Solutions Data Integration Challenge © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
7
7 Traditional Physical Data Consolidation Big DataFilesRDBMS Web Services Packaged Applications Enterprise Data Warehouse Physical Data Marts Physical Operational Data Stores Physical Intermediate Stores & ETL Middleware Source Data Business Solutions BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
8
8 Traditional Physical Data Consolidation Big DataFilesRDBMS Web Services Packaged Applications BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives Enterprise Data Warehouse Physical Data Marts Physical Operational Data Stores Physical Intermediate Stores & ETL Middleware Source Data Business Solutions More silos & complexity Slows future IT progress More silos & complexity Slows future IT progress Physical consolidation Forces the business to wait longer for solutions Physical consolidation Forces the business to wait longer for solutions Wait, wait, wait! Uncontrolled data replication Reduced data quality Significant hidden costs Uncontrolled data replication Reduced data quality Significant hidden costs $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ Batch integration Delay real-time information Batch integration Delay real-time information Customer X Invoice UNPAID Customer X Invoice UNPAID Customer X Invoice PAID IN FULL Customer X Invoice PAID IN FULL Batch Data On-Demand Data OLD © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
9
Data Integration Architectures and Patterns: Build a Portfolio to Address the Range of Needs Physical Movement and Consolidation (e.g., ETL) Abstraction/Virtual Consolidation (Data Federation) Change-Capture and Propagation (Replication or Messaging) Common Metadata (Location, Format, Structure, Quality, Meaning) Common Connectivity (Full range of source/target types) BI Tools/Apps. Master Data Mgmt. Operational Apps. Interenterprise Leading organizations support multiple styles of data integration and delivery to address a range of business requirements — breadth enables leverage and agility. Common Design, Admin., Governance
10
10 © 2010 Composite Software, Inc. / Composite Proprietary and Confidential Physical Movement and Consolidation (ETL, CDC) Abstraction / Virtual Consolidation (Data Federation) Middle- ware ETLCDCData VirtualizationEAI / ESB Purpose Attribute How Data Virtualization Differs Synchronization and Propagation (Messaging) DB DB Scheduled Event Driven Application Application DB Application On Demand Event Driven DB DB © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
11
11 Traditional Physical Data Consolidation Big DataFilesRDBMS Web Services Packaged Applications BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives Physical Data Marts Physical Operational Data Stores Enterprise Data Warehouse Physical Intermediate Stores & ETL Middleware © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
12
12 Data Virtualization Increases Agility Big DataFilesRDBMS Web Services Packaged Applications BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards Enterprise Search Physical Data Marts Physical Operational Data Stores Virtual Data Marts Virtual Operational Data Stores Enterprise Data Warehouse Data Virtualization Physical Intermediate Stores & ETL Middleware © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
13
13 Shared Data Services & Relational Views Further Extend Flexibility and Agility Big DataFilesRDBMS Web Services Packaged Applications Physical Data Marts Physical Operational Data Stores Virtual Data Layer Virtual Data Marts Virtual Operational Data Stores Web Data Services & Relational Views Enterprise Data Warehouse Composite Information Server Physical Intermediate Stores & ETL Middleware BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
14
14 A Complete Data Integration Architecture Big DataFilesRDBMS Web Services Packaged Applications BI, CPM, and Reporting Custom and Composite Apps Portals and Dashboards SOA Initiatives Physical Data Consolidation Layer Virtual Data Layer Virtual Data Marts Virtual Operational Data Stores Shareable Data Services & Relational Views Physical Data Marts Physical Operational Data Stores Enterprise Data Warehouse © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
15
15 Forrester Data Management Reference Architecture © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
16
16 Agenda State of Enterprise Information The Case for Data Virtualization How Data Virtualization Works Data Virtualization Adoption Patterns © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
17
17 How Data Virtualization Works – Example Scenario 1) I need to build an application that looks like this… 2) The view or data service needs to look like this… 3) And the data comes from these sources, in these formats… © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
18
18 Composite Information Server Studio Data Discovery and Design Design Steps 1.Discover data and relationships 2.Model individual view/service 3.Validate view/service 4.Modify as required Benefits Faster time to solution Easy to learn and use Extensible / reusable objects Discovery © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
19
19 Composite Information Server Data Virtualization Production Production Steps 1.Application invokes request 2.Optimized query (single statement) executes 3.Deliver data in proper form Benefits Up-to-the-minute data High performance Less replication required Optimizer © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
20
20 Composite Information Server Data Virtualization Production with Caching Production Steps 1.Cache essential data 2.Application invokes request 3.Optimized query (leveraging cached data) executes 4.Deliver data in proper form Benefits Removes network constraints 7-24 availability Optimal performance CacheOptimizer © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
21
21 Agenda State of Enterprise Information The Case for Data Virtualization How Data Virtualization Works Data Virtualization Adoption Patterns © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
22
22 Data Federation DW Extension Cloud Data Integration Data Virtualization Adoption Patterns Data Virtualization Layer Big Data Integration © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
23
23 Data Federation Data Federation for Business Intelligence “My application requires data from multiple incompatible sources.” Project Manager © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
24
24 “My data warehouse does not contain all the data required for the reports we need to build.” Data Warehouse Extension Data Warehouse Extension for 360 o View Data Warehouse Owner © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
25
25 Data Virtualization Layer Data Virtualization Layer for Business & IT Agility “How do I build an agile data layer for easy data access and delivery.” IT Director © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
26
26 Cloud Data Integration Cloud Data Integration for IT Extensibility “I need to integrate data between on- premise systems and applications running in the cloud.” CIO © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
27
27 “More and more of my data now lives in MPP and Hadoop sources. How do I combine big data with traditional data for analysis? Big Data Integration Big Data for Analytics Business Analyst © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
28
28 Data Federation DW Extension Cloud Data Integration Data Virtualization Adoption Patterns Data Virtualization Layer Big Data Integration Semantic Abstraction Federated Query Loose Coupling Caching Location Independence = Data Virtualization © 2011 Composite Software, Inc. / Composite Proprietary and Confidential
29
29 Composite Software Contact For more information please contact: Pamela Sotnick Director, Federal Accounts Mobile 240.460.9566 team@compositesw.com Katy Mann Director, Federal Accounts Mobile 301.452.7042 team@compositesw.com David Besemer CTO dbesemer@compositesw.com
30
Questions
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.