Oracle Data Integration Strategy and Roadmap Oracle Fusion Middleware Product Management

Agenda Introduction to Oracle Data Integration Business Drivers for Data Integration Benefits from a Modern Data Integration Platform Key Oracle Data Integration Products Oracle Data Integration Solution Oracle GoldenGate Overview Data Integrator Overview ODI & GG Together Best of Breed Integration for Batch and Realtime Data Integration Support any Type of Data Integration Use Case Implementing Best-Practice Technical Pattern for Data Warehousing Technical Details – How it Works Demonstration and Q&A (if available)

Data Integration Portfolio I N T R O D U C T I O N

Business Drivers for Data Integration Essential Ingredient for Information Agility Strategic Value of Data Integration Consistency for major enterprise initiatives like BI, DW, & MDM Common technical foundation platform across data silos Central point for data governance, availability and controls Today’s organizations are looking at multiple paradigms for agility either separately or together. In each case, there is a common element among them. Data Integration. Data Integration provides the important aspects for moving data and turning it into re-usable information – it acts as a key enabler for each architecture principle (SOA, BI, or Data Warehousing) and can also act as a key bridge between them. SOA and DI Focus example: For example, lets say I want to implement a process in my Service Oriented Architecture that accesses large amounts of data. Unfortunately that data is part of a data warehouse, to ensure that I only get the updated information that is constantly being synchronized, I’ll need to leverage capabilities in Data Integration and make a call to it from the service in my SOA. If it isn’t connected easily the architects don’t know how to get to it. It also needs to be optimized for performance, the same way that it is from it being managed by my data warehouse. Key Data Integration Use Cases BI, DW, and OLTP Data Integration & Replication SOA, Enterprise Integration & Modernization Migrations and Master Data Management

Benefits from a Modern DI Platform Data Integration is Infrastructure that enables Business Value Do More with Less Design metadata-driven integration Leverage skills & dictate patterns Compete Globally 24X7 Ensure continuous uptime Access data in real time Use Data for Competitive Advantage Ensure the quality of your data Actively govern most valuable asset Automate and Adapt Business Processes Expose data services for reuse Orchestrate processes using SOA

Key Data Integration Products Real Time Data Replication Changed Data Capture DBMS High Availability Disaster Tolerance Heterogeneous E-LT & ETL High-speed Transformations OLAP Data Loading Data Warehouse Loading Comprehensive Integration ELT/ETL for Bulk Data Service Bus Process Orchestration Human Workflow Data Grid Data Service Modeling XQuery Data Federation Data Security/Redaction XA Compliance The Empirix family of products – now called the Application Test Suite (or ATS) comprises three main solutions: the first two tools to automate the actual testing: One for Functional testing – which is exercising the various features of an application as a single user to ensure that everything works The other is for Load testing (also know as performance testing) – which allows customers to drive large amounts of traffic against their applications to ensure they can scale to expected levels. The third solution is Test Process Management – which as the name suggests, allows customers to manage their entire QA process, includes things like: Definition of requirements Developing test cases Tracking defects, etc. These solutions can be sold individually or as a suite – depending on the customer’s need. But they now all fall under the Enterprise Manager Brand – within the SAM bundle, and I’ll describe what that is in a second. One important point to note, though, is that ATS is currently focused on WEB Applications and Web derivatives (such as web services, SOA, AJAX, etc). So from a selling standpoint the immediate sweet spot for you are Oracle customers running or deploying web-based applications (such as Siebel or Peoplesoft) or customer with websites powered by Oracle middleware or databases. For Web-based applications ATS is the best solution on the market – in terms of ease-of-use and feature set and the accuracy of the test conducted. Feel free to contact me offline to learn more about these differentiated capabilities. There is a bunch of stuff on the Enterprise Manager website as well. The roadmap calls for supporting the non-web applications such as EBS - product management will be providing these details and timelines shortly. Business Data / Metadata Statistical Analysis Time Series Reporting Integrated Data Quality Cleansing & Parsing De-duplication High Performance Integrated w/ODI 6

Business Intelligence Oracle Data Integration Solution Best-in-class Heterogeneous Platform for Data Integration MDM Applications SOA Platforms Oracle Applications Business Intelligence Activity Monitoring Custom Applications Comprehensive Data Integration Solution SOA Abstraction Layer Process Manager Service Bus Data Services Data Federation Oracle Data Integrator Oracle GoldenGate Oracle Data Quality ELT/ETL Real-time Data Data Profiling Data Transformation Log-based CDC Data Parsing Bulk Data Movement Bi-directional Replication Data Cleansing Data Lineage Data Verification Match and Merge Oracle already had a comprehensive offering for Data Integration which helped move data in bulk across heterogeneous sources and targets. With the best-in-class ELT/ETL technology of ODI-EE and the best-in-class data cleansing solution: Oracle Data Profiling and Data Quality, these solutions could be integrated together with Service Oriented Architecture approaches to build re-usable data services. Oracle GoldenGate now completes the picture with the addition of real-time change data capture and replication for high availability. Storage Data Warehouse/ Data Mart OLTP System OLAP Cube Flat Files Web 2.0 Web and Event Services, SOA

Data Integration Portfolio G O L D E N G A T E

Oracle GoldenGate Overview Enterprise-wide Solution for Real Time Data Needs Standardize on Single Technology for Multiple Needs Deploy for Continuous Availability and Real-time Data Access for Reporting / BI Highly Flexible Fast Deployments Lower TCO & Improved ROI Disaster Recovery, Data Protection Standby (Open & Active) Zero Downtime Migration and Upgrades Log Based, Real-Time Change Data Capture Operational Reporting Reporting Database OGG ETL ODS EDW ETL Heterogeneous Source Systems Real-time BI EDW Query Offloading Data Distribution 9

How Oracle GoldenGate Works Modular De-Coupled Architecture Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity, transforming the data as required. Trail Trail Capture Pump Delivery LAN/WAN Internet TCP/IP Source Database(s) Target Database(s) Bi-directional 10

Data Integration Portfolio D A T A I N T E G R A T O R

Oracle Data Integrator Enterprise Edition Optimized E-LT for High Performance, Productivity and Low TCO Legacy Sources E-LT Transformation vs. E-T-L Any Data Warehouse Application Sources Declarative Set-based design Change Data Capture Any Planning System Main Points: ODI-EE is a the strategic ETL tool from Oracle; it’s ELT based which is unique in the industry, it combines the offering from both ODI and OWB. Oracle Data Integrator Enterprise Edition delivers unique next-generation Extract Load and Transform (E-LT) technology that improves performance, reduces data integration costs, even across heterogeneous systems. Oracle Data Integrator offers the productivity of a declarative design approach, as well as the benefits of an active integration platform for seamless batch and real-time integration. In addition, Knowledge Modules provide modularity, flexibility, and extensibility. Oracle Data Integrator Enterprise Edition is optimized for Oracle Database to provide real-time data warehousing with advanced ETL/ELT and data modeling. Oracle Data Integration Enterprise Edition, a foundational component to the Oracle Data Integration Suite is Oracle Fusion Middleware's strategic data integration offering; which has emerged from combining the best of both Oracle Data Integrator (ODI) and Oracle Warehouse Builder (OWB) into a single unified product offering Hot-pluggable Architecture OLTP DB Sources Pluggable Knowledge Modules 12

Business Intelligence & Data Warehouse How ODI Works: E-LT Architecture High Performance, Flexible, Lightweight Architecture Key Architecture Benefits: 100% Java, Open APIs, fast E-LT Packaged Application Business Intelligence & Data Warehouse ODI Agent ODI Agent may be deployed in any part of the architecture LKM IKM JKM A D B C$_0 Embeddable Java Agent consumes very little CPU or RAM E-LT Architecture uses DBMS CPUs, not the Application CPUs Open APIs, Open Metadata, Open XML Knowledge Modules E-LT architecture enables that the agent can be deployed on any part of the source, target system I$ E$ (Errors) LKM C$_1 File C IKM CKM RKM Extract-Load Transform Check-Load

Data Integration Portfolio O D I & G G T O G E T H E R

Oracle Data Integration Solution Best-in-class Heterogeneous Platform for Data Integration Oracle Applications Custom Applications MDM Applications Business Intelligence Activity Monitoring SOA Platforms Comprehensive Data Integration Solution SOA Abstraction Layer Process Manager Service Bus Data Services Data Federation Oracle Data Integrator Oracle GoldenGate Oracle Data Quality ELT/ETL Real-time Data Data Profiling Data Transformation Log-based CDC Data Parsing Bulk Data Movement Bi-directional Replication Data Cleansing Data Lineage Data Verification Match and Merge Oracle already had a comprehensive offering for Data Integration which helped move data in bulk across heterogeneous sources and targets. With the best-in-class ELT/ETL technology of ODI-EE and the best-in-class data cleansing solution: Oracle Data Profiling and Data Quality, these solutions could be integrated together with Service Oriented Architecture approaches to build re-usable data services. Oracle GoldenGate now completes the picture with the addition of real-time change data capture and replication for high availability. Storage Data Warehouse/ Data Mart OLTP System OLAP Cube Flat Files Web 2.0 Web and Event Services, SOA

Oracle E-LT + Real-time Best-of-Breed Data Integration Heterogeneous, Real-time, Non-Invasive, High Performance E-LT, and Low Hardware Costs Traditional ETL + CDC Oracle E-LT + Real-time Invasive Capture on OLTP systems using complex Adapters Transformations in ETL engine on expensive middle tier servers Bulk load to the data warehouse with large nightly/daily batch Continuous feeds from operational systems Non-invasive data capture Thin middle tier with transformations on the database platform (target) Mini-batches throughout the day or bulk processing nightly Extract Trickle Xform Xform GG+ ODI Bulk GG+ ODI Lookup Data Lookup Data Staging Load Heterogeneous 16

Support Any Type of Data Integration Best of Breed means using the Right Tools for the Job! Analytical OLTP Query / Report ODS EDW ODS OLTP OLTP Operational 57% of Survey Respondents say that ETL is preferred technology for Operational DI Projects Key Business Initiatives for Op DI projects Reported: - 38% for improving business operations 34% for data as enterprise asset 35% for governance and compliance programs 40% to IT platform standardization, modernization / centralization “Half of organizations experience continuous OpDI work …Solutions commonly include multiple tool types and users report using combinations of ETL and Replication… Phil Russom, TDWI Operational Data Integration Q2, 09 Report OLTP Old OLTP New OLTP OLTP OLTP OLTP OLTP Heterogeneous Heterogeneous Heterogeneous 17

Replicated Source Tables ODI-EE Integration with GoldenGate Non-invasive Data Capture combined with ODI ELT strengths Key Benefits: Eliminate Overhead  no need for DB API overhead on the Source, or the invasiveness of the ODI J$ objects on the Source system, Automate GoldenGate  automation of GG deployment directly from ODI GUI Provide Common DW Pattern  supplies a common pattern for mini-batch style (non-real-time) DW aggregate loads Source Systems ODI Staging & Target Source DB’s ODI J$ Tables Target EDW Replicated Source Tables Transactional RDBMS Replicated Source Tables ODI CDC Framework Target Tables Source Tables J$ Generate all ODI CDC infrastructure Execute end-to-end CDC ODI Generate all GG deployment files 18

Oracle’s Data Integration Joint Solution Best-of-Breed and Proven Oracle GoldenGate Oracle Data Integrator Enterprise Edition Technology Differentiators: E-LT architecture for best performance of high data volume transformations Lowest latency and highest throughput; non-invasive, low overhead Performance De-coupled architecture; multiple deployment styles; open and extensible Knowledge Module architecture for extensibility and flexible connectivity Extensible & Flexible Embedded Strategy Reinforces Standalone Strategy Leverage Oracle’s Unique Position – Whole Stack, Single Vendor Offer Customers Lower TCO than Competition Can Improve the Product through Challenging Use Cases Establish Market Share & References to Drive Standalone Business Standalone Strategy Reinforces Embedded Strategy Customers Reluctant to Accept Non-Proven Embedded Technology Internal Teams Reluctant to Accept Non-Proven Embedded Technology Upgrading to Full Use Is Attractive for Strong Standalone Product Allows Customers Opportunity to Further Leverage Investment You need both, and only Oracle can do that. Maintain transactional integrity; resilient against interruptions and failures SOA-native, integrated with Fusion MW to fit future enterprise architectures Enterprise 19

ODI & GG Together T E C H N I C A L D E T A I L S

Overview of the ODI KM Framework Pluggable Knowledge Module Architecture Reverse Engineer Metadata Journalize Read from CDC Source Load From Sources to Staging Check Constraints before Load Integrate Transform and Move to Targets Service Expose Data and Transformation Services Reverse Integration w/GoldenGate is here! WS WS WS Staging Tables Load Integrate Services CDC Journalize Check Target Tables Sources Error Tables Benefits Leverage Database Optimizations: Native SQL; Native Functions; Native Loads; Native Journaling / CDC Tailor to an organization’s existing best practices Ease administration work Reduce cost of ownership Main Point: Knowledge Modules are a core architectural element of ODI that help to reduce TCO and improve flexibility and extensibility. Knowledge Modules (KMs) are the core architectural element of ODI. They drive its modularity, flexibility and extensibility. KMs define the technical implementation of integration processes defined using declarative design. They implement the actual data flows and define the templates for code generation across the multiple systems involved in each process. KMs are at the same time generic (they allow the generation of data flows regardless of the transformation rules) and highly specific (the code they generate and the integration strategy they implement are finely tuned for a given technology). A comprehensive library of KMs (100+) is provided with ODI out of the box, but they can be tailored to implement existing best practices (e.g. for highest performance, for adhering to corporate standards, for specific vertical know-how, etc.). ODI’s KM framework helps companies capture and reuse technical expertise and best practices, thus reducing cost of ownership. 21 21

ODI CDC in a Nutshell A General Framework for Change Capture on Source DBs Automatic w/JKMs Journal Tables Capture Services Create Capture Process Start/Stop Capture Process Subscription Services Manage Consistency Sets Register/Un-register Subscriber Consumption Services Consumption Views Consumption Operations Extend Window Lock/Unlock Subscriber Purge Operations Consumers Consumption Services ExtendWindow() PurgeJournals() LockSubscriber() UnLockSubscriber() View: CONT CONTID CUSTNAME CUSTID I003 Thomas S. C003 View: CUST CUSTID CUSTNAME EMPID C002 YAHOO E003 View: EMP EMPID ENAME E001 Joe Celko Subscription Services Journal Tables J$CONT CONTID WID I002 - I003 20 J$CUST CUSTID WID C002 20 J$EMP EMPID WID E001 - 20 19 RegisterSubscriber() AddTableToConsistencySet() Capture Services Capture Process CDC_SET_SUBSCRIBER CDC_SET SUBSCRIBER MIN_WID MAX_WID CDC000 FUSION_BI 10 PILLAR_HCM 9 Source Data Table: CONT CONTID CUSTNAME CUSTID I001 Vijay R. C003 I002 Raghu M. C002 I003 Thomas S. Table: CUST CUSTID CUSTNAME EMPID C001 AT&T E003 C002 YAHOO C003 GOOGLE E002 Table: EMP EMPID ENAME E001 Joe Celko E002 Albert Einstein E003 John Doe

Overview of the Integration Using ODI & OGG Together Transactional RDBMS Source Tables Staging DB Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces Datapump Extract WAN Replicat Source trail files Staging trail files

Scenario: Analytics & Reporting Transactional RDBMS Source Tables Staging DB1 Replicated Source Tables ODI CDC Framework Target DB Target Tables Datapump Staging DB2 WAN Replicat Extract Staging trail files Historic Analytics/ Reporting Datapump Replicat Source trail files WAN Replicated Source Tables Staging trail files Realtime Reporting

Step-by-step How It Works Transactional RDBMS Source Tables Staging DB Target DB Target Tables

1. Replicated tables created with Common Format Designer (using ODI) Transactional RDBMS Source Tables Staging DB Target DB Target Tables Replicated Source Tables

2. Start Capturing Changed Data in Source (OGG Extract process) Transactional RDBMS Source Tables Staging DB Target DB Target Tables Replicated Source Tables ODI CDC Framework Extract Source trail files

3. Initialize Staging and Target Data (with ODI or optionally, OGG) Transactional RDBMS Source Tables Staging DB Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces ODI Interfaces Extract Source trail files

4. Start Replication / Propagate Changes to Target DB (OGG and ODI) Transactional RDBMS Source Tables Staging DB Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces Datapump Extract WAN Replicat Source trail files Staging trail files

Simplified & Productive Development D E M O N S T R A T I O N Show the DT portion of the end-to-end demonstration here. You can either use the viewlet, available here: http://bpel.us.oracle.com/11/collaterals/demo_scripts/END2END/ Or do it live using JDev 11g and the project stored at the above location.