Presentation is loading. Please wait.

Presentation is loading. Please wait.

Oracle for Data Warehousing

Similar presentations


Presentation on theme: "Oracle for Data Warehousing"— Presentation transcript:

1 Oracle for Data Warehousing

2 Packaged Applications Business Intelligence Performance Management
Today’s Reality Analytics Packaged Applications Business Intelligence Custom Reporting Performance Management Data Replication Data Migration Data Warehousing Data Federation Data Marts Data Silos Data Hubs Data Access SQL Batch Scripts Custom Java Fragmented data silos, data-marts, and data-centric applications are the norm Inconsistent data, risk from data exceptions; untrustworthy data Inaccessible information, lack of re-use Lack of visibility to how information assets are used and leveraged Poor business insight, incomplete information for decision making Data Warehouse Data Mart SAP, Oracle PeopleSoft, Siebel, Custom Apps Files Excel XML OLTP & ODS Systems OLAP 2 2 2

3 What’s the Impact on Your Business?
Fragmented data Silos Untrustworthy Data Inaccessible Information Limited Scalability Difficult to Manage Higher Costs Increased Risk Poor Decisions Fragmented data Silos Higher Costs Increased Risk Poor Decision Making Fragmented data silos, data-marts, and data-centric applications are the norm Inconsistent data, risk from data exceptions; untrustworthy data Inaccessible information, lack of re-use Lack of visibility to how information assets are used and leveraged Poor business insight, incomplete information for decision making 3 3 3

4 Moving toward a Clean Architecture
APPLICATIONS ANALYTICS / APPS SERVERS ORCHESTRATION MASTER DATA BPEL PM CRM APPS SCM APPS WEBSITES FINANCIALS DISTRIBUTION CUSTOM APPS OPERATIONS ERP APPS MDM APPLICATIONS Master Data EPM BI ... BUSINESS INTELLIGENCE CLUSTERS ETL Master Data Data Warehouse ETL ESB

5 Data Warehouse Reference Architecture

6 Oracle’s Integrated Stack
Customer Service Ad Hoc Analysis Interactive Dashboards Performance Management Reporting & Publishing Proactive Detection Data Integration & Management Data Warehousing Business Intelligence Foundation PM and BI Applications Data Mining Storage Compression OLAP Predictive PM Applications BI Applications Partitioning Modeling Data Federation Data Quality ETL/ELT Data Services Database and Middle Tier Servers

7 Oracle: #1 in Data Warehousing
Numbers Show Oracle #1 Database (Again). Oracle Confidential – Do Not Distribute Source: IDC, August 2010 – “Worldwide Data Warehouse Platform Software 2009 Vendor Shares”

8 Oracle Data Warehouse Customers
Retail Communications Financial Services Manufacturing Manufacturing CPG/Healthcare Transportn/Energy Leading companies across virtually every industry have chosen Siebel Analytics. Some of the most sophisticated users of analytics technologies—companies like Procter & Gamble and UPS, for example—have standardized on Siebel as their analytics platform. The benefits these companies are achieving with Siebel Analytics are very significant. Honeywell, for example, has improved customer satisfaction by 30% and increased deal close rate. They have 1,000+ users at all levels of the organization. General Motors is using Siebel Analytics to analyze dealer sales, service, and parts data from 21 disparate data sources, providing 1600 users in the field access to complete business insight they need At Cisco, Siebel Analytics provides 500 top execs with near real-time global picture of business performance and prospects At Royal Bank of Canada -- Integrates over 14 terabytes of data from checking, credit card, and mortgage systems, delivering multiple analytic applications to thousands of users enterprise wide; Millions saved through improved collections and reduced defaults

9 Oracle Exadata Database Machine Extreme Performance
Fastest growing new product in Oracle’s history Server & Storage Integrated Hardware & Software Platform Data Warehousing OLTP Consolidation “After carefully testing several data warehouse platforms, we chose the Oracle Database Machine. Oracle Exadata was able to speed up one of our critical processes from days to minutes.” Brian Camp SVP, Infrastructure Services Knowledge Base Marketing

10 Oracle for Data Warehousing
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression

11 Exadata Hardware Architecture
Scaleable Grid of industry standard servers for Compute and Storage Eliminates long-standing tradeoff between Scalability, Availability, Cost Database Grid Intelligent Storage Grid 8 Dual-processor x64 database servers OR 2 Eight-processor x64 database servers 14 High-performance low-cost storage servers 100 TB High Speed disk, or 336 TB High Capacity disk 5.3 TB PCI Flash Data mirrored across storage servers InfiniBand Network Redundant 40Gb/s switches Unified server & storage network

12 Complete Family Of Database Machines For OLTP, Data Warehousing & Consolidated Workloads
Oracle Exadata X2-2 Oracle Exadata X2-8 Quarter, Half, Full and Multi-Racks Full and Multi-Racks 12

13 Exadata is Smart Storage
Compute Intensive Processing Database Server Compute and memory intensive data processing executes in database servers Fully-parallelized joins and aggregations Bandwidth Intensive Searches Exadata Storage Server IO-bandwidth intensive database operations executes in storage servers Exadata Smart Scans and Exadata Storage Indexes filter out data that is not relevant to a query Database servers and Exadata storage work in conjunction to execute SQL Exadata cell is smart storage, not a complete database node

14 Exadata Database Processing in Storage
Exadata storage servers implement data intensive processing in storage Row filtering based on “where” predicate Column filtering Join filtering Incremental backup filtering Scans on Hybrid Columnar Compressed data Scans on encrypted data Data Mining model scoring 10x reduction in data sent to DB servers is common No application changes needed Processing is automatic and transparent Even if cell or disk fails during a query

15 Exadata Flash Extreme Performance
Oracle Database Machine has 5 TB of flash storage 4 high-performance flash cards in every Exadata Storage Server Smart Flash Cache caches hot data Not just simple LRU Knows when to avoid caching to avoid flushing cache Allows optimization by application table Oracle is the First Flash Optimized Database 15

16 Oracle Database Machine: Optimized for large scans
10 TB of user data Requires 10 TB of IO 1 TB with compression 100 GB with partition pruning Subsecond On Database Machine 20 GB with Storage Indexes 5 GB with Smart Scans 2000X less data needs to be processed

17 What does Extreme Performance mean for your business?
Massive data volumes More granular data Daily data instead of weekly Store data instead of account More history 5 years instead of 1 year New data sources Consumer-level data Entirely new analytics Queries that were never possible now run in minutes Near-real-time data loading

18 Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression

19 Fast data loading Access Transform Load
Performant data loading and transformation with in-database ETL Direct flat file access with external tables Transformation inside the database with SQL and PL/SQL Bulk and trickle load Up to 5 TB/hr of raw data loading on a Database Machine Access Transform Load

20 Transform Data Where Data Resides In-database ETL technology
Extract Load Transform Insert Data Pump Transportable Tablespaces Partition Exchange Loading Change Data Capture Distributed Queries SQL*Loader External Tables Table Functions Multi-Table Insert MERGE DML error logging

21 DBFS - Scalable Shared File System
Database Machine comes with DBFS shared Linux file system Shared storage for ETL staging, scripts, reports and other application files Files stored as SecureFile LOBs in database tables stored in Exadata Protected like any DB data – mirroring, DataGuard, Flashback, etc. 5 to 7 GB/sec file system I/O throughput Load into database using External Tables Using DBFS, it is possible to implement a shared file system in the Oracle Database Machine. An example use of such a shared file system, would be an ETL Staging Area. In this case, data is copied onto the DBFS in the Exadata Storage Servers using traditional OS utilities (like ftp, scp, rcp) and then ETL load tool can load the data from the DBFS into the database tables using the eternal tables interface. ETL Files in DBFS ETL More File Throughput than High-End NAS Filer 21

22 Real Time Data Integration with Oracle GoldenGate
Real time extracts from transactional systems Non-invasive on sources Continuous streaming load into ODS Schema of target Latency in seconds Source 1 EMP DEPT On-Disk Logs Oracle GoldenGate Source 2 EMP DEPT Oracle GoldenGate On-Disk Logs

23 Oracle is uniquely capable of concurrent query and updates
report Concurrent small data loads and queries Looks like... OLTP Oracle's read consistency Readers never block writers Writers never block readers Queries are always consistent and auditable No deadlocks Introduced in Oracle V4 (1982) Teradata: lock rows for reads and writes to obtain consistency Budget table update Rollback Segment Before Image update accurate report

24 Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression

25 Data Warehouse Reference Architecture
Base data warehouse schema Atomic-level data, 3nf design Supports general end-user queries Data feeds to all dependent systems Application-specific performance structures Summary data / materialized views Dimensional view of data Supports specific end-users, tools, and applications

26 Data Warehouse Reference Architecture
IO-bandwidth intensive workloads Random-IO intensive workloads

27 Oracle delivers complete capabilities for tactical warehouse queries
Fast lookups B-Tree indexes Dimensional queries Star query optimizations Bitmap indexes Bitmap join indexes Aggregate management Materialized Views Cube-organized materialized views

28 Workload Management for DW Setting Up a Workload Management System
Define Workloads Filter Exceptions Manage Resources Monitor Workloads Adjust Plans Execute Workloads Monitor Workloads Adjust Workload Plans IORM RAC OEM DBRM Define Workload Plans The RAC piece includes things like: Services Server Pools (Grid Infrastructure) to provide elasticity (add servers to pool to increase memory) Instance Caging (consolidation) © 2010 Oracle Corporation

29 Workload Management Request Queue Execute Assign Ad-hoc Workload
Each request: Executes on a RAC Service Which limits the physical resources Allows scalability across racks Assign Each request assigned to a consumer group: OS or DB Username Application or Module Action within Module Administrative function Ad-hoc Workload Each consumer group has: Resource Allocation (example: 10% of CPU/IO resources) Directives (example: 20 active sessions) Thresholds (example: no jobs longer than 2 min) Reject Downgrade © 2010 Oracle Corporation

30 Workload Management Request Real-Time ETL Batch ETL Analytic Reports
Assign Execute Execute OLTP Requests Ad-hoc Workload Queue Downgrade Reject © 2010 Oracle Corporation

31 Workload Management Request Real-Time ETL Queue R-T 10% Batch ETL
Analytic Reports Analytic Reports 50% Queue Assign OLTP Requests OLTP 5% Reject Downgrade Queue Ad-hoc 25% Ad-hoc Workload Queue © 2010 Oracle Corporation

32 Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression

33 In-database Analytics Bring Algorithms to the Data, Not Data to the Algorithms
Analytic computations done in the database Dimensional analysis Statistical analysis Data Mining Scalability Security Backup & Recovery Simplicity OLAP Statistics Data Mining

34 Deeper insights from Oracle Data Warehouses
Oracle Spatial: GIS data available for analysis and displayed via MapViewer

35 Deeper insights from Oracle Data Warehouses
Oracle OLAP: Hierarchically aware rankings, shares, alerts and time series calculations are easily defined in the cube and queried by OBIEE using simple and efficient SQL

36 Deeper insights from Oracle Data Warehouses
Hierarchically aware rankings, shares, alerts and time series calculations are easily defined in the cube and queried by OBIEE and other tools using simple and efficient SQL Oracle Data Mining: Predictions & probabilities are calculated within database and available for reporting using OBIEE

37 Deeper insights from Oracle Data Warehouses
OLAP Data Mining Spatial In-Database Analytics Deeper insights for business users Pervasive benefits across stack Simple and seamless embedded analytics Extreme performance and scalability

38 Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression

39 Exadata Hybrid Columnar Compression Highest Capacity, Lowest Cost
Data is organized and compressed by column Dramatically better compression Speed Optimized Query Mode for Data Warehousing 10X compression typical Runs faster because of Exadata offload! Space Optimized Archival Mode for infrequently accessed data 15X to 50X compression typical Query Faster and Simpler Backup, DR, Caching, Reorg, Clone Benefits Multiply © 2010 Oracle Corporation

40 More Data Capacity Systems with Equal User Data
Exadata V2 10x Compression Systems with Equal User Data All with Largest Disks, Best Compression Netezza TwinFin 2x to 4x Compression EMC VMAX 3x Oracle Compression Exadata has more disk drives per rack, larger disk drives (2TB) and much better compression. This means that Exadata can hold much more User Data than other systems, and costs much less per user Terabyte. Exadata User Data per SATA rack with 10x compression is 500TB. Teradata has fallen very far behind in compression technology which makes them much more costly for large data environments. Teradata 2580 holds 45 TB per cabinet using max sized 1TB drive and 1.3x compression (taken from Teradata specifications). A single Exadata rack matches the user data capacity of the largest size Teradata (12s cabinet holding 517 TB user data). The flagship Teradata 5600 is hold even less data per rack than the There is approximately a 20:1 Ratio of user data per rack comparing Exadata to Teradata 5600. Netezza Twinfin 32 TB Uncompressed per rack, 128 TB Compressed (assuming their maximum 4x compression). Teradata x Compression

41 Summary

42 Enterprise Availability
Active Data Guard RAC ASM Secure Backup WAN Flashback Online Redefinition GoldenGate Replication Exadata has fully redundant hardware. Redundant servers, redundant storage servers, redundant network. So any component can fail and the system as a whole will keep running. Our measurements to date show that the hardware failure rate is dominated by disk failures. The Oracle database software tolerates failures by continuing to run when various hardware components fail. For example Oracle RAC continues to run after server failures. ASM mirrors data across storage servers so that the failure of a storage server does not cause an outage of the system as a whole. Oracle has unique capabilities for rolling back erroneous changes called flashback. Oracle has unique capabilities for making changes to databases online called online redefinition. All truly highly available systems should have a remote replica. Oracle has the industry’s leading technologies for creating and maintaining remote replica databases. Golden gate provides a powerful symmetric replication capability. Active Data Guard provides an extremely high performance and simple way to create a readable remote replica database. Redundant Hardware Servers, Storage, Network Database Level HA Tolerate failures and changes Real-Time Active Replica © 2010 Oracle Corporation 42

43 First Secure Database Machine
Moves decryption from software to hardware Over 5x faster Near zero overhead for fully encrypted database Queries decrypt data at hundreds of Gigabytes/second © 2010 Oracle Corporation 43

44 Standardized and Simple to Deploy
All Database Machines are the same Delivered Tested and Ready-to-Run Highly Optimized Highly Supportable No unique configuration issues Identical to config used by Oracle Engineering Runs existing OLTP and DW applications Full 30 years of Oracle DB capabilities No Exadata certification required Leverages Oracle ecosystem Skills, knowledge base, people, partners Ready- to-Run Eliminates the complexity of deploying a high performance database system. Database machines are tested in the factory and delivered ready to run. Because all database machines are the same, their characteristics and operations are well known and understood by Oracle field engineers and support. Each customer will not need to diagnose and resolve unique issues that only occur on their configuration. Performance tuning, and stress testing performed at Oracle is done on the exact same configuration that the customer has ensuring better performance and higher quality. Applications do not need to be certified against Exadata. Applications that are certified with Oracle Database 11.2 RAC will run against Exadata. Very few applications need to certify the storage subsystem underneath a database, and Exadata fundamentally is the Oracle Database with a very fast storage subsystem. Deploy in Days, Not Months © 2010 Oracle Corporation

45 Faster than DW Appliances
50 GB/sec! Flash Query Throughput GB/sec Uncompressed Data Single Rack Faster Throughput from Disk Much Faster with Flash 50 TB of data fits in Flash Using 10x Query Compression Effective Query Throughput on compressed data is even higher Hundreds of GB/sec Disk Teradata 2580 Netezza TwinFin 12 Exadata V2 Why is Oracle Faster DB Processing in Storage Smart Flash Cache Faster Interconnect (40Gb/sec) More Disks Faster Disks (15K RPM) Exadata simultaneously scans from Flash and Disk to maximum query throughput. Exadata allows flash directives at the table or partition level easily implementing ILM Note that the limiting factor to query throughput with compressed flash cache data is generally CPU. The high rate of IO generally moves the bottleneck out of the IO system.

46 Oracle Exadata Momentum Rapid adoption in all geographies and industries

47 Oracle Exadata Database Machine: Changes the Game
Database intelligence and massively parallel scaling in the storage tier Using state of the art industry standard hardware Complete, Integrated Data Warehouse, OLTP and consolidation solution High availability Enterprise Security Advanced analytics Innovative new technologies: Hybrid Columnar Compression In-memory parallel execution FlashFire hardware + flash-optimized software

48


Download ppt "Oracle for Data Warehousing"

Similar presentations


Ads by Google