Download presentation
1
Oracle for Data Warehousing
2
Packaged Applications Business Intelligence Performance Management
Today’s Reality Analytics Packaged Applications Business Intelligence Custom Reporting Performance Management Data Replication Data Migration Data Warehousing Data Federation Data Marts Data Silos Data Hubs Data Access SQL Batch Scripts Custom Java Fragmented data silos, data-marts, and data-centric applications are the norm Inconsistent data, risk from data exceptions; untrustworthy data Inaccessible information, lack of re-use Lack of visibility to how information assets are used and leveraged Poor business insight, incomplete information for decision making Data Warehouse Data Mart SAP, Oracle PeopleSoft, Siebel, Custom Apps Files Excel XML OLTP & ODS Systems OLAP 2 2 2
3
What’s the Impact on Your Business?
Fragmented data Silos Untrustworthy Data Inaccessible Information Limited Scalability Difficult to Manage Higher Costs Increased Risk Poor Decisions Fragmented data Silos Higher Costs Increased Risk Poor Decision Making Fragmented data silos, data-marts, and data-centric applications are the norm Inconsistent data, risk from data exceptions; untrustworthy data Inaccessible information, lack of re-use Lack of visibility to how information assets are used and leveraged Poor business insight, incomplete information for decision making 3 3 3
4
Moving toward a Clean Architecture
APPLICATIONS ANALYTICS / APPS SERVERS ORCHESTRATION MASTER DATA BPEL PM CRM APPS SCM APPS WEBSITES FINANCIALS DISTRIBUTION CUSTOM APPS OPERATIONS ERP APPS MDM APPLICATIONS Master Data EPM BI ... BUSINESS INTELLIGENCE CLUSTERS ETL Master Data Data Warehouse ETL ESB
5
Data Warehouse Reference Architecture
6
Oracle’s Integrated Stack
Customer Service Ad Hoc Analysis Interactive Dashboards Performance Management Reporting & Publishing Proactive Detection Data Integration & Management Data Warehousing Business Intelligence Foundation PM and BI Applications Data Mining Storage Compression OLAP Predictive PM Applications BI Applications Partitioning Modeling Data Federation Data Quality ETL/ELT Data Services Database and Middle Tier Servers
7
Oracle: #1 in Data Warehousing
Numbers Show Oracle #1 Database (Again). Oracle Confidential – Do Not Distribute Source: IDC, August 2010 – “Worldwide Data Warehouse Platform Software 2009 Vendor Shares”
8
Oracle Data Warehouse Customers
Retail Communications Financial Services Manufacturing Manufacturing CPG/Healthcare Transportn/Energy Leading companies across virtually every industry have chosen Siebel Analytics. Some of the most sophisticated users of analytics technologies—companies like Procter & Gamble and UPS, for example—have standardized on Siebel as their analytics platform. The benefits these companies are achieving with Siebel Analytics are very significant. Honeywell, for example, has improved customer satisfaction by 30% and increased deal close rate. They have 1,000+ users at all levels of the organization. General Motors is using Siebel Analytics to analyze dealer sales, service, and parts data from 21 disparate data sources, providing 1600 users in the field access to complete business insight they need At Cisco, Siebel Analytics provides 500 top execs with near real-time global picture of business performance and prospects At Royal Bank of Canada -- Integrates over 14 terabytes of data from checking, credit card, and mortgage systems, delivering multiple analytic applications to thousands of users enterprise wide; Millions saved through improved collections and reduced defaults
9
Oracle Exadata Database Machine Extreme Performance
Fastest growing new product in Oracle’s history Server & Storage Integrated Hardware & Software Platform Data Warehousing OLTP Consolidation “After carefully testing several data warehouse platforms, we chose the Oracle Database Machine. Oracle Exadata was able to speed up one of our critical processes from days to minutes.” Brian Camp SVP, Infrastructure Services Knowledge Base Marketing
10
Oracle for Data Warehousing
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression
11
Exadata Hardware Architecture
Scaleable Grid of industry standard servers for Compute and Storage Eliminates long-standing tradeoff between Scalability, Availability, Cost Database Grid Intelligent Storage Grid 8 Dual-processor x64 database servers OR 2 Eight-processor x64 database servers 14 High-performance low-cost storage servers 100 TB High Speed disk, or 336 TB High Capacity disk 5.3 TB PCI Flash Data mirrored across storage servers InfiniBand Network Redundant 40Gb/s switches Unified server & storage network
12
Complete Family Of Database Machines For OLTP, Data Warehousing & Consolidated Workloads
Oracle Exadata X2-2 Oracle Exadata X2-8 Quarter, Half, Full and Multi-Racks Full and Multi-Racks 12
13
Exadata is Smart Storage
Compute Intensive Processing Database Server Compute and memory intensive data processing executes in database servers Fully-parallelized joins and aggregations Bandwidth Intensive Searches Exadata Storage Server IO-bandwidth intensive database operations executes in storage servers Exadata Smart Scans and Exadata Storage Indexes filter out data that is not relevant to a query Database servers and Exadata storage work in conjunction to execute SQL Exadata cell is smart storage, not a complete database node
14
Exadata Database Processing in Storage
Exadata storage servers implement data intensive processing in storage Row filtering based on “where” predicate Column filtering Join filtering Incremental backup filtering Scans on Hybrid Columnar Compressed data Scans on encrypted data Data Mining model scoring 10x reduction in data sent to DB servers is common No application changes needed Processing is automatic and transparent Even if cell or disk fails during a query
15
Exadata Flash Extreme Performance
Oracle Database Machine has 5 TB of flash storage 4 high-performance flash cards in every Exadata Storage Server Smart Flash Cache caches hot data Not just simple LRU Knows when to avoid caching to avoid flushing cache Allows optimization by application table Oracle is the First Flash Optimized Database 15
16
Oracle Database Machine: Optimized for large scans
10 TB of user data Requires 10 TB of IO 1 TB with compression 100 GB with partition pruning Subsecond On Database Machine 20 GB with Storage Indexes 5 GB with Smart Scans 2000X less data needs to be processed
17
What does Extreme Performance mean for your business?
Massive data volumes More granular data Daily data instead of weekly Store data instead of account More history 5 years instead of 1 year New data sources Consumer-level data Entirely new analytics Queries that were never possible now run in minutes Near-real-time data loading
18
Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression
19
Fast data loading Access Transform Load
Performant data loading and transformation with in-database ETL Direct flat file access with external tables Transformation inside the database with SQL and PL/SQL Bulk and trickle load Up to 5 TB/hr of raw data loading on a Database Machine Access Transform Load
20
Transform Data Where Data Resides In-database ETL technology
Extract Load Transform Insert Data Pump Transportable Tablespaces Partition Exchange Loading Change Data Capture Distributed Queries SQL*Loader External Tables Table Functions Multi-Table Insert MERGE DML error logging
21
DBFS - Scalable Shared File System
Database Machine comes with DBFS shared Linux file system Shared storage for ETL staging, scripts, reports and other application files Files stored as SecureFile LOBs in database tables stored in Exadata Protected like any DB data – mirroring, DataGuard, Flashback, etc. 5 to 7 GB/sec file system I/O throughput Load into database using External Tables Using DBFS, it is possible to implement a shared file system in the Oracle Database Machine. An example use of such a shared file system, would be an ETL Staging Area. In this case, data is copied onto the DBFS in the Exadata Storage Servers using traditional OS utilities (like ftp, scp, rcp) and then ETL load tool can load the data from the DBFS into the database tables using the eternal tables interface. ETL Files in DBFS ETL More File Throughput than High-End NAS Filer 21
22
Real Time Data Integration with Oracle GoldenGate
Real time extracts from transactional systems Non-invasive on sources Continuous streaming load into ODS Schema of target Latency in seconds Source 1 EMP DEPT On-Disk Logs Oracle GoldenGate Source 2 EMP DEPT Oracle GoldenGate On-Disk Logs
23
Oracle is uniquely capable of concurrent query and updates
report Concurrent small data loads and queries Looks like... OLTP Oracle's read consistency Readers never block writers Writers never block readers Queries are always consistent and auditable No deadlocks Introduced in Oracle V4 (1982) Teradata: lock rows for reads and writes to obtain consistency Budget table update Rollback Segment Before Image update accurate report
24
Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression
25
Data Warehouse Reference Architecture
Base data warehouse schema Atomic-level data, 3nf design Supports general end-user queries Data feeds to all dependent systems Application-specific performance structures Summary data / materialized views Dimensional view of data Supports specific end-users, tools, and applications
26
Data Warehouse Reference Architecture
IO-bandwidth intensive workloads Random-IO intensive workloads
27
Oracle delivers complete capabilities for tactical warehouse queries
Fast lookups B-Tree indexes Dimensional queries Star query optimizations Bitmap indexes Bitmap join indexes Aggregate management Materialized Views Cube-organized materialized views
28
Workload Management for DW Setting Up a Workload Management System
Define Workloads Filter Exceptions Manage Resources Monitor Workloads Adjust Plans Execute Workloads Monitor Workloads Adjust Workload Plans IORM RAC OEM DBRM Define Workload Plans The RAC piece includes things like: Services Server Pools (Grid Infrastructure) to provide elasticity (add servers to pool to increase memory) Instance Caging (consolidation) © 2010 Oracle Corporation
29
Workload Management Request Queue Execute Assign Ad-hoc Workload
Each request: Executes on a RAC Service Which limits the physical resources Allows scalability across racks Assign Each request assigned to a consumer group: OS or DB Username Application or Module Action within Module Administrative function Ad-hoc Workload Each consumer group has: Resource Allocation (example: 10% of CPU/IO resources) Directives (example: 20 active sessions) Thresholds (example: no jobs longer than 2 min) Reject Downgrade © 2010 Oracle Corporation
30
Workload Management Request Real-Time ETL Batch ETL Analytic Reports
Assign Execute Execute OLTP Requests Ad-hoc Workload Queue Downgrade Reject © 2010 Oracle Corporation
31
Workload Management Request Real-Time ETL Queue R-T 10% Batch ETL
Analytic Reports Analytic Reports 50% Queue Assign OLTP Requests OLTP 5% Reject Downgrade Queue Ad-hoc 25% Ad-hoc Workload Queue © 2010 Oracle Corporation
32
Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression
33
In-database Analytics Bring Algorithms to the Data, Not Data to the Algorithms
Analytic computations done in the database Dimensional analysis Statistical analysis Data Mining Scalability Security Backup & Recovery Simplicity OLAP Statistics Data Mining
34
Deeper insights from Oracle Data Warehouses
Oracle Spatial: GIS data available for analysis and displayed via MapViewer
35
Deeper insights from Oracle Data Warehouses
Oracle OLAP: Hierarchically aware rankings, shares, alerts and time series calculations are easily defined in the cube and queried by OBIEE using simple and efficient SQL
36
Deeper insights from Oracle Data Warehouses
Hierarchically aware rankings, shares, alerts and time series calculations are easily defined in the cube and queried by OBIEE and other tools using simple and efficient SQL Oracle Data Mining: Predictions & probabilities are calculated within database and available for reporting using OBIEE
37
Deeper insights from Oracle Data Warehouses
OLAP Data Mining Spatial In-Database Analytics Deeper insights for business users Pervasive benefits across stack Simple and seamless embedded analytics Extreme performance and scalability
38
Oracle Database Machine
Optimized for strategic warehousing 25 GB/sec IO bandwidth, with up to 50 GB/sec with Flash Optimized for real-world data loading Read consistency with the ability to load at 5TB/hr Optimized for operational warehousing Advanced indexing capabilities running at 1M IOPS Optimized for advanced analytics Integrated OLAP, data mining, spatial and statistics Optimized for large data sets 10x user data compression
39
Exadata Hybrid Columnar Compression Highest Capacity, Lowest Cost
Data is organized and compressed by column Dramatically better compression Speed Optimized Query Mode for Data Warehousing 10X compression typical Runs faster because of Exadata offload! Space Optimized Archival Mode for infrequently accessed data 15X to 50X compression typical Query Faster and Simpler Backup, DR, Caching, Reorg, Clone Benefits Multiply © 2010 Oracle Corporation
40
More Data Capacity Systems with Equal User Data
Exadata V2 10x Compression Systems with Equal User Data All with Largest Disks, Best Compression Netezza TwinFin 2x to 4x Compression EMC VMAX 3x Oracle Compression Exadata has more disk drives per rack, larger disk drives (2TB) and much better compression. This means that Exadata can hold much more User Data than other systems, and costs much less per user Terabyte. Exadata User Data per SATA rack with 10x compression is 500TB. Teradata has fallen very far behind in compression technology which makes them much more costly for large data environments. Teradata 2580 holds 45 TB per cabinet using max sized 1TB drive and 1.3x compression (taken from Teradata specifications). A single Exadata rack matches the user data capacity of the largest size Teradata (12s cabinet holding 517 TB user data). The flagship Teradata 5600 is hold even less data per rack than the There is approximately a 20:1 Ratio of user data per rack comparing Exadata to Teradata 5600. Netezza Twinfin 32 TB Uncompressed per rack, 128 TB Compressed (assuming their maximum 4x compression). Teradata x Compression
41
Summary
42
Enterprise Availability
Active Data Guard RAC ASM Secure Backup WAN Flashback Online Redefinition GoldenGate Replication Exadata has fully redundant hardware. Redundant servers, redundant storage servers, redundant network. So any component can fail and the system as a whole will keep running. Our measurements to date show that the hardware failure rate is dominated by disk failures. The Oracle database software tolerates failures by continuing to run when various hardware components fail. For example Oracle RAC continues to run after server failures. ASM mirrors data across storage servers so that the failure of a storage server does not cause an outage of the system as a whole. Oracle has unique capabilities for rolling back erroneous changes called flashback. Oracle has unique capabilities for making changes to databases online called online redefinition. All truly highly available systems should have a remote replica. Oracle has the industry’s leading technologies for creating and maintaining remote replica databases. Golden gate provides a powerful symmetric replication capability. Active Data Guard provides an extremely high performance and simple way to create a readable remote replica database. Redundant Hardware Servers, Storage, Network Database Level HA Tolerate failures and changes Real-Time Active Replica © 2010 Oracle Corporation 42
43
First Secure Database Machine
Moves decryption from software to hardware Over 5x faster Near zero overhead for fully encrypted database Queries decrypt data at hundreds of Gigabytes/second © 2010 Oracle Corporation 43
44
Standardized and Simple to Deploy
All Database Machines are the same Delivered Tested and Ready-to-Run Highly Optimized Highly Supportable No unique configuration issues Identical to config used by Oracle Engineering Runs existing OLTP and DW applications Full 30 years of Oracle DB capabilities No Exadata certification required Leverages Oracle ecosystem Skills, knowledge base, people, partners Ready- to-Run Eliminates the complexity of deploying a high performance database system. Database machines are tested in the factory and delivered ready to run. Because all database machines are the same, their characteristics and operations are well known and understood by Oracle field engineers and support. Each customer will not need to diagnose and resolve unique issues that only occur on their configuration. Performance tuning, and stress testing performed at Oracle is done on the exact same configuration that the customer has ensuring better performance and higher quality. Applications do not need to be certified against Exadata. Applications that are certified with Oracle Database 11.2 RAC will run against Exadata. Very few applications need to certify the storage subsystem underneath a database, and Exadata fundamentally is the Oracle Database with a very fast storage subsystem. Deploy in Days, Not Months © 2010 Oracle Corporation
45
Faster than DW Appliances
50 GB/sec! Flash Query Throughput GB/sec Uncompressed Data Single Rack Faster Throughput from Disk Much Faster with Flash 50 TB of data fits in Flash Using 10x Query Compression Effective Query Throughput on compressed data is even higher Hundreds of GB/sec Disk Teradata 2580 Netezza TwinFin 12 Exadata V2 Why is Oracle Faster DB Processing in Storage Smart Flash Cache Faster Interconnect (40Gb/sec) More Disks Faster Disks (15K RPM) Exadata simultaneously scans from Flash and Disk to maximum query throughput. Exadata allows flash directives at the table or partition level easily implementing ILM Note that the limiting factor to query throughput with compressed flash cache data is generally CPU. The high rate of IO generally moves the bottleneck out of the IO system.
46
Oracle Exadata Momentum Rapid adoption in all geographies and industries
47
Oracle Exadata Database Machine: Changes the Game
Database intelligence and massively parallel scaling in the storage tier Using state of the art industry standard hardware Complete, Integrated Data Warehouse, OLTP and consolidation solution High availability Enterprise Security Advanced analytics Innovative new technologies: Hybrid Columnar Compression In-memory parallel execution FlashFire hardware + flash-optimized software
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.