Presentation is loading. Please wait.

Presentation is loading. Please wait.

Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing.

Similar presentations


Presentation on theme: "Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing."— Presentation transcript:

1 Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing

2 Heterogeneous Connectivity & Workloads Data Integrity & Quality Compliance & Security Data Warehouse Scale Data Warehouse Management 2005 2008 Futures PB Warehouses >64 Core Processing Scale out through MPP Perf. Management Tools BI Resource Governance Improved Predictability Mixed workload support Continuous Loading Integrated DQ Services (Zoomix) Master Data Management (Stratature Integration) Rights Management 10s of TB Warehouses Parallel partitioning Data compression New Reference Architectures Policy Based Admin. DB Resource Governance High Perf. Connectors (Oracle, Teradata, SAP BW) Data Profiling Policy based auditing Multi TB Warehouses Enterprise scalability DW Reference Architectures Unified manageability Enterprise class ETL tool Data Cleansing (Fuzzy lookup/matching) Data Protection & Tracing

3 Microsoft Confidential Fast Track Overview The Bridge to Madison – Hub & Spoke Madison Overview MTP & TAP programs Quick Start Summary

4 Building a traditional DW Time consuming Expensive Performance varies Scalability issues Potential bottlenecks in standard DW architecture The DW appliance model Pre-built & tuned h/w + s/w Views entire stack holistically Known performance & scalability Encapsulates best practices Leverages Sequential I/O Lower TCO Faster deployment Better performance Minimised DBA time Benefits

5 5

6 6 Microsoft NDA-only Software: SQL Server 2008 Enterprise Windows Server 2008 Hardware: Tight specifications for servers, storage and networking ‘Per core’ building block Configuration guidelines: Physical table structures Indexes Compression SQL Server settings Windows Server settings Loading

7 2 Processor Configuration Server: HP ProLiant DL385 G5p with 2 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 4 – 8 TB 4 Processor Configuration Server: HP ProLiant DL 585 G5 with 4 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 4 – 16 TB 8 Processor Configuration Server: HP ProLiant DL 785 G5 with 8 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 16 – 32 TB Note - Compression assumes 2.5:1

8 2 Processor Configuration Server: Dell Power Edge 2950 MLK with 2 Quad-core Intel Xeon processors Storage server: EMC CX4-240 Scalability: 4 – 8 TB 4 Processor Configuration Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors Storage server: EMC CX4-240 Scalability: 12 – 24 TB Note - Compression assumes 2.5:1 - Fully loaded only adds drives to minimum HW required - Data space can be increased by using 450GB drives

9 9 Microsoft NDA-only

10 Fast Track offers appliance-like ease of deployment, scalability and performance for SMP Madison to offer massively parallel (MPP) scale and performance Madison hub-and-spoke architecture includes support for SMP spokes

11 Centralized EDW Model Is Challenged Too Expensive & Inflexible Complex ETL Overwhelmed By Mixed Workload Siloed Data Marts Outgrow their platforms No Centralized Management Madison Enables Best Of Both Worlds, Hub and Spoke Data Source DM1 DM2 DM3 DM4 Data Source Data Source Data Sourc e Data Sourc e Data Sourc e Enterprise Data Warehouse

12 12 Microsoft NDA-only Central EDW Hub Regional Reporting Departmental Reporting ETL Tools High Performance HQ Reporting

13 All systems connect via a dedicated high speed network Parallel database copy – speeds of up to 500 GB per min Simplify data mart ETL / ELT processes with publishing model Separate management and user workloads Integrate SMP SS08 and MPP systems Independently expand any system Add spokes without impacting other users Deploy development and test environments that leverage parallel connectivity Microsoft Confidential

14 INDUSTRY STANDARD NETWORKING INDUSTRY STANDARD SERVERS Reference Hardware Platforms Project Madison INDUSTRY STANDARD STORAGE

15

16 Database Servers Dual Infiniband Control Nodes Active / Passive Spare Database Server Dual Fiber Channel

17

18

19 Query No cheating (cache flushed) Inner joins Results 625K rows returned in 11 seconds from 1 trillion row table Final product will be even faster! Report Retailer: day-part analysis (Sales+Time+Date+Prod type)

20 Two Programs MTP – Madison Technology Preview 20 – 30 participants Duration of 4 to 6 weeks TAP – Beta production implementation 6 – 8 customers First iteration 9 to 12 weeks Requirements Focus on EDW and large data marts Migration projects, not green field Open to customers & prospects 30+ TB of data…at least 4 100+ TB

21 MTP High touch Support MS or partner will provide HW and will host the MTP Customer may have opportunity to engage with TAP MS will work with customer to define scope and success criteria MS will perform the bulk of MTP work (2 -3 resources) TAP Customer must procure the Madison reference architecture and conduct the TAP in their own data center Premier support will be provided Services will be provided (2 -3 resources) Training / mentoring will be provided MS will work with customer to define scope and success criteria

22 MTP Customer to provide data, queries, concurrency model, existing data model, etc. Customer to provide SME and DBA to answer questions of MTP team Customer to provide existing benchmarks Customer to define priorities for testing and areas of interest Customer to attend 2-3 day MTP interactive session and review TAP Customer to provide data, queries, concurrency model, existing data model, etc. Customer to provide SME, DBA and other resources to work with MS TAP team For onsite – customer to provide building access, internet access, etc Customer to provide existing benchmarks

23 Fast Track data warehouse reference architectures are available today Scale up today with SMP, scale out tomorrow with MPP Hub-and-Spoke will integrate SMP reference architectures with Madison Madison will scale SQL Server to 1PB Madison will set a new bar in appliance pricing and performance MTP and TAP for Madison are this year!

24 “Hubs” – 400TB Total 1 primary, 1 DR System Mirrored DR via ETL Mirror not shown “Spokes” – 350TB Total 3 DA v3 Appliances 1 SSE 2008 “SMP” Spoke “Hubs” – 400TB Total 1 primary, 1 DR System Mirrored DR via ETL Mirror not shown “Spokes” – 350TB Total 3 DA v3 Appliances 1 SSE 2008 “SMP” Spoke Dedicated IB Network −“Grid” enabled appliance network −SQL “Copy” between appliances −Up to 500GB/min transfer Teradata Migration −Direct migration of historical data −Automated Extract & Load Dedicated IB Network −“Grid” enabled appliance network −SQL “Copy” between appliances −Up to 500GB/min transfer Teradata Migration −Direct migration of historical data −Automated Extract & Load

25 © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.


Download ppt "Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing."

Similar presentations


Ads by Google