Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing.

Slides:



Advertisements
Similar presentations
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
Advertisements


© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
SQL Server Fast Track & Project Madison – SQL MPP
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
1 Vladimir Knežević Microsoft Software d.o.o.. 80% Održavanje 80% Održavanje 20% New Cost Reduction Keep Business Up & Running End User Productivity End.
Microsoft Data Warehouse Vision Massive Scalability at Low Cost Improved Business Agility and Alignment Democratized Business Intelligence Hardware.
Performance and Scalability. Optimizing PerformanceScaling UpScaling Out.
Web RoleWorker Role At runtime each Role will execute on one or more instances A role instance is a set of code, configuration, and local data, deployed.
MIX 09 4/15/ :14 PM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
Co- location Mass Market Managed Hosting ISV Hosting.
Windows Server Scalability And Virtualized I/O Fabric For Blade Server
Microsoft Data Warehousing Vision SQL Server 2008 R2 DW enhancements High speed connectors Change Data Capture Star join enhancement features Partition.
Windows 7 Training Microsoft Confidential. Windows ® 7 Compatibility Version Checking.
Feature: OLE Notes Migration Utility
PlacePlace TypeType ServiceService Analysis Caching Integration Sync Search Relational BLOB Query BackupLoad Multi Dim In Memory File XML Reporting.
Session 1.
Built by Developers for Developers…. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names.
Training Workshop Windows Azure Platform. Presentation Outline (hidden slide): Technical Level: 200 Intended Audience: Developers Objectives (what do.
SQL Server Data Warehousing Overview
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
Connect with life Connect with life
SESSION CODE: BIE07-INT Eric Kraemer Senior Program Manager Microsoft Corporation.
Data Management Conference Introducing SQL Server 2008 R2 Mark Linton Director of WW Marketing SQL Server Business Group
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
Data Management Conference Data Warehousing John Plummer TSP Architect
Feature: Customer Combiner and Modifier © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are.
Ian Ellison-Taylor General Manager Microsoft Corporation PC27.
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or.

customer.
Solution to help customers and partners accelerate their data.
demo © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names.
demo Demo.
demo QueryForeign KeyInstance /sm:body()/x:Order/x:Delivery/y:TrackingId1Z
Windows Azure SQL Data Sync Name Title Microsoft Corporation.
projekt202 © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are.
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks.
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or.

Patrick Ortiz Global SQL Solution Architect Dell Inc. BIN209.
Microsoft Forefront and System Center Driving to Dynamic IT with System Center and Forefront Eric Berg Director of Product Management Server & Tools.
SQL Server 2008 R2 Report Builder 3.0 SQL Server 2008 Feature Pack Report Builder 2.0 SQL Server 2008 General Availability Authoring & Collaboration (Acquisition:
Data Platform and Analytics Foundational Training
PowerApps & Flow Licensing Overview for Partners
System Center Marketing
SAP on Azure Proactive proposal.
RD Role Service Scale Limitations
SQL 2016 new Hosting Offers Secure Database Hybrid HyperScale
Data Platform and Analytics Foundational Training
Installation and database instance essentials
SQL Server BI on Windows Azure Virtual Machines
SQL Server OLTP with Microsoft Azure Virtual Machines
Business Intelligence for Project Server/Online
SAP on Azure Proactive proposal.
Fast Track Data Warehouse for SQL SERVER 2012
Title of Presentation 12/2/2018 3:48 PM
12/4/ :40 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
LitwareHR v2: an S+S reference application
Enabling the hybrid cloud with remote access appliances
8/04/2019 9:13 PM © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
Andrew Fryer Microsoft UK
Виктор Хаджийски Катедра “Металургия на желязото и металолеене”
Feature: Multi-user Editing Allowed in RMA Entry
Шитманов Дархан Қаражанұлы Тарих пәнінің
Title of Presentation 5/24/2019 1:26 PM
Presentation transcript:

Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing

Heterogeneous Connectivity & Workloads Data Integrity & Quality Compliance & Security Data Warehouse Scale Data Warehouse Management Futures PB Warehouses >64 Core Processing Scale out through MPP Perf. Management Tools BI Resource Governance Improved Predictability Mixed workload support Continuous Loading Integrated DQ Services (Zoomix) Master Data Management (Stratature Integration) Rights Management 10s of TB Warehouses Parallel partitioning Data compression New Reference Architectures Policy Based Admin. DB Resource Governance High Perf. Connectors (Oracle, Teradata, SAP BW) Data Profiling Policy based auditing Multi TB Warehouses Enterprise scalability DW Reference Architectures Unified manageability Enterprise class ETL tool Data Cleansing (Fuzzy lookup/matching) Data Protection & Tracing

Microsoft Confidential Fast Track Overview The Bridge to Madison – Hub & Spoke Madison Overview MTP & TAP programs Quick Start Summary

Building a traditional DW Time consuming Expensive Performance varies Scalability issues Potential bottlenecks in standard DW architecture The DW appliance model Pre-built & tuned h/w + s/w Views entire stack holistically Known performance & scalability Encapsulates best practices Leverages Sequential I/O Lower TCO Faster deployment Better performance Minimised DBA time Benefits

5

6 Microsoft NDA-only Software: SQL Server 2008 Enterprise Windows Server 2008 Hardware: Tight specifications for servers, storage and networking ‘Per core’ building block Configuration guidelines: Physical table structures Indexes Compression SQL Server settings Windows Server settings Loading

2 Processor Configuration Server: HP ProLiant DL385 G5p with 2 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 4 – 8 TB 4 Processor Configuration Server: HP ProLiant DL 585 G5 with 4 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 4 – 16 TB 8 Processor Configuration Server: HP ProLiant DL 785 G5 with 8 Quad-core AMD Opteron processors Storage server: EMC or MSA Storage Scalability: 16 – 32 TB Note - Compression assumes 2.5:1

2 Processor Configuration Server: Dell Power Edge 2950 MLK with 2 Quad-core Intel Xeon processors Storage server: EMC CX4-240 Scalability: 4 – 8 TB 4 Processor Configuration Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors Storage server: EMC CX4-240 Scalability: 12 – 24 TB Note - Compression assumes 2.5:1 - Fully loaded only adds drives to minimum HW required - Data space can be increased by using 450GB drives

9 Microsoft NDA-only

Fast Track offers appliance-like ease of deployment, scalability and performance for SMP Madison to offer massively parallel (MPP) scale and performance Madison hub-and-spoke architecture includes support for SMP spokes

Centralized EDW Model Is Challenged Too Expensive & Inflexible Complex ETL Overwhelmed By Mixed Workload Siloed Data Marts Outgrow their platforms No Centralized Management Madison Enables Best Of Both Worlds, Hub and Spoke Data Source DM1 DM2 DM3 DM4 Data Source Data Source Data Sourc e Data Sourc e Data Sourc e Enterprise Data Warehouse

12 Microsoft NDA-only Central EDW Hub Regional Reporting Departmental Reporting ETL Tools High Performance HQ Reporting

All systems connect via a dedicated high speed network Parallel database copy – speeds of up to 500 GB per min Simplify data mart ETL / ELT processes with publishing model Separate management and user workloads Integrate SMP SS08 and MPP systems Independently expand any system Add spokes without impacting other users Deploy development and test environments that leverage parallel connectivity Microsoft Confidential

INDUSTRY STANDARD NETWORKING INDUSTRY STANDARD SERVERS Reference Hardware Platforms Project Madison INDUSTRY STANDARD STORAGE

Database Servers Dual Infiniband Control Nodes Active / Passive Spare Database Server Dual Fiber Channel

Query No cheating (cache flushed) Inner joins Results 625K rows returned in 11 seconds from 1 trillion row table Final product will be even faster! Report Retailer: day-part analysis (Sales+Time+Date+Prod type)

Two Programs MTP – Madison Technology Preview 20 – 30 participants Duration of 4 to 6 weeks TAP – Beta production implementation 6 – 8 customers First iteration 9 to 12 weeks Requirements Focus on EDW and large data marts Migration projects, not green field Open to customers & prospects 30+ TB of data…at least TB

MTP High touch Support MS or partner will provide HW and will host the MTP Customer may have opportunity to engage with TAP MS will work with customer to define scope and success criteria MS will perform the bulk of MTP work (2 -3 resources) TAP Customer must procure the Madison reference architecture and conduct the TAP in their own data center Premier support will be provided Services will be provided (2 -3 resources) Training / mentoring will be provided MS will work with customer to define scope and success criteria

MTP Customer to provide data, queries, concurrency model, existing data model, etc. Customer to provide SME and DBA to answer questions of MTP team Customer to provide existing benchmarks Customer to define priorities for testing and areas of interest Customer to attend 2-3 day MTP interactive session and review TAP Customer to provide data, queries, concurrency model, existing data model, etc. Customer to provide SME, DBA and other resources to work with MS TAP team For onsite – customer to provide building access, internet access, etc Customer to provide existing benchmarks

Fast Track data warehouse reference architectures are available today Scale up today with SMP, scale out tomorrow with MPP Hub-and-Spoke will integrate SMP reference architectures with Madison Madison will scale SQL Server to 1PB Madison will set a new bar in appliance pricing and performance MTP and TAP for Madison are this year!

“Hubs” – 400TB Total 1 primary, 1 DR System Mirrored DR via ETL Mirror not shown “Spokes” – 350TB Total 3 DA v3 Appliances 1 SSE 2008 “SMP” Spoke “Hubs” – 400TB Total 1 primary, 1 DR System Mirrored DR via ETL Mirror not shown “Spokes” – 350TB Total 3 DA v3 Appliances 1 SSE 2008 “SMP” Spoke Dedicated IB Network −“Grid” enabled appliance network −SQL “Copy” between appliances −Up to 500GB/min transfer Teradata Migration −Direct migration of historical data −Automated Extract & Load Dedicated IB Network −“Grid” enabled appliance network −SQL “Copy” between appliances −Up to 500GB/min transfer Teradata Migration −Direct migration of historical data −Automated Extract & Load

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.