Nicholas Dritsas

Represents the customer-facing resources from the Server Product Groups. Azure CAT is comprised of product and solution experts that regularly engage in the largest, most complex, and most unique customer deployments worldwide.

Objectives −Learn about the largest SQL Server and Azure SQL DB projects −See the techniques they use to scale −Fit for purpose −We divide the projects into on-premise and cloud and present a brief architecture and numbers. Takeaways −SQL Server and Azure SQL DB will scale to handle very large projects −Both are enterprise ready −Very flexible architecture choices −DBAs more critical than ever

CategoryMetric Largest single database350 TB Largest table1.5 trillion rows Biggest total data 1 application88 PB Highest database transactions per second 1 server (from Perfmon) 130,000 Fastest I/O subsystem in production (SQLIO 64k buffer serial read test) 18 GB/sec Fastest “real time” cubeMillisecond latency data load for 1TB30 minutes* Largest MOLAP cube24 TB

CategoryMetric Largest sharded database20 TB Largest number of databases 1 app 14,000 Most concurrent users 1 app3 M Largest Deployment?*.cs = 14,832 Web.config=149 *.csproj=889 *.sln = 210

Compression Table Partitioning Partitioned Views Service Broker File Group and File management Caching SODA – Services Oriented Database Architecture DDR – Data Dependent Routing Scale up or Scale out

DATA WAREHOUSE / BI Building an 80TB data warehouse Very fast disk subsystem Scale out AS and RS

A large electronics company wanted to build a system to support 80TB of data for their reporting needs. The data is coming from sensors in their manufacturing line. They hold 36 months of data, one per filegroup, to manage compression and HA.

H/W Configurations Storage SAN VMAX SE * 3 ea Cache 128GB 8Gbps FC * 16 Port 300GB 15k * 88ea Raid-5 : U17TB for Data Area 300GB 15k * 6ea : Hot Spare 600GB 10k * 44ea Raid-5 : U17TB for BCV 600GB 10k * 4ea : Hot Spare Same Specs. For 3 VMAX SE EMC Replication Manager, TimeFinder/Clone, PowerPath HP DL980 (Production) MS SQL Server 2008 R2 Windows Server 2008 R2 2 CPU * 10 Core = 20 Core 512GB memory 8Gbps FC * 20port Production Server Standby Server PowerPath HP DL980 (Standby) MS SQL Server 2008 R2 Windows Server 2008 R2 2 CPU * 10 Core = 20 Core 512GB memory 8Gbps FC * 20port SAN Switch DS5300B * 2ea 8Gbps FC * 96port Total U17TB BCV U17TB BCV U17TB BCV

VMAX SE Building Block VMAX-SE GB 15K rpm Disks −8 Hot Spares −80 “Data disks” Raid (3RAID5) −20 RAID sets −Each “set” is a single 3+1 group 2,800 MB/sec target throughput

Room forecasting system Full suite of SQL products (SQL, AS, IS, RS) Scale out AS and RS If you need more capacity, just add another server Load Balanced Analysis Services reader machines 40 to 50 concurrent users per RS server Complex queries Large data sets returned to many clients

OLTP Scale out one database Scale up one database No downtime allowed – ever 16,000 + instances

The Commerce Transaction Platform supports Billing and subscriptions (eCommerce) for Microsoft products such as adCenter, Xbox Live, Zune, Windows Live Hotmail Plus, and Azure. The Commerce Transaction Platform supports payments using 13 payment methods spanning 42 currencies across 65 localized markets.13 payment methods42 currencies65 localized markets 5 DBAs

2 Datacenters 5 Webstore Clusters 15 Replicas (Financial Reporting; Fraud etc) 220 SQL 2012 servers (no VM) 736 databases (excluding Mirrors and Secondary's) 121 TB of datafiles (excluding Mirrors and Secondary's) 420 TB of storage – DAS & SAN (EMC/HDS) 12 TB monthly growth 82 DB Mirror Pairs – in DC most asynch, some with auto- failover 70 Log Shipping pairs – cross DC 400 Replication subscription streams, 6 distributors

Chose to scale up 17 TB OLTP database 10,000 concurrent user connections Replication for reporting database SAN hardware replication used for Disaster Recovery

Primary Data Centre (Active) Application Tower 1 Load Balancer OLTP DB Server SQL2008 How do we scale the solution ? “Performance Engineering” is at the heart of our methodology How do we scale the solution ? “Performance Engineering” is at the heart of our methodology Private WAN SQL Replication Reporting DB Server SQL2008 CPU Scale DB by adding CPU’s  MS support for large SMP hardware platforms is fundamental Segregation of OLTP and Reporting workloads  this allows specific tuning of the workloads Scale Application layer by adding additional servers into the VIP

World’s biggest publicly listed online gaming platform 15 million page views and up to 980,000 unique users a day Environment 5 DBA’s & 1 Database Architect 200+ SQL Server Instances 150+ TB of data, 4,000+ Databases 2+ PB storage 10+ TB RAM 450,000+ SQL Statements per second on a single server 500+ Billion database transactions per day No downtime allowed

Fujitsu RX-600 S5 64 Cores 1TB RAM Multiple 1 Gbit NIC interfaces Separate VLANs for Client Access, Cluster Intercom, Backup and Mirroring traffic External SAS Diskshelves for Data Files Attached with 4x 8GB Fibre Channel FusionIO PCI-E Solid State Devices for Transaction Log

Scale UP Table Partitioning NUMA Worker Threads Scale Out Table Partitioning Partitioned Views SODA Data Dependent Routing

Windows Server 2012 limit is 640 Cores New concept: Kernel Groups A bit like NUMA, but an extra layer in the hierarchy SQL Server 2012 the limit is 640 cores 4 TB RAM, both Windows and SQL Server

PDW Appliance Database Servers Control Nodes Active / Passive Landing Zone Backup Node Storage Nodes Spare Database Server Dual Fiber Channel Management Servers Compute RackControl Rack Compute Nodes Dual Infiniband

Separate your data by business function −Example: HR, Payroll, Accounting, etc Or by user function −Example: Login, chat, , pictures, etc Each function goes in a different database A common database for shared tables May replicate common tables to all databases (to keep all joins local)

36

Smart TVs requires frequent updates with new apps and software changes for better support and compatibility Due to the high success of the Smart TV line (20m TVs live and counting), Samsung needs a more scalable and elastic system. Achieved 10x performance levels compared to On Premises system, from 500 web requests / sec to 5,000. Main optimizations: Dropped using ibatis, a.net library to access SQL. Now, use stored procedures and ado.net Used IIS cache for the lookup data. We have a web role that refreshes the cache every 5 minutes Moved heavy sequential log inserts to table storage Moved from 20L to 10XL web roles, and we raised connection limit to 500 from 100 per web role. Having XL, we have better bandwidth. Switched from wcf to mvc. Host their SQL Azure DB in XL Resource Reservation to guarantee cores, IOPS and threads for it.

Web Role – Firmware download System Administrator Web Role – Firmware Upload System Website Devices (Smart TV x N) Device Validation Reporting Update Management Administration Blob Storage Smart TV firmware SQL Azure Device logs & update status Worker Role Task Automation – Firmware encryption and batch update ASP.NET HTML ASP.NET User ID/Password Single Tenant / No Federation Single Tenant The Solution/Architecture Windows Azure CDN

39

Large Australian based ISV developing Accounting software for small businesses. User count in the thousands, coming from desktop era

44 Workload Review: Cash Converters

45 Victoria (VIC) Azure Subscription Architecture: Cash Converters (Proposed) Store N On-premises Aggregations Monitoring Store N-5 QLD DB (HK) QLD.trafficmgr.cloudapp.net VIC DB (HK) Queensland (QLD) Store 1 Store 2 VIC.trafficmgr.cloudapp.net QLD DB (SGP) GeoDR Failover GeoDR VIC DB (SGP)

46 Current/Next Steps… Preparing application for multi-tenancy (in progress) Test the entire stack as a multi-tenanted application Tooling for moving store data from one DB to another Intent is to migrate single-tenanted store data to multi-tenanted DBs Multi-tenanted DBs to be initially put under RR and GeoDR’ed Gap here is the time to move a DB under RR - manual action during preview. This is also required after a failover to the original target (now source) and re-seeding to a new target

47

Read, Explore and Re-discover the Story Reflect and Share your experience Play and participate in the Story Buy and give the Story

Harry Potter Online Experience Why Cloud? Original Beta Launch July 2011 on-premises solution 4.2M Page views < 2 minutes 1M Registrations < 10 hours Could not scale easily Target User Base Unknown popularity, estimated ~20M users Since port to Azure - Open to all 1 billion page views in first 2 weeks ~15K Registrations/day >5M signups Peak ~84K concurrent users, now 25K Silent Launch: on April 14, 2012 and for 2 nd book in July, 2012 Pottermore = Very Happy CTO “Overwhelmed by support from Microsoft”

Web * (Web Role) App * (Web Role) Distributed Cache (Worker Role) Cache Builder (Worker Role) Users Federated by UserID Activities Comments Games (Wins) Others…. Shared Feeds Great Hall Common Room Others… Cache Builder OnStart: Content OnTime – Shared Feeds 30 to 120 seconds Stateless application servers serving to web site… Sender (Worker Role) Job Scheduler (Worker Role) ProxyRMF Database (Sharded SQL) (*) Web and App Server hosted together – different App Pools All User Data Profile Friends Inventory Games Others… Local Cache External System On-Premise Read/Write Write Only Read/Write Content Taxonomy Resources Session State (Cookie) Pottermore Architecture

52AZ-302-A| Largest Azure Projects