Extreme Scaling with SQL Azure SQL Bits 7, York, October 2010 Martin Schmidt – Miracle A/S Denmark

Subtitle: The Safetrack story

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Speaker info O Name: Martin Schmidt O O Blog: O

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Safetrack facts O Danish Company – started 2005 O GPS tracking of trucks and excavators O Started out as an insurance service against theft O Today: A complete platform for fleet management O Tracks more than units O Website:

Unit Types

How does this Work?

How does this Work O All units are equipped with a GPS transmitter O Sends binary packages over GPRS O Built-in SIM card O Sends a package every 2 min when running O Sends a package every 15 min when parked O Data is sent using UDP Protocol

ME Length = 75 mm Width = 45 mm Height = 18 mm

Package data O Reporttime O Longitude O Latitude O Speed O KM Status O Inputs status information O Is the lift up / down O Is the engine running O Timecounter info on inputs O For how long has the engine been running O How long has the lift been in use

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

System Archietecture

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Database Design

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Issues ? O Database and application on same box O Everything runs on the same single server O Single Point of Failure O Hardware O Database O Services / Website O No HA solution O Limited scaling options (Buy bigger HW)

Issues ? O All elements are independend O Easy to split things O No ”secret” data O No personal or banking information O Not breaking the Danish data law

Issues ? O SQL Azure in the future? O Windows Azure in the future? O Why Not

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

SQL Azure O High Availibity = Free O 1 DB = 3 DB’s (automatic failover) O Size limitations O 1 – 50 GB O Safetrack database > 250 GB  O Get the database off the current box O Release more power to the Windows service

SQL Azure O Let Microsoft be your primary DBA! O SQL Azure = Black boks O Running your database with other unknown applications

Windows Azure O Moving the website to Windows Azure O Easy peasy O The Windows Service is not compatible with Azure at present O No support for UDP – TCP Only 

SQL Azure O Lets move the database to the Cloud O Latency from the local Windows service to SQL Azure is not a big issue. O Size is a issue, we will have to Shard the data across multiple databases O Sharding gives us scalability

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Sharding facts O Partition data across multiple databases O Each database holds a subset of the data O A query needs only to look in one database O Sharding is no silver bullet, you still need to think O Important to choose the correct sharding key O Try to avoid fan-out queries, if possible

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Solution walkthrough O Shard key candidates O UnitID O CustomerID O The winner is: O CustomerID O All reports on the web front end is based on customerID. This gives the customer the option to se the status of all his units.

Data Syncronisation O MS Data sync Framework O Requires SQL Agent to do scheduling O SQL Azure Data Sync (Dev Preview) O The winner so far O We will have a look in a minute O Pretty Slow  O SQL Server Integration Services O Still no scheduler O Windows Azure worker role O We need to test this

Database Design

DEMO O Enough Power Pointing

Agenda O Speaker Info O Safetrack facts & system info O Safetrack architecture O Database design O Issues with current setup O The Azure platform O Sharding O Solution Walkthrough O Moving Exsisting Data to SQL Azure

Loading data into Azure O Task: Load existing data into SQL Azure O Rows (ReportMessage) O 150 GB O Possible solutions O BCP Utility O SSIS

Loading data into Azure

O Slow, poor performance O Still working on improving performance O Tested throughput (Laptop) O 1000 Rows / sec

SQL Azure Ressources O SQL Azure webfrontend O O SQL Azure Labs O O SQL Azure team Blog O

More Ressources O PerformanceDUO O O Miracle A/S O O MiracleDB O O Safetrack O

Q & A O Questions?

Thank You

Coming up… P/X001 Understanding and Preventing SQL Injection Attacks Kevin Kline P/L001 SSIS Fieldnotes Darren Green P/L002 The (Geospatial) Shapes of Things to Come Simon Munro P/L005 End to End Master Data Management with SQL Server Master Data Services Jeremy Kashel P/T007 Understanding Microsoft Certification in SQL Server Chris Testa-O'Neill # SQLBITS