Tony Petrossian Microsoft Corporation Best Practices

Service Recap “Server” Management Connectivity and Tracing Using multiple databases Migrating data and schema in and out of SQL Azure

Relational database service Built on SQL Server technology foundation Highly scaled Highly secure Database “as a Service” – beyond hosting Customer Value Props Self-provisioning and capacity on demand Symmetry w/ on-premises database platform Automatic high-availability and fault-tolerance Automated DB maintenance (infrastructure) Simple, flexible pricing – “pay as you grow”

TDS Gateway Front-end Node Protocol Parser Gateway Logic TDS Session 1 Front-end Node Protocol Parser Gateway Logic Master Cluster Master Node Data Node Components Partition Manager Master Node Data Node Components Partition Manager Machine 5 SQL Instance SQL DB DB5 DB3 DB2 Scalability and Availability: Fabric, Failover, Replication, and Load balancing Machine 6 SQL Instance SQL DB Master1 Meta 1 DB2 DB3 Machine 4 SQL Instance SQL DB DB2 DB3 DB4 Machine 7 SQL Instance SQL DB DB4 DB5 DB1 DB7 Scalability and Availability: Fabric, Failover, Replication, and Load balancing 4466 Master1 Applications connect directly to a database “Initial Catalog = ” in connection string No support for context switching (no USE ) SQL Azure connection strings follow SQL syntax Except for an unusual username format Format of username for authentication: ADO.Net: Data Source=server.database.windows.net; User ODBC: Driver={SQL Server Native Client 10.0}; Server=server.database.windows.net;

Where should I create my server? Co-locate server with Windows Azure app to reduce latency When should I create a new server? Trade off between geo/admin/billing Managing Server Through the Portal Add/Drop server Establish admin credentials View usage reports Network access configuration Through the Master Database Fine-tune firewall settings through code User logins Usage and metrics reporting (billing) Create/Drop databases Admin roles permissions CREATE/DROP database CREATE/DROP/ALTER login GRANT/REVOKE rights Modifying server firewall settings Master DB HTTP TDS SQL Azure Server

Each server defines a set of firewall rules Determines access policy based on client IP By default, there is NO ACCESS to server Controlled using TSQL API against Master DB: sys.firewall_rules, sys.sp_set_firewall_rule, sys.sp_delete_firewall_rule Portal UX

sys.bandwidth_usage: usage in KB sys.database_usage: instance count by type per day

Most-applicable SQL Best Practices Connection Pooling Query Parameterization Batching Scaling with data and load Sharding Building copies Deploying and uploading data Bulk copy, BCP.EXE SSIS Microsoft Sync Framework Power Pack for SQL Azure

// When pooling, use connection and return immediately // Do not hold for a long time – pool ensures fast turnaround // on second use using (SqlConnection conn = new SqlConnection(…)) { conn.Open(); using (SqlCommand cmd = conn.CreateCommand()) { cmd.CommandText = …; … } using (SqlConnection conn = new SqlConnection(…)) { conn.Open(); … Increases efficiency by removing re-login

Connections can drop for variety of reasons Idleness for more than 30-minutes Transient (network) errors Intentional throttling First step: reconnect immediately Handles idleness- and transient-disconnects The SQL Azure Gateway handles connection retry for app Connections attempted for ~30s before failure What to do on connection failure? Wait (10 seconds), then retry Check the error messages and follow recommendation Server health can be checked via Portal TSQL APIs will come in later releases

while (true) { using (SqlConnection connection = new SqlConnection(connStr)) { try { connection.Open(); using (SqlCommand cmd = connection.CreateCommand()) { cmd.CommandText cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.Add(new BCPJobId)); cmd.ExecuteNonQuery(); } catch (SQLException Sqexc) { // deal with error } catch (Exception exc) { // deal with error } // more stuff // ….. } Keep it inside the main loop

Each session assigned a unique ‘sessionId’ Tracks session state and service errors Retrievable from CONTEXT_INFO() Save this with each new connection If you need support, support personnel will greatly appreciate that value

Guid? sessionId = null; using (SqlConnection conn = new SqlConnection(…)) { // Grab sessionId from new connection using (SqlCommand cmd = conn.CreateCommand()) { conn.Open(); cmd.CommandText = "SELECT CONVERT(NVARCHAR(36), CONTEXT_INFO())"; sessionId = new Guid(cmd.ExecuteScalar().ToString()); } // continue processing... }

Batching: push logic to Server Use stored procedures and batching Limit number of round trips to server Example: batch 10 statements vs. 10 round-trips Parameterized queries Parameterize queries (limits compiles) Declare all parameters, type and length Lack of parameter size leads to cache bloat

// Length inferred: leads to cache bloat  cmd.CommandText = "SELECT c1 FROM dbo.tbl WHERE c2 SqlDbType.NVarChar).Value = "1"; … SqlDbType.NVarChar).Value = "22"; // Length supplied: no cache bloat cmd.CommandText = "SELECT c1 FROM dbo.tbl WHERE c2 SqlDbType.NVarChar, 128).Value = "1"; nvarchar(1)) SELECT c1 FROM dbo.tbl WHERE c2 nvarchar(2)) SELECT c1 FROM dbo.tbl WHERE c2 nvarchar( 128 )) SELECT c1 FROM dbo.tbl WHERE c2

SQL Azure balances databases across machines Divide your data into smaller chunks Makes for better load-balancing Ensures we can place your data on the most-appropriate servers Using shorter transactions Ensures we can respond to issues faster Avoid impacting others sharing the same box Thus avoiding being throttled by SQL Azure

Provider of vehicle valuation data > 13 million visitors to their site per month 2.5 GB Database Size Data refreshed every week Replicate data across 6 databases for increased perf Increase/Decrease database count based on demand Kelly Blue Book

SQL Azure Cluster Server DBC1 DBC2 DBC3 DBC4 DBC5 DBC6

SQL Azure Castellan Castellan Venue DB Castellan Venue DB’s Venue 1 Partition(s) Venue 1 Partition(s) Castellan Venue DB Castellan Venue DB’s Venue 2 Partition(s) Castellan Venue DB Castellan Venue DB’s Venue N Partition(s) One application DB, many venue DB’s – each partitioned in to many parts (40+)... Azure Roles TicketDirect.* Dynamic Worker (tasks uploaded as blobs) Partitioner Worker Azure Storage Queues for communication between clients and roles Tables to record server & partition information Blobs to store web and worker role resources Client Applications Castellan.old (VB6) Castellan.Azure Box Office sales Ticket Printing System Administration Venue/Event Management Partitioning Castellan.Azure Box Office sales Ticket Printing System Administration Venue/Event Management Partitioning.Net Service Bus WCF On Premise SQL Server Castellan Venue Distributed Cache Worker MemCache

SQL Azure Cluster Node 55 DB3 Server DB1 DB2 DB3 DB4 DB5 DB6

Moving between SQL Server & SQL Azure Use SSMS script generation tool SQL Azure Migration Wizard Use Data-Tier Application VS2010, SQL Server 2008 R2, SQL Azure SU2 (April 2010) Build your own scripts Watch out for unsupported T-SQL Don’t depend on instance or server level objects

SQL Azure supports standard SQL data import and export patterns Use bulk loading patterns where possible BCP – console.EXE bulk load/export tool SSIS – SQL integration server Bulk APIs in ODBC and ADO.Net SQL Azure supports data synchronization With on-premises DBs and client stores Always good advice: Break batches up into smaller, consumable chunks Add retry and tracing logic to ensure robust resume in face of failures

// Bulk data import using (SqlBulkCopy bulk = new SqlBulkCopy(new SqlConnection(conn)) { DestinationTableName = "dbo.data", BatchSize = 2000, // Transaction size (length) BulkCopyTimeout = 10000, // Transaction timeout NotifyAfter = 1000, // Progress callback }) { bulk.SqlRowsCopied += new SqlRowsCopiedEventHandler( myProgressCallback); bulk.WriteToServer(sourceDataReader); }

// BCP example SET SRV=somesrv.database.windows.net SET SET PW=something SET S_DR=C:\flats SET DB=TPCH bcp %DB%.dbo.supplier in %S_DR%\supplier.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.nation in %S_DR%\nation.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.region in %S_DR%\region.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.customer in %S_DR%\customer.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.part in %S_DR%\part.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|“ bcp %DB%.dbo.supplier out %S_DR%\supplier.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.nation out %S_DR%\nation.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.region out %S_DR%\region.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.customer out %S_DR%\customer.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|" bcp %DB%.dbo.part out %S_DR%\part.tbl -c -U %LOGIN% -P %PW% -S %SRV% -t "|"

Many SQL Server patterns apply to SQL Azure Use SQL best practices wherever possible Patterns discussed: Connectivity (to database, not server) Tracing and support Batching, Pooling and Parameterization Getting data in and out

Azure Prices and Packages: SQL Azure Information: MSDN Resources

For more Information please contact Tony Petrossian Principal Program Manager SQL Azure Development Microsoft Corporation