How to Build Scalable & Secure Database Applications Noel Jerke & Erin Welker Scalability Experts

Part 1 – Agenda Scaling SQL Server Application Databases Scaling SQL Server Application Databases  End-To-End Scalability  Application Data Management .Net Tips and Tricks

End-To-End Scalability Business Requirements

Scalability starts with the analysis of the business requirements Scalability starts with the analysis of the business requirements  Fast hardware, software, and tuning are only a small part of the scalability equation  The base of the pyramid is design, which has the greatest influence on scalability  Technical design is based on meeting business requirements

A critical juncture in developing scalable systems is aligning business requirements and the architectural direction A critical juncture in developing scalable systems is aligning business requirements and the architectural direction  Budgets  Critical nature of the system to the core business  Functional requirements  Time frame  Risks  Acceptable  Unacceptable

Other Issues to Consider Complexity Complexity  Is the application well understood?  Can it be reasonably controlled to scale properly? Manageability Manageability  Is the application manageable?  Do you have the right resources that are capable of scaling the system?

Application Data Management Key techniques to reduce the impact of the application on the database Key techniques to reduce the impact of the application on the database  Data caching  Pre-processing data  Load-balanced asynchronous processing  Mirroring data

Data Caching Commonly utilized data Commonly utilized data Data that is infrequently updated Data that is infrequently updated Reasonable in size to cache in the middle or client tier Reasonable in size to cache in the middle or client tier

Data Application Server Cache

Pre-processing Data Real-Accurate versus Real-Time Real-Accurate versus Real-Time Focus on having data calculations done ahead of time to meet accuracy requirements Focus on having data calculations done ahead of time to meet accuracy requirements Reduce the real time calculation requirements to a sub-set of data that has not been pre-processed Reduce the real time calculation requirements to a sub-set of data that has not been pre-processed

Data Application Server Cache - Processing Engine

Load Balanced Asynchronous Processing When real time analysis of large volumes of data is required, move the calculations into a middle-tier When real time analysis of large volumes of data is required, move the calculations into a middle-tier Allow several servers to run the middle tier objects and federate the data to be processed Allow several servers to run the middle tier objects and federate the data to be processed

Data Application Server Cache - Processing Engine - Calculations - Processing Engine - Calculations

Mirrored Data Use techniques for mirroring data between two SQL Server to separate analysis transactions from OLTP transactions Use techniques for mirroring data between two SQL Server to separate analysis transactions from OLTP transactions Techniques can include using replication and double commit of transactions Techniques can include using replication and double commit of transactions

Application Server Cache - Processing Engine - Calculation - Processing Engine - Calculations Reporting Data Core Data

.Net Tips and Tricks Server Controls Server Controls  Server controls add overhead to the server  Only use when needed  Consider using alternative direct coding and not using a control – this may take more programming time but might reduce overhead significantly

.Net Tips and Tricks Cache Data and Pages Cache Data and Pages  Plan caching into your application architectures  Caching will significantly improve performance Minimize Number of Assemblies Minimize Number of Assemblies  Assembly loading can be costly  Try to place logic of small assemblies in another assembly

.Net Tips and Tricks Use Stored Procedures Use Stored Procedures  Preferred data access method  Compiled in the database  Optimize for ADO.net Use SQL Data Reader for Reads Use SQL Data Reader for Reads  Always use for forward read-only data access  Note the connection remains open for the data reader, close as soon as possible  Use type accessors (GetInt32, GetString, etc.) to reduce type conversion

.Net Tips and Tricks When to Utilize DataSets When to Utilize DataSets  Powerful ability to relationally work with data outside of the database  Can add significant overhead to the server  Beware of synchronization issues with data updated on the server having been also changed in the database  Very useful for complex data manipulation that can be handled on the server  Useful for working with non-database relational and XML data

Part 2 - Agenda Indexing Strategies Indexing Strategies Minimizing Table Sizes Minimizing Table Sizes Efficient and Reusable Query Plans Efficient and Reusable Query Plans Effective Locking Effective Locking Appropriate Database Maintenance Appropriate Database Maintenance Minimal Use of Cursors Minimal Use of Cursors Knowledge of Available Tools Knowledge of Available Tools Securing SQL Server Application Data Securing SQL Server Application Data

Indexing Strategies – Heap Table

Indexing Strategies – Clustered Index

Indexing Strategies – Non-Clustered Index

Effective Table Indexing Create an effective clustered index, such as: Create an effective clustered index, such as:  Most frequently used search criteria  Primary Key  Frequently used range Keep index keys small Keep index keys small Only index selective columns Only index selective columns  Make sure the left-most column is selective Use Indexed Views, as appropriate Use Indexed Views, as appropriate Verify results and monitor over time Verify results and monitor over time

Minimizing Table Sizes Archive data, when able Archive data, when able Consider partitioned views and create partitions Consider partitioned views and create partitions Current DataHistory Indexed View with Check Constraint

Efficient and Reusable Query Plans Dynamic queries Dynamic queries  Auto-parameterization: SELECT * FROM Orders WHERE Order_ID = 1 sp_prepare/sp_execute sp_prepare/sp_execute sp_executesql sp_executesql Stored Procedures Stored Procedures

Effective Locking Keep transactions short Keep transactions short Apply proper indexes Apply proper indexes Retain SQL Server’s default behavior Retain SQL Server’s default behavior Monitor for locking issues: Monitor for locking issues:  sp_who / sp_lock  Alerts  System Performance Monitor  SQL Profiler  SQLDIAG.exe

Appropriate Database Maintenance Mix Full, Differential and Transaction log backups effectively Mix Full, Differential and Transaction log backups effectively Only defragment fragmented indexes Only defragment fragmented indexes Use DBCC INDEXDEFRAG if batch maintenance window is an issue Use DBCC INDEXDEFRAG if batch maintenance window is an issue  Starter script in SQL Server Books Online Consider manual Shrink DB and Update Statistics Consider manual Shrink DB and Update Statistics

Minimal Use of Cursors Use set-based instead of row-based operations Use set-based instead of row-based operations Row-based can be unknowingly implemented by: Row-based can be unknowingly implemented by:  Cursors  DTS Lookup  Functions to perform lookups

Knowledge of Available Tools SQL Profiler!! SQL Profiler!!  Many events and data columns  Can be saved to SQL table for query analysis System Performance Monitor System Performance Monitor SQL Query Analyzer SQL Query Analyzer  Graphical or textual showplan  SET STATISTICS IO ON  SET STATISTICS TIME ON

Securing SQL Server Application Data Ensure up-to-date service packs and security patches are installed Ensure up-to-date service packs and security patches are installed Implement a layered security approach Implement a layered security approach Use Windows Authentication Use Windows Authentication  If this cannot be used, be very careful about how password information is stored Use Application or User roles Use Application or User roles

Securing SQL Server Application Data Implement security at the object level Implement security at the object level Use views and stored procedures to provide a layer of abstraction from tables Use views and stored procedures to provide a layer of abstraction from tables Encrypt sensitive data such as credit card numbers Encrypt sensitive data such as credit card numbers Beware of opportunities for SQL Injection Beware of opportunities for SQL Injection

Call To Action Sign up to meet with the Scalability Experts/Microsoft Technical team. For more information, please You can download all presentations at

Scalability Experts (972) www. scalabilityexperts.com Michael Handshy (sales) (972) Office (214) Mobile Noel Jerke (972) Office(214) Mobile Erin Welker (972) Office(817) Mobile Contact Information

Questions ?

SQL Server Summit Brought To You By:

© 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.