2. How to Build Scalable & Secure Database Applications Noel Jerke & Erin Welker Scalability Experts

3. Part 1 – Agenda Scaling SQL Server Application Databases Scaling SQL Server Application Databases  End-To-End Scalability  Application Data Management .Net Tips and Tricks

4. End-To-End Scalability Business Requirements

5. Scalability starts with the analysis of the business requirements Scalability starts with the analysis of the business requirements  Fast hardware, software, and tuning are only a small part of the scalability equation  The base of the pyramid is design, which has the greatest influence on scalability  Technical design is based on meeting business requirements

6. A critical juncture in developing scalable systems is aligning business requirements and the architectural direction A critical juncture in developing scalable systems is aligning business requirements and the architectural direction  Budgets  Critical nature of the system to the core business  Functional requirements  Time frame  Risks  Acceptable  Unacceptable

7. Other Issues to Consider Complexity Complexity  Is the application well understood?  Can it be reasonably controlled to scale properly? Manageability Manageability  Is the application manageable?  Do you have the right resources that are capable of scaling the system?

8. Application Data Management Key techniques to reduce the impact of the application on the database Key techniques to reduce the impact of the application on the database  Data caching  Pre-processing data  Load-balanced asynchronous processing  Mirroring data

9. Data Caching Commonly utilized data Commonly utilized data Data that is infrequently updated Data that is infrequently updated Reasonable in size to cache in the middle or client tier Reasonable in size to cache in the middle or client tier

10. Data Application Server Cache

11. Pre-processing Data Real-Accurate versus Real-Time Real-Accurate versus Real-Time Focus on having data calculations done ahead of time to meet accuracy requirements Focus on having data calculations done ahead of time to meet accuracy requirements Reduce the real time calculation requirements to a sub-set of data that has not been pre-processed Reduce the real time calculation requirements to a sub-set of data that has not been pre-processed

12. Data Application Server Cache - Processing Engine

13. Load Balanced Asynchronous Processing When real time analysis of large volumes of data is required, move the calculations into a middle-tier When real time analysis of large volumes of data is required, move the calculations into a middle-tier Allow several servers to run the middle tier objects and federate the data to be processed Allow several servers to run the middle tier objects and federate the data to be processed

14. Data Application Server Cache - Processing Engine - Calculations - Processing Engine - Calculations

15. Mirrored Data Use techniques for mirroring data between two SQL Server to separate analysis transactions from OLTP transactions Use techniques for mirroring data between two SQL Server to separate analysis transactions from OLTP transactions Techniques can include using replication and double commit of transactions Techniques can include using replication and double commit of transactions

16. Application Server Cache - Processing Engine - Calculation - Processing Engine - Calculations Reporting Data Core Data

17. .Net Tips and Tricks Server Controls Server Controls  Server controls add overhead to the server  Only use when needed  Consider using alternative direct coding and not using a control – this may take more programming time but might reduce overhead significantly

18. .Net Tips and Tricks Cache Data and Pages Cache Data and Pages  Plan caching into your application architectures  Caching will significantly improve performance Minimize Number of Assemblies Minimize Number of Assemblies  Assembly loading can be costly  Try to place logic of small assemblies in another assembly

19. .Net Tips and Tricks Use Stored Procedures Use Stored Procedures  Preferred data access method  Compiled in the database  Optimize for ADO.net Use SQL Data Reader for Reads Use SQL Data Reader for Reads  Always use for forward read-only data access  Note the connection remains open for the data reader, close as soon as possible  Use type accessors (GetInt32, GetString, etc.) to reduce type conversion

20. .Net Tips and Tricks When to Utilize DataSets When to Utilize DataSets  Powerful ability to relationally work with data outside of the database  Can add significant overhead to the server  Beware of synchronization issues with data updated on the server having been also changed in the database  Very useful for complex data manipulation that can be handled on the server  Useful for working with non-database relational and XML data

21. Part 2 - Agenda Indexing Strategies Indexing Strategies Minimizing Table Sizes Minimizing Table Sizes Efficient and Reusable Query Plans Efficient and Reusable Query Plans Effective Locking Effective Locking Appropriate Database Maintenance Appropriate Database Maintenance Minimal Use of Cursors Minimal Use of Cursors Knowledge of Available Tools Knowledge of Available Tools Securing SQL Server Application Data Securing SQL Server Application Data

22. Indexing Strategies – Heap Table

23. Indexing Strategies – Clustered Index

24. Indexing Strategies – Non-Clustered Index

25. Effective Table Indexing Create an effective clustered index, such as: Create an effective clustered index, such as:  Most frequently used search criteria  Primary Key  Frequently used range Keep index keys small Keep index keys small Only index selective columns Only index selective columns  Make sure the left-most column is selective Use Indexed Views, as appropriate Use Indexed Views, as appropriate Verify results and monitor over time Verify results and monitor over time

26. Minimizing Table Sizes Archive data, when able Archive data, when able Consider partitioned views and create partitions Consider partitioned views and create partitions Current DataHistory Indexed View with Check Constraint

27. Efficient and Reusable Query Plans Dynamic queries Dynamic queries  Auto-parameterization: SELECT * FROM Orders WHERE Order_ID = 1 sp_prepare/sp_execute sp_prepare/sp_execute sp_executesql sp_executesql Stored Procedures Stored Procedures

28. Effective Locking Keep transactions short Keep transactions short Apply proper indexes Apply proper indexes Retain SQL Server’s default behavior Retain SQL Server’s default behavior Monitor for locking issues: Monitor for locking issues:  sp_who / sp_lock  Alerts  System Performance Monitor  SQL Profiler  SQLDIAG.exe

29. Appropriate Database Maintenance Mix Full, Differential and Transaction log backups effectively Mix Full, Differential and Transaction log backups effectively Only defragment fragmented indexes Only defragment fragmented indexes Use DBCC INDEXDEFRAG if batch maintenance window is an issue Use DBCC INDEXDEFRAG if batch maintenance window is an issue  Starter script in SQL Server Books Online Consider manual Shrink DB and Update Statistics Consider manual Shrink DB and Update Statistics

30. Minimal Use of Cursors Use set-based instead of row-based operations Use set-based instead of row-based operations Row-based can be unknowingly implemented by: Row-based can be unknowingly implemented by:  Cursors  DTS Lookup  Functions to perform lookups

31. Knowledge of Available Tools SQL Profiler!! SQL Profiler!!  Many events and data columns  Can be saved to SQL table for query analysis System Performance Monitor System Performance Monitor SQL Query Analyzer SQL Query Analyzer  Graphical or textual showplan  SET STATISTICS IO ON  SET STATISTICS TIME ON

32. Securing SQL Server Application Data Ensure up-to-date service packs and security patches are installed Ensure up-to-date service packs and security patches are installed Implement a layered security approach Implement a layered security approach Use Windows Authentication Use Windows Authentication  If this cannot be used, be very careful about how password information is stored Use Application or User roles Use Application or User roles

33. Securing SQL Server Application Data Implement security at the object level Implement security at the object level Use views and stored procedures to provide a layer of abstraction from tables Use views and stored procedures to provide a layer of abstraction from tables Encrypt sensitive data such as credit card numbers Encrypt sensitive data such as credit card numbers Beware of opportunities for SQL Injection Beware of opportunities for SQL Injection

34. Call To Action Sign up to meet with the Scalability Experts/Microsoft Technical team. For more information, please email SCDLITE@microsoft.com You can download all presentations at www.microsoft.com/usa/southcentral/

35. Scalability Experts (972) 478-4333 www. scalabilityexperts.com Michael Handshy (sales) mhandshy@scalabilityexperts.com (972) 478-4333 Office (214) 629-6729 Mobile Noel Jerke njerke@scalabilityexperts.com (972) 745-3735 Office(214) 215-0787 Mobile Erin Welker ewelker@scalabilityexperts.com (972) 478-4333 Office(817) 239-4425 Mobile Contact Information

36. Questions ?

37. SQL Server Summit Brought To You By:

38. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.