2. Darren Shaffer Microsoft MVP Handheld Logic WMB403

3. Agenda SQL Compact Edition Architecture SQL Compact Edition Sweet Spot Measuring & Understanding Performance Optimal Query & DML Performance High-Performance Development Optimal Data Synchronization

4. SQL Server CE Data Provider OLEDB CE VS 2005/2008 (C++) OLEDB Provider SQL Compact Edition Architecture CLR /.NET CF ADO.NET VB.NET & C#

5. SQL CE Query Processor A Heuristic, Optimizing QP in under 1MB! Heuristic Rewrites query into semantically equivalent form that leads to a better execution plan Based on syntax Adjacent inner joins are merged together so alternative join orders may be considered... FROM (Table_1 INNER JOIN Table_2 ON Table_1.Col = Table_2.Col) INNER JOIN Table_3 ON Table_1.Col = Table_3.Col...... FROM Table_1, Table_2, Table_3 WHERE Table_1.Col = Table_2.Col AND Table_1.Col = Table_3.Col... Cost-Based Optimization Determines Base table scan type (File scan / Index scan) What indexes, if any, are used Join order, join algorithm Sort / filter placement How it works Enumerates a selected subset of all possible execution plans Finds out the plan with lowest estimated cost Generates executable data structures that implement the plan

6. SQL CE Storage Engine Completely New in v3.1 4KB Page Size ACID Transaction Support Smart Device Awareness Row-Level Locking Automatic Reuse of Empty Pages Improved Tools for Database Health and Maintenance

7. Management Tools Communications and Messaging Device Update Agent Software Update Services Live Communications Server Exchange Server Internet Security and Acceleration Server Speech Server Image Update Location Services Multimedia MapPoint DirectX Windows Media Visual Studio 2005/2008 Development Tools MFC 8.0, ATL 8.0 Win32 NativeManaged Server Side LightweightRelational SQL Server 2005 Express Edition EDB Data Programming Model Device Building Tools Device Building Tools Hardware/Drivers Windows XP DDK Windows Embedded Studio Platform Builder OEM/IHV Supplied BSP (ARM, SH4, MIPS) OEM Hardware and Standard Drivers Standard PC Hardware and Drivers ASP.NET Mobile Controls ASP.NET.NET Compact Framework.NET Framework Microsoft Operations Manager Systems Management Server

8. SELECT IMEI, ProductCode, Quantity FROM (SELECT NULL AS IMEI, product AS ProductCode, (physicalqty - allocatedqty) AS Quantity FROM importstock WHERE (NOT mpstype IN(N'U', N'C', N'M', N'X', N'Y', N'P')) AND product IN (SELECT ProductCode FROM (SELECT importstock.product AS ProductCode FROM StockCountSchedule INNER JOIN StockCountProductCategories ON (StockCountSchedule.ID = StockCountProductCategories.ID) INNER JOIN importstock ON (StockCountProductCategories.Product_Type = importstock.product_type) WHERE (StockCountSchedule.IsRecount = 0) AND (StockCountSchedule.ID = 121231) UNION SELECT ProductCode FROM StockCountSchedule INNER JOIN CrossDevice_ProductsToRecount ON (StockCountSchedule.ID = CrossDevice_ProductsToRecount.StockCountID) WHERE (StockCountSchedule.IsRecount = 1) AND (StockCountSchedule.ID = 121231)) AS StockCountProducts) UNION SELECT IMEI.imei AS IMEI, NULL AS ProductCode, NULL AS Quantity FROM importstock INNER JOIN IMEI ON importstock.product = IMEI.product WHERE (mpstype IN(N'U', N'C', N'M', N'X', N'Y', N'P')) AND importstock.product IN (SELECT ProductCode FROM (SELECT StockCountSchedule.ID AS StockCountID, importstock. product AS ProductCode FROM StockCountSchedule INNER JOIN StockCountProductCategories ON (StockCountSchedule.ID = StockCountProductCategories.ID) INNER JOIN importstock ON (StockCountProductCategories.Product_Type = importstock.product_type) WHERE (StockCountSchedule.IsRecount = 0) UNION SELECT StockCountSchedule.ID AS StockCountID, ProductCode FROM StockCountSchedule INNER JOIN CrossDevice_ProductsToRecount ON (StockCountSchedule.ID = CrossDevice_ProductsToRecount.StockCountID) WHERE (StockCountSchedule.IsRecount = 1)) AS StockCountProducts)) AS StockCountItems Actual SQL CE Newsgroup Post:

9. SQL CE vs. SQL Express The real story… Footprint vs. capability Schema complexity Query complexity Off-line experience Data synchronization Security Ease of deployment Mindset

10. Measuring Performance Code instrumentation System.Diagnostics.StopWatch (new in NET CF 3.5) System.Environment.TickCount (.1 -.5 sec resolution) System.DateTime.Now (1 sec or worse resolution).NET CF profiling tools System Information on SOTI’s PocketController HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETCompactFramework\PerfM onitor\  set (DWORD) Counters = 1 Performance Counters http://blogs.msdn.com/davidklinems/archive/2005/12/09/502125.aspx Remote Performance Monitor (.NET CF 3.5 PowerToys) http://www.microsoft.com/downloads/details.aspx?FamilyID=c8174c14-a27d-4148-bf01- 86c2e0953eab&displaylang=en SQL Server Management Studio Display Query Execution Plan

11. Discard the first measurement Take sufficient number of samples Restart the application between tests Use realistic data, devices, and storage Measuring Performance Measurement tips

12. Measuring SQL CE Performance Darren Shaffer Microsoft MVP Handheld Logic

13. Match schema to SQL CE’s capabilities Base Table Cursors/TableDirect SqlCeResultSet versus SqlCeDataAdapter Leverage useful indexes Optimal Query Performance Big hitters

14. Minimize column count on tables Use variable length columns, narrow as possible Avoid Using: MaxFootprint ntext536M2n bytes nvarchar/nchar(big #) 40002n bytes varbinary/binary(big #) 80001n bytes Consider de-normalizing schema Keep JOINs to no more than 3 or 4 if possible Pre-calculate, pre-aggregate when possible Schema Tips

15. SELECT OrderID, SUM(UnitPrice * Quantity * (1.0 - Discount)) AS Total FROM OrderDetails GROUP BY OrderID SELECT OrderID, OrderTotal AS Total FROM Orders Pre-Computing Impact

16. Base Table Cursors/TableDirect Example: // create and execute SqlCeCommand SqlCeCommand cmd = new SqlCeCommand(“Authors",cnn); cmd.CommandType = CommandType.TableDirect; SqlCeDataReader dr = cmd.ExecuteReader(); // process results as usual while(dr.Read()) { MessageBox.Show("Name = " + dr["au_lname"]); } dr.Close(); dr.Dispose(); Bypasses the query processor Returns all columns in a row Fastest way to read from a table when you need all columns

17. Seek/SetRange Example: cmd.CommandType = CommandType.TableDirect; cmd.CommandText = "Orders"; cmd.IndexName = "idxDateTime"; object[] start = new object[1]; object[] end = new object[1]; start[0] = new SqlDateTime(2007, 1, 1); end[0] = new SqlDateTime(2008, 2, 3; cmd.SetRange(DbRangeOptions.Match, start, end); SqlCeDataReader dr = cmd.ExecuteReader(); dr.Seek(DbSeekOptions.FirstEqual, new SqlDateTime(2007,3,4)); while(dr.Read()) { // process results as usual } Bypasses the query processor Open a base table index Fastest way to select a range of values

18. Base Table Cursors/TableDirect

19. private SqlCeResultSet resultSet = null; private ResultSetView view1 = null; private void _bindData() { this.command.CommandText = “SELECT * FROM Orders”; ResultSetOptions options = ResultSetOptions.Scrollable | ResultSetOptions.Updatable; this.resultSet = this.command.ExecuteResultSet(options); this.view1 = this.resultSet.ResultSetView; int[] ordinals = new int[] { 1,3,5,8 }; this.view1.Ordinals = ordinals; this.dataGrid.DataSource = view1; } Query Performance Similar to SqlCeDataReader Excellent DML Performance Bi-Directional Scrolling and Update in Place SqlCeResultSet

20. SELECT * FROM OrdersSqlCeDataAdapter.Fill(DataSet) SELECT * FROM Orders cmd.ExecuteResultSet(ResultSetOptions) ResultSet Vs. DataAdapter 1,078 Orders

21. Scrollability Example: // Set the index range cmd.SetRange(DbRangeOptions.InclusiveStart, start, end); SqlCeDataReader dr = cmd.ExecuteReader(); // Seek to a value (customer name) dr.Seek(DbSeekOptions.FirstEqual,”Shaffer”); dr.Read(); // Process the row // Seek to another value (customer name) dr.Seek(DbSeekOptions.LastEqual,”Snerdley”); dr.Read(); // Process the row SqlCeDataReader is forward-only SqlCeDataAdapter SELECT does not leverage SetRange/Seek SqlCeResultSet is scrollable & fast. This is even faster:

22. Useful = Selective and Chosen by the Query Processor Selectivity is ratio of qualifying rows to total rows (low is good) Index on Orders.OrderID is selective Index on Orders.ShipVia is not selective Use sp_show_statistics_steps ‘table’, ‘index’ SQL CE uses only one index per table in an execution plan Indexes increase database size Avoid indexing small tables; table scan is more efficient Heavy DML, use fewer indexes Heavy querying, use more indexes Max of 249 indexes per table, 16 columns per index Leverage Useful Indexes Aren't they all useful?

23. SELECT OrderID, ProductID FROM OrderDetails WHERE OrderID = 10900 ADD INDEX on OrderID Impact of Indexes 2,820 OrderDetails records

24. Limit Requested Columns (70% better) SELECT * FROMSELECT ColumnName FROM Write SARGABLE Clauses (55% better) SELECT OrderID FROM Orders WHERE DATEPART(YEAR, OrderDate) = 1992 AND DATEPART(MONTH, OrderDate) = 4 SELECT Order ID FROM Orders WHERE OrderDate >= '04/01/1992' AND OrderDate < '05/01/1992‘ NON-SARGABLE Clauses: IS NULL, <>, !=, !>, !<, NOT, NOT EXISTS, NOT IN, NOT LIKE, LIKE %ABCD Query Performance Miscellaneous recommendations

25. JOINs versus SUBQUERIES (88% better) SELECT OrderID FROM Orders O WHERE EXISTS (SELECT OrderID FROM OrderDetails OD WHERE O.OrderID = OD.Order ID AND Discount >= 0.25) SELECT DISTINCT O.OrderID FROM Orders O INNER JOIN OrderDetails OD ON O.OrderID = OD.OrderID WHERE Discount >= 0.25 Query Performance Miscellaneous recommendations

26. Avoid Redundant DISTINCT(55% better) SELECT DISTINCT C.CustomerID, O.OrderID FROM Customers C INNER JOIN Orders O ON C.CustomerID = O.CustomerID SELECT C.CustomerID, O.OrderID FROM Customers C INNER JOIN Orders O ON C.CustomerID = O.CustomerID Use GetValues() versus GetXXX()(17% better) Index WHERE, ORDER BY, GROUP BY Columns(39% better) Query Performance Miscellaneous recommendations

27. Optimum Query Performance Darren Shaffer Microsoft MVP Handheld Logic

28. Pre-Load reference tables Parameterized DML Queries (specify precision) SqlCeResultSet update-in-place Consider removing/re-adding indexes Increase MaxBufferSize Use faster storage Optimal DML Performance Big hitters

29. Optimum DML Performance Darren Shaffer Microsoft MVP, Chief Software Architect Handheld Logic, LLC

30. Version of SQL Compact Edition Deployment Platform SQL CE Connection String Connection caching Encryption Options for deploying the “Starter Database” Database maintenance Recovery planning High-Performance Development Considerations

31. Put SQL in the UI layer Concatenate a bunch of strings to form SQL Use string. Replace for parameter values Use SqlCeDataAdapter (period) Forget to close and dispose of SqlCeDataReaders Forget to dispose of SqlCeCommands Forget to use SqlCeTransactions for DML Show users SqlCeExceptions One connection, > 1 Thread High-Performance Development What not to do…

32. High-Performance Development Presentation Logic Data Access DB Manager SQL Compact

33. High-Performance Development Darren Shaffer Microsoft MVP, Chief Software Architect Handheld Logic, LLC

34. Batch-Mode SDF File Exchange Remote data access Merge replication Custom web services Sync services for ADO.NET (Devices) Optimal Data Synchronization Options

35. The Secret to Data Sync Success Make a plan! Database TablePurposeTable Needed in Local Cache? All Columns Required? Primary Key Indexes Required? Filtering Possible? Lookup/ Reference only? Add/Change /Remove Endpoints Conflict Potential Business Logic? CreditCardTypesEnumerate valid credit card types Yes GUIDNo YesservernoneN/A NewSubscriptionsNew subscriptions are stored here Yes GUIDNoYesNodevice onlynoneYes - authorize credit card on delivery to server PostalCodesReverse Lookup City and State based on entering Zip Code YesNo (can omit Country) GUIDYes (PK is sufficient) NoYesservernoneN/A ProductsNewspapers and periodicals which can be subscribed to YesNo (can omit PublicationT ypeFk) GUIDNo YesservernoneN/A ProductsPromotionsJoin tableYes GUIDNo YesservernoneN/A PromotionsA PremiumPack offered for a specific time period Yes GUIDNo YesservernoneN/A PublicationTypesEnumerate valid publication types NoN/A SignaturesSubscriber's digital signature Yes GUIDNoYesNodevice onlynoneN/A UsersAuthenticate the mobile user YesNo (can omit Name) GUIDNoYes servernoneN/A VendorsEach user works for a Sales Vendor Yes GUIDNoYes servernoneN/A VendorsPromotionsJoin TableYes GUIDNoYes servernoneN/A

36. CRITERIA BEST CHOICE VIABLECHALLENGING Connectivity - Firewall Friendly WS/SSRDA/Merge Connectivity - WWAN/Dialup RDAWS/SS/Merge Connectivity - WLAN RDAWS/SS/Merge Conflict Resolution MergeSSWS/RDA Code to Implement RDAMerge/SSWS Setup/Deployment Effort RDAWSMerge On-Going Administration RDAWS/SSMerge Enterprise Management MergeWS/SSRDA Large Data Volumes Merge/RDAWS/SS Server DBMS Independence WS/SS Auto SQL CE DBMS Creation Merge/SS Ability to Secure Data Sync MergeRDA/WS/SS Overall Complexity RDA/SSWSMerge Choosing a Data Sync Strategy

37. V1 available now, new version coming Sweet-spot Atomic control over performance tuning Great way to create starter SDF files Concerns Sync Services for ADO.NET (Devices) Schedule some time to evaluate this!

38. Take only what you need Avoid implicit column type conversions Save a column, eliminate identity range pain Use uniqueidentifier PKs Set IsRowGuid to True Download-only vs. Bi-Directional articles Defrag indexes on the distributor DBCC INDEXDEFRAG (Driver, MSMerge_Contents, 1-4) Defrag all indexes on MSmerge_*, MSrepl_*, Mssnapshot_* Daily is not too often… Merge Replication Tuning Important considerations

39. Row-Level Tracking vs. Column-Level Tracking MaxBuffer Size on Subscriber Database Investigate lowering retention period from default of 14 days Set Merge Agent profile to match network Investigate mixing Merge replication with other techniques (RDA, Sync Services) Replication Monitor does not tell the whole story – instrument your subscriber code Merge Replication Tuning (cont'd)

40. Hardware matters Separate distributor from publisher Data and log files on separate, fast spindles Increase SQL Server minimum memory Tune IIS – the ISAPI DLL is the bottleneck More information: Merge Replication Tuning Hundreds of subscribers

41. www.microsoft.com/teched Sessions On-Demand & Community http://microsoft.com/technet Resources for IT Professionals http://microsoft.com/msdn Resources for Developers www.microsoft.com/learning Microsoft Certification & Training Resources Resources

42. Windows Mobile ® Resources TechNet TechCenter – System Center Mobile Device Manager 2008 http://technet.microsoft.com/scmdm http://technet.microsoft.com/scmdm TechNet TechCenter – Windows Mobile http://technet.microsoft.com/windowsmobile http://technet.microsoft.com/windowsmobile MSDN Center – Windows Mobile http://msdn.microsoft.com/windowsmobile http://msdn.microsoft.com/windowsmobile Webcasts and Podcasts for IT – Windows Mobile http://www.microsoft.com/events/series/msecmobility.aspx http://www.microsoft.com/events/series/msecmobility.aspx General Information – Windows Mobile http://www.windowsmobile.com http://www.windowsmobile.com General Information – System Center Mobile Device Manager 2008 http://www.windowsmobile.com/mobiledevicemanager http://www.windowsmobile.com/mobiledevicemanager Windows Marketplace Developer Portal http://developer.windowsmobile.com http://

43. Windows Mobile ® is giving away Blackjack IIs ! Stop by the Windows Mobile Technical Learning Center to learn how to enter

44. Complete an evaluation on CommNet and enter to win!

45. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.