1. DAT 360: DTS in SQL Server 2000 Best Practices Euan Garden Group Manager, SQL Server Microsoft Corporation

2. Technologies Covered SQL Server 2000 DTS SQL Server 2000 DTS SQL Server 2000 SQL Server 2000 SQL Server 2000 Analysis Services SQL Server 2000 Analysis Services Transact SQL Transact SQL ActiveX Scripting ActiveX Scripting SQL Server Accelerator for Business Intelligence (SSABI) SQL Server Accelerator for Business Intelligence (SSABI)

3. “Best Practices” Agenda Package Design Package Design Data Loading Data Loading Analysis Services Management Analysis Services Management Auditing and Error Handling Auditing and Error Handling Putting it all together - SSABI Putting it all together - SSABI

4. Package Design Practices Drive Packages through Metadata – Dynamic Properties Task Access to Property of any object Access to Property of any object Can get property values from.INI, SQL, etc. Can get property values from.INI, SQL, etc. – Global Variables Building blocks for communicating metadata Building blocks for communicating metadata Populated by the Dynamic Properties Task Populated by the Dynamic Properties Task – UDL Files Configurable File for storing Connection Information Configurable File for storing Connection Information

5. Package Design Practices Designing Modular Parent/Child Packages – Execute Package Task Allows joining of Transaction Allows joining of Transaction Supports passing of Globals Supports passing of Globals – Execute through COM Automation LoadfromSQLServer, LoadfromRepository, LoadfromStorageFile methods LoadfromSQLServer, LoadfromRepository, LoadfromStorageFile methods Eliminates Version Affinity Eliminates Version Affinity Programmatic access to Package objects Programmatic access to Package objects

6. Parent/Child Packages Parent Package Fact ETL DimensionETL Global Variables, Return Code CubeProcessing

7. Parent/Child Package(s) and UDL Files demo demo

8. “Best Practices” Agenda Package Design Package Design Data Loading Data Loading Analysis Services Management Analysis Services Management Auditing and Error Handling Auditing and Error Handling Putting it all together - SSABI Putting it all together - SSABI

9. Data Loading Practices Leveraging High Perf. Interface(s) Leveraging High Perf. Interface(s) – Execute SQL Task Rowset Parameter – ADO Disconnected Recordsets – DTS Multiphase Data Pump Benefits Benefits – Cache Rowset on Client (fast lookups) – Hold updates on Client – Conditional Inserts/Updates w/out Data Driven Query

10. Conditional Inserts and Updates demo demo

11. Advanced Practices Approaches Approaches – Multi-Threaded Data Loading – Task Looping – Communication through Global Variables Benefits Benefits – Higher Utilization during Data Load – Compressed Data Loading Window – Easy Orchestration within DTS

12. Multi-Threaded Data File Load demo demo

13. “Best Practices” Agenda Package Design Package Design Data Loading Data Loading Analysis Services Management Analysis Services Management Auditing and Error Handling Auditing and Error Handling Putting it all together - SSABIPutting it all together - SSABI

14. Analysis Services and DTS Analysis Services Processing Task Analysis Services Processing Task – Supports Processing of all OLAP Objects – Customizable Processing through TreeKey and other properties Prominent Properties Prominent Properties – TreeKey ServerName\DBName\CubeFolder\CubeName ServerName\DBName\CubeFolder\CubeName – ItemType 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension – ProcessingOption 0=Full Process, 1=Refresh Data and 2=Incremental Update 0=Full Process, 1=Refresh Data and 2=Incremental Update

15. Analysis Services and DTS Decision Support Objects Decision Support Objects – Provide access to the AS Management Interfaces – Automated via AxScript – Supports Partition Management Creating Creating Cloning Cloning Dropping Dropping Merging of partitions Merging of partitions – Should be an OLAP Administrator to Use

16. DSO Object Hierarchy

17. Dynamic Partition Creation and Processing demo demo

18. Analysis Services and DTS Benefits – Opportunity to Dynamically Manage Partitions and other Objects – Support for Dynamic Processing – Inclusion in the BI Solution Load

19. “Best Practices” Agenda Package Design Package Design Data Loading Data Loading Analysis Services Management Analysis Services Management Auditing and Error Handling Auditing and Error Handling Putting it all together - SSABIPutting it all together - SSABI

20. Auditing and Error Handling Logging Options Logging Options – Logging to MSDB – Logging to Event Log – Logging to Text File Enabled through Properties Enabled through Properties – Granularity at Step and Package Level Enabled through Data Pump Properties Enabled through Data Pump Properties – Capture Erroneous Data

21. Auditing and Error Handling Scripting Options DTSPackageLog Object DTSPackageLog Object – WriteTaskRecord() – WriteStringToLog() Lineage Options DTS Lineage Global Variables DTS Lineage Global Variables – DTSLineage_Full – DTSLineage_Short

22. Custom Logging using Intrinsic DTS Options demo demo

23. Auditing and Error Handling Monitoring Execution – Simple as Writing SQL select package.name, package.description, step.stepname, step.errorcode, step.errordescription from msdb..sysdtspackagelog package, msdb..sysdtssteplog step where package.lineagefull = step.lineagefull and step.errorcode <> 0

24. “Best Practices” Agenda Package Design Package Design Data Loading Data Loading Analysis Services Management Analysis Services Management Auditing and Error Handling Auditing and Error Handling Putting it all together - SSABIPutting it all together - SSABI

25. ETLM Architecture Overview SourceSystems DTS ETL Subject Matter Db StagingDb KPI’sCubes *.txt DTS DTS

26. Loading Staging Tables Source Data Requirements Source Data Requirements – Cleansed, validated data – Separate data file for each table – Data files are | delimited Data Load Process Data Load Process – DTS Package calls the BULK INSERT T-SQL Statement – BULK INSERT parameters controlled by DTS Package global variables StagingDb *.txt 1|ABC Co.|110… DTS

27. Staging Data Options Staging Data Options – Inserts and Updates can be inferred or indicated – Slowly Changing Dimension support Data Load Process Data Load Process – DTS Package calls Stored Procedures – Stored Procedures provide error handling – Transform Data task uses Copy Column transformation Loading Dimension and Fact Tables StagingDb DTS

28. Processing Dimensions and Cubes Dimensions Dimensions – Incremental vs. Full Processing Cubes Cubes – Data Refresh – Incremental vs. Full Processing – Partitioning by Month DTS KPI’sCubes

29. SQL Server Accelerator for Business Intelligence demo demo

30. For More Information Encore Development Encore Development www.encoredev.com Professional Association for SQL Server Professional Association for SQL Serverwww.sqlpass.org Professional SQL Server 2000 DTS Professional SQL Server 2000 DTS wrox press, www.wrox.com DTS newsgroup DTS newsgroupmicrosoft.public.sqlserver.dts SQLServer Central SQLServer Centralwww.sqlservercentral.com SQLDTS Site SQLDTS Sitewww.sqldts.com SQL Server Developers Network SQL Server Developers Networkwww.sqldev.net

31. Questions?  euang@microsoft.com

32. Don’t forget to complete the on-line Session Feedback form on the Attendee Web site https://web.mseventseurope.com/teched/ https://web.mseventseurope.com/teched/

33. © 2002 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.