1. DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation

2. Technologies Covered Data Transformation Services Analysis Services (Decision Support Objects) Transact SQL ADO ActiveX Scripting

3. “Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

4. Package Design Practices Drive Packages through Metadata Dynamic Properties Task Access to Property of any object Sources values from many locales Global Variables Building blocks for communicating metadata Populated by the Dynamic Properties Task UDL Files Configurable File for storing Connection Information

5. Package Design Practices Designing Modular Parent/Child Packages Execute Package Task Allows joining of Transaction Supports passing of Globals Execution through COM Automation Programmatic access to Package objects Eliminates Version Affinity

6. Parent Child Packages Child Package is Invoked by ActiveX Script Can also be done with Execute Package Task DTS Parent DTS Child – Error Handler Example

7. Demo Demo Parent/Child Package(s) And UDL Files Design Practices

8. “Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

9. Data Loading Practices Leverage Fast Load Interface(s) Bulk Insert (Not always possible) Data Pump (For More Complex Logic) Characteristics of this Approach Bulk Insert Task Execute SQL Task Rowset Parameter ADO Disconnected Recordsets DTS Multiphase Data Pump

10. Data Loading Practices Benefits to this Approach Cache Rowset on Client (fast lookups) Hold updates on Client Conditional Inserts/Updates without Data Driven Query

11. Demo Demo Conditional Inserts And Updates With The Multiphase Data Pump Design Loading

12. “Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

13. Advanced Practices Multi-Threaded Data Loading Requires Multiple Connections Data is Loaded in Parallel Avoid Contention by Loading into Separate Entities

14. Advanced Practices Task Looping Requires at least two tasks (Executive and Trigger) Facilitated by Manipulating Workflow.ExecutionStatus = DTSStepExecStat_Waiting Maintain State in Global Variables

15. Advanced Practices Communication through Global Variables Only Means for Communicating among Tasks Examples Pointer to Position within Global Array Status of a Stored Procedure Call Flag to Shutdown Parallel Load Processes

16. Advanced Practices – Benefits Higher Utilization during Data Load Compressed Data Loading Window Easy Orchestration within DTS

17. Multi-Threaded Loading Task Marshall Thread 1 Loading Task Initialization Task Thread n Loading Task Thread 1 Restart Task Thread n Restart Task

18. Demo Demo Multi-Threaded Data File Loading Advanced Practices

19. “Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

20. Auditing And Error Handling Logging Options Logging to MSDB Logging to Text File Logging to Event Log Enabled through Package Properties Capture Step/Package Errors Enabled through Data Pump Properties Capture Erroneous Data

21. Auditing And Error Handling DTSPackageLog Object WriteStringToLog() WriteTaskRecord() DTS Lineage Global Variables DTSLineage_Full DTSLineage_Short DTSErrorRecords Object Add() Clear()

22. Auditing And Error Handling Monitoring Execution – Simple as Writing SQL select package.name, package.description, step.stepname, step.errorcode, step.errordescription from msdb..sysdtspackagelog package, msdb..sysdtssteplog step where package.lineagefull = step.lineagefull and step.errorcode <> 0

23. Demo Demo Custom Logging Using Intrinsic DTS Options Error Handling

24. “Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

25. Analysis Services And DTS Analysis Services Processing Task Supports Processing of all OLAP Objects Customizable Processing through TreeKey and other properties Prominent Properties TreeKey ServerName\DBName\CubeFolder\CubeName ItemType 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension ProcessingOption 0=Full Process, 1=Refresh Data and 2=Incremental Update

26. Analysis Services And DTS Decision Support Objects Provide access to the AS Management Interfaces Automated via AxScript Supports Partition Management Creating Cloning Dropping Merging of partitions Should be an OLAP Administrator to Use

27. DSO Hierarchy

28. Analysis Services And DTS Benefits Opportunity to Dynamically Manage Partitions and other Objects Support for Dynamic Processing Inclusion in the BI Solution Load

29. Demo Demo Dynamic Partition Creation And Processing Analysis Services

30. Community Resources DTS Whitepaper – Search on MSDN for “Using DTS for Business Intelligence” Microsoft Site www.microsoft.com/sql SQLDTS Site www.sqldts.com SQL Server Developers Network www.sqldev.net SQLServer Central www.sqlservercentral.com

31. Ask The Experts Get Your Questions Answered I’ll be in the Experts booth right after this session

32. euang@microsoft.com Questions? Questions?

33. Community Resources http://www.microsoft.com/communities/default.mspx Most Valuable Professional (MVP) http://www.mvp.support.microsoft.com/ Newsgroups Converse online with Microsoft Newsgroups, including Worldwide http://www.microsoft.com/communities/newsgroups/default.mspx User Groups Meet and learn with your peers http://www.microsoft.com/communities/usergroups/default.mspx

34. evaluations evaluations

35. © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.