DAT 320 SQL Server 2000: DTS and .NET 7/29/2019 6:40 PM DAT 320 SQL Server 2000: DTS and .NET Trey Johnson Practice Director, Business Intelligence Encore Development © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

The Development Environment Microsoft® SQL Server™ 2000 and Microsoft Windows® 2000 Visual Studio® .NET Enterprise Edition Visual C#™ .NET .NET SDK command-line tools Type Library Importer tools (Tlbimp.exe) Strong Name tool (Sn.exe) Global Assembly Cache tool (Gacutil.exe) Registry Assembly tool (Regasm.exe) GUIDGen (Guidgen.exe) © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

DTS Architecture Overview 7/29/2019 6:40 PM DTS Architecture Overview Copy Column Transform DataPumpTransform Connection (OLE-DB) Connection (OLE-DB) Transform Data Task CustomTask Execute SQL Task Send Mail Task CustomTask CustomTask DTS Package DTS Package Object Model (DTSPKG.DLL) DTS Wizard DTS Designer DTSRUN.EXE DTSRUNUI.EXE Your Application © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

SQL Server 2000 DTS Object Model 7/29/2019 6:40 PM SQL Server 2000 DTS Object Model Connections Connection2 ConnectionProperties OLEDBProperty GlobalVariables GlobalVariable2 Steps Step2 PrecedenceConstraints PrecedenceConstraint Tasks Task CustomTask CustomTaskUI PersistPropertyBag PropertyBag PackageLog SavePackageInfos SavePackageInfo Package2 © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Interop between .NET and DTS 7/29/2019 6:40 PM Interop between .NET and DTS DTS is implemented as a set of free threaded in-process COM servers DTSPKG.DLL exposes all package and custom task interfaces through a Type Library DTSPUMP.DLL (Custom Transforms) does not expose a Type Library Calling DTS from .NET (execution of packages) Requires the creation of a Runtime Callable Wrapper (RCW) that wraps the COM interfaces into a .NET assembly. Extending DTS using .NET (building Custom Tasks) Requires creation .NET assemblies that expose themselves via a COM callable wrapper (CCW) to interact between COM/DTS and .NET © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7/29/2019 6:40 PM COM Interop A Runtime Callable Wrapper (RCW) must be created for DTSPkg.dll The RCW allows a .NET component to work with a COM component Install RCW in the Global Assembly Cache (GAC) © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

COM Interop (2) Use the Strong Name tool to create the key file 7/29/2019 6:40 PM COM Interop (2) Use the Strong Name tool to create the key file Sn.exe -k DTS.key Create RCW of DTSPKG.DLL tlbimp.exe "C:\Program Files\Microsoft SQL Server\80\Tools\Binn\dtspkg.dll" /out:Microsoft.SQLServer.DTSPkg80.dll /keyfile:dts.key The /keyfile parameter signs the RCW with a strong name because the RCW must be installed in the GAC © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7/29/2019 6:40 PM COM Interop (3) Use the Global Assembly Cache tool to install the RCW in the GAC gacutil.exe -i Microsoft.SQLServer.DTSPkg.dll Now we can reference the RCW in a Visual Studio .NET project © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Configuring the Development Environment © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS Custom Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

DTS Package Execution Threading Thread/Step Scheduler 7/29/2019 6:40 PM DTS Package Execution Threading DTS is Free Threaded Can cause problems with non Free Threaded providers and COM Objects (e.g. Visual Basic Custom Tasks) Tasks are marshaled onto worker thread Thread/Step Scheduler Constantly runs, if a step can be ran and there is a free thread (ConcurrentSteps < MaxConcurrentSteps) it will be executed The scheduler can be “fooled” & “bypassed”, which can used to implement “looping of steps” in a package © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Executing Packages DTS Object Model exposes methods for: 7/29/2019 6:40 PM Executing Packages DTS Object Model exposes methods for: Loading a Package LoadFromSQLServer, LoadFromStorageFile Executing a Package Execute() Cleaning Up a Package Execution Uninitialize() © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Executing Packages – Simple Approach 7/29/2019 6:40 PM Executing Packages – Simple Approach Load package, Execute, Cleanup [MTAThread] public void Main() { try { Package2Class package = new Package2Class(); object pVarPersistStgOfHost = null; package.LoadFromSQLServer("(local)", null, null, DTSSQLServerStorageFlags.DTSSQLStgFlag_UseTrustedConnection, null, null, null, “MyPackageName", ref pVarPersistStgOfHost); package.Execute(); package.UnInitialize(); package = null; } catch(COMException e) { // handle COM exceptions first } catch(System.Exception e) { // handle all other exceptions last } © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Running A Package 7/29/2019 6:40 PM © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

More Sophisticated… Checking Step Results 7/29/2019 6:40 PM More Sophisticated… Checking Step Results Why? The Process After calling Execute(), iterate over the step results Determine status and if failed call GetExecutionErrorInfo() Method Decide what you want to do (i.e. Log to file, Log to Console, etc…) Interact with package state until Unitialize() is invoked © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Checking Step Results… The Code 7/29/2019 6:40 PM Checking Step Results… The Code The Code – Not tough at all // Look for steps that completed and failed. foreach (Step2 step in package.Steps) { string stepName = step.Name; if (step.ExecutionStatus == DTSStepExecStatus.DTSStepExecStat_Completed) { if (step.ExecutionResult == DTSStepExecResult.DTSStepExecResult_Success) Console.WriteLine("Step {0} succeeded", stepName); } else if (step.ExecutionResult == DTSStepExecResult.DTSStepExecResult_Failure) step.GetExecutionErrorInfo( out pErrorCode, out pbstrSource, out pbstrDescription, out pbstrHelpFile, out pHelpContext, out pbstrIDofInterfaceWithError); Console.WriteLine("Step {0} failed", stepName); Console.WriteLine("error : {0}\nsource: {1}\ndescr.: {2}", pErrorCode, pbstrSource, pbstrDescription); } // completed } // foreach © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Checking Step Results 7/29/2019 6:40 PM © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS Custom Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Execute Package using Events 7/29/2019 6:40 PM Execute Package using Events Why? The Process Delegates are commonly used as event handlers When handling DTS Package Events do not use delegates Use the UCOMIConnectionPt and UCOMIConnectionPtContainer interfaces and create your own event handler class © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Execute Package using Events 7/29/2019 6:40 PM Execute Package using Events class ExecPkgWithEvents { static void Main(string[] args) ExecPkgWithEvents app = new ExecPkgWithEvents(); app.Run(); } public Package2Class package; public void Run() package = new Package2Class(); UCOMIConnectionPointContainer CnnctPtCont = (UCOMIConnectionPointContainer) package; UCOMIConnectionPoint CnnctPt; PackageEventsSink PES = new PackageEventsSink(); // UUID of PackageEvents Interface Guid guid = new Guid("10020605-EB1C-11CF-AE6E-00AA004A34D5"); CnnctPtCont.FindConnectionPoint(ref guid, out CnnctPt); int iCookie; CnnctPt.Advise(PES, out iCookie); object pVarPersistStgOfHost = null; package.LoadFromStorageFile(…); package.Execute(); package.UnInitialize(); } // class ExecPkgWithEvents © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Execute Package using Events 7/29/2019 6:40 PM Execute Package using Events class PackageEventsSink : DTS.PackageEvents { public void OnQueryCancel(string EventSource, ref bool pbCancel) Console.WriteLine("OnQueryCancel({0})", EventSource); pbCancel = false; } public void OnStart(string EventSource) Console.WriteLine("OnStart({0})", EventSource); public void OnProgress(string EventSource, string ProgressDescription, int PercentComplete, int ProgressCountLow, int ProgressCountHigh) Console.WriteLine("OnProgress({0})”, EventSource); public void OnError(string EventSource, int ErrorCode, string Source, string Description, string HelpFile, int HelpContext, string IDofInterfaceWithError, ref bool pbCancel) Console.WriteLine("OnError({0}”, EventSource); pbCancel = false; public void OnFinish(string EventSource) Console.WriteLine("OnFinish({0})", EventSource); } } // class PackageEventsSink : DTS.PackageEvents © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Executing Package with Events 7/29/2019 6:40 PM Executing Package with Events © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7/29/2019 6:40 PM Passing Parameters Pass/set dynamic context by setting and adding global variables Package.GlobalVariables Set value of existing global variable Add global variable (1) Add global variable (2) package.GlobalVariables.Item("GlobInt").Value = 2631; package.GlobalVariables.AddGlobalVariable("GlobStr", "Hello World"); GlobalVariable2 gv = pkg.GlobalVariables.New("GlobStr2"); gv.Value = "Hello World2"; package.GlobalVariables.Add(gv); © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS Custom Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Creating Packages Basically the reverse of saving a Package 7/29/2019 6:40 PM Creating Packages Basically the reverse of saving a Package Package creations steps: Create new Package object Set Package properties Add GlobalVariables Set values (and types) Add Connections Set ConnectionProperties (OleDBProperties) Add Steps Set Step properties Add PrecedenceConstraints Add Tasks Set Task properties Add CustomTask Set CustomTask properties (NOTE: Type specific!) Save and/or Execute Package © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Creating Packages Programmatically 7/29/2019 6:40 PM Creating Packages Programmatically © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS Custom Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7/29/2019 6:40 PM Custom DTS Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (2) CustomTask interface CustomTaskUI 7/29/2019 6:40 PM Custom DTS Tasks (2) CustomTask interface All custom tasks must implement this interface Provides a default properties sheet for the custom task CustomTaskUI Create a custom interface for your task © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (3) CustomTask interface contains the following: 7/29/2019 6:40 PM Custom DTS Tasks (3) CustomTask interface contains the following: Name and Description properties Properties collection Execute method All custom tasks must implement the items in the three previous bullets, else compilation will fail © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (4) CustomTaskUI methods Initialize New Edit Delete 7/29/2019 6:40 PM Custom DTS Tasks (4) CustomTaskUI methods Initialize New Edit Delete Help GetUIInfo CreateCustomToolTip (not supported in DTS Designer) You will likely not do anything with the last two methods and the Delete method. However, these need to be implemented or else a compilation error will result. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (5) Drag the task onto Designer surface 7/29/2019 6:40 PM Custom DTS Tasks (5) Drag the task onto Designer surface Initialize (your code should display UI) Dismiss UI for the first time New Dismiss UI subsequent times Edit Remove task from DTS Designer surface Initialize (custom UI not shown), Delete Right-click task and select Help Initialize (custom UI not shown), Help © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (6) Implementing a custom task 7/29/2019 6:40 PM Custom DTS Tasks (6) Implementing a custom task Create a Visual C# .NET class library project Add a reference to the RCW Add the plumbing code GUID (GUIDGen.exe) and ProgID for the task Registration and deregistration methods Add the functionality of the task in the Execute method © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (7) Registration function Deregistration function 7/29/2019 6:40 PM Custom DTS Tasks (7) Registration function [System.Runtime.InteropServices.ComRegisterFunctionAttribute()] static void RegisterServer(Type t) { //code to register custom task } Deregistration function [System.Runtime.InteropServices.ComUnregisterFunctionAttribute()] static void UnregisterServer(Type t) //code to unregister custom task © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (8) The custom task should be installed in the GAC 7/29/2019 6:40 PM Custom DTS Tasks (8) The custom task should be installed in the GAC Create a strong name using Sn.exe Add the following to the AssemblyInfo.cs file [assembly:AssemblyKeyFile("<path to key file>") Build the custom task using Visual Studio .NET Register the custom task using the Registry Assembly tool Regasm.exe CustomTask.dll Install the custom task in the GAC Gacutil.exe /i CustomTask.dll Create a DTS Package in Enterprise Manager and place the custom task on the Designer surface © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Custom DTS Tasks (9) TIP: Use fixed GUID for Custom Task 7/29/2019 6:40 PM Custom DTS Tasks (9) TIP: Use fixed GUID for Custom Task [Guid("C5A81539-7D87-3A86-9F4C-FC61FAFEA373")] Not required but this eases the development and debugging of custom tasks, otherwise a new GUID will be assigned per compile © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

DTS Custom Task using C# 7/29/2019 6:40 PM DTS Custom Task using C# public class MyCustTask : CustomTask { private string _Name; private string _Description; public MyCustTask() { // TODO: Add constructor logic here } public void Execute(object pPackage, object pPackageEvents, object pPackageLog, ref DTSTaskExecResult pTaskResult) { // Implement Execute task logic here public string Description { get {return _Description;} set {_Description = value;} public string Name { get {return _Name;} set {_Name = value;} public Properties Properties { get {return null;} © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7/29/2019 6:40 PM .NET Custom Task © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Debugging Custom Tasks 7/29/2019 6:40 PM Debugging Custom Tasks Easy to use Visual Studio .NET to debug DTS applications and custom tasks Either start or attach to Enterprise Manager C:\WINNT\System32\Mmc.exe /s "C:\Program Files\Microsoft SQL Server\80\Tools\BINN\SQL Server Enterprise Manager.MSC" Open custom task source file and set breakpoints Run the package and the debugger will halt on the breakpoint set in your custom task © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Agenda The Dev Environment, Architecture and Object Model 7/29/2019 6:40 PM Agenda The Dev Environment, Architecture and Object Model DTS and COM Interop Executing DTS Packages Handling DTS Package Events Developing Custom DTS Tasks in .NET Debugging DTS Custom Tasks © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

References SQL Server 2000 Books Online, “DTS Programming” chapter 7/29/2019 6:40 PM References SQL Server 2000 Books Online, “DTS Programming” chapter Q326909, “HOW TO: Create a Data Transformation Services Custom Task by Using Visual C# .NET” Q319985, “HOW TO: Handle Data Transformation Services Package Events in Visual C# .NET” Portions of this presentation: Copyright (C) SQLDev.Net - Gert E.R. Drapers 1991-2004 © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Professional Association for SQL Server SQLServer BI Resource Site 7/29/2019 6:40 PM Professional Association for SQL Server www.sqlpass.org SQLServer BI Resource Site www.sqlserverbi.com SQL Server Developers Network www.sqldev.net (Excellent Site!) SQLDTS Site www.sqldts.com SQLServer Central www.sqlservercentral.com © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Speaker Information Trey Johnson Encore Development tjohnson@encoredev.com trey@sqlserverbi.com Encore Development www.encoredev.com Microsoft Gold Certified Partner for Business Intelligence Microsoft Gold Certified Partner for Commerce Solutions Microsoft Gold Certified Partner for Collaboration Solutions © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Please fill out a session evaluation on CommNet 7/29/2019 6:40 PM Please fill out a session evaluation on CommNet Q1: Overall satisfaction with the session Q2: Usefulness of the information Q3: Presenter’s knowledge of the subject Q4: Presenter’s presentation skills Q5: Effectiveness of the presentation © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

© 2004 Microsoft Corporation. All rights reserved. 7/29/2019 6:40 PM © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.