1. Jemini Joseph

2. About me Working in Microsoft BI field since 2003. Mostly consulting in SSIS Worked as programmer in Visual Basic before moving to BI jeminijoseph@bi-datasolutions.com https://www.linkedin.com/in/jeminijoseph When I’m not working…

3. What’s it Read SSIS packages programmatically to report details using C# script Create new SSIS 2012/2014 package and deploy using new project deployment model Convert 2008 packages into 2012/2014 and deploy

4. Why Need to find the package that affects a table Need to convert 100’s of packages to 2012/2014 (end of server life) Need to create package for unknown format files

5. What do we need SQL Server 2008 installed including SDK SQL Server 2012/2014 installed including SDK SQL Server Data tools Knowledge of C# (or VB.net) scripting. Don’t have to be an expert

6. Controls in a package (Control flow) TaskHost (Executable) Containers Precedence Constraints

7. Controls in Data Flow Source Transformations Destination Paths

8. Deployment difference 2005/2008 used package deployment Configure each package 2012/2014 use project deployment model Can share configuration (Environment) Uses new SSISDB database and Integration Services Catalogs

9. Libraries References C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\Microsoft.CSharp.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.ConnectionInfo.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.DTSPipelineWrap.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SQLServer.DTSRuntimeWrap.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.ManagedDTS.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.IntegrationServices\12.0.0.0__89845dcd80 80cc91\Microsoft.SqlServer.Management.IntegrationServices.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.Sdk.Sfc\12.0.0.0__89845dcd8080cc91\Micr osoft.SqlServer.Management.Sdk.Sfc.dll C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Tasks\Microsoft.SqlServer.ScriptTask.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Smo\12.0.0.0__89845dcd8080cc91\Microsoft.SqlServer.S mo.dll C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Tasks\Microsoft.SqlServer.SQLTask.dll 100 = SQL 2008, 110=2012, 120 = 2014 GAC – 11= 2012 and 12=2014

10. Namespaces using System; using System.Data; using Microsoft.SqlServer.Dts.Runtime; using System.Windows.Forms; using System.Collections.Generic; using System.Text; using System.IO; using Microsoft.SqlServer.Server; using Microsoft.SqlServer.Dts.Pipeline.Wrapper; using Microsoft.SqlServer.Dts.Runtime; using Microsoft.SqlServer.Management.IntegrationServices; using Microsoft.SqlServer.Management.Smo; using System.Collections; using System.Data.SqlClient; using Microsoft.SqlServer.Management.Common;

11. Demo Read 2008 Package Create new 2012 package and project Copy 2008 to 2012 and deploy

12. The program flow create new Set the SSISDB and folder on 2014 box Create a 2014 project and package Create Parameters for the project Deploy project to SSISDB Create Environment (Get data from each package) Configure the project using new environment

13. Program Flow copy/read 2008 Copy connections to project connection managers Read through control flow If the control is TaskHost (Executable) copy it If the control is container Read through controls in container If the control is executable copy it Copy Precedence constraints to link controls If the control is DFT Copy controls Copy paths between objects Refresh metadata to get columns Copy the mapping for destination controls

14. Some code samples Opening and reading a package object obj ssisApp = new Microsoft.SqlServer.Dts.Runtime.Application(); Microsoft.SqlServer.Dts.Runtime.Package Srcpkg = ssisApp.LoadPackage(SrcPkgFile, null);; for (int i = 0; i < Srcpkg.Executables.Count; i++) { obj = SrcPackage.Executables[i]; //ExecuteSQL and Data Flow Task if (obj.ToString() == "Microsoft.SqlServer.Dts.Runtime.TaskHost") (Data flow task and execute SQL) if (obj.ToString() == "Microsoft.SqlServer.Dts.Runtime.Sequence")

15. Reading TaskHost TaskHost SourcetaskHost = (TaskHost)th; if (th.InnerObject is MainPipe) Data flow if (th.InnerObject.ToString() == "Microsoft.SqlServer.Dts.Tasks.ExecuteSQLTask.Execu teSQLTask") Execute SQL

16. Creating controls Executable ExecSQL ExecSQL = DstPackage.Executables.Add("STOCK:SQLTask"); Executable dataFlowTask = null; dataFlowTask = DstPackage.Executables.Add("STOCK:PipelineTask");

17. Creating controls in DFT MainPipe DstPipeline = dataFlowTask.InnerObject as MainPipe; IDTSComponentMetaData100 Targetcomponent = DstPipeline.ComponentMetaDataCollection.New(); Targetcomponent.ComponentClassID = "DTSAdapter.OleDbSource"; Targetcomponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(SrcCM); Targetcomponent.RuntimeConnectionCollection[0].ConnectionManagerID = SrcCM.ID; CManagedComponentWrapper srcDesignTime = Targetcomponent.Instantiate(); srcDesignTime.ProvideComponentProperties(); srcDesignTime.SetComponentProperty("AccessMode", 0); srcDesignTime.SetComponentProperty("OpenRowset", TableName); srcDesignTime.AcquireConnections(null); srcDesignTime.ReinitializeMetaData(); srcDesignTime.ReleaseConnections();

18. Some properties of OLE source and destination

19. What’s not done Doesn’t work for multiple inputs or outputs (Union all merge join, multicast) Reading packages from MSDB ([dbo].[sysssispackages]) Events

20. Some points Consider Sequence as a package The order of the object is unknown. So when you create path, need to find the control without any input (OLE Source) and start from there. The order of executables in package is unknown. Need to creating precedenceconstraints in right order when you copy existing package Need to add input columns to Derived column May need to fix the LineageID between Derived column and OLE Destination BIML can create new packages. Can it work with flat files or excel?

21. Some important classes IDTSComponentMetaData100 – Data flow controls IDTSComponentMetaDataCollection100 IDTSOutput100 – Output of a control (the pipe) IDTSOutputCollection100 IDTSOutputColumn100 IDTSOutputColumnCollection100 – Generally one, multicast will have multiple IDTSInput100 IDTSInputCollection100 IDTSInputColumn100 IDTSInputColumnCollection100 Generally one. Union all will have multiple IDTSCustomProperty100 – Custom property of a control like expression IDTSCustomPropertyCollection100 IDTSPath100 IDTSPathCollection100 – Collection of path. My example have two paths. PrecedenceConstraint PrecedenceConstraints

22. Some links https://msdn.microsoft.com/en-us/library/ms136025.aspx (MSDN Developers guide) https://msdn.microsoft.com/en-us/library/ms136025.aspx http://www.codeproject.com/Articles/18853/Digging-SSIS-object-model (SSIS Object Model) http://www.codeproject.com/Articles/18853/Digging-SSIS-object-model http://www.codeproject.com/Articles/547311/ProgrammaticallyplusCreateplus andplusDeployplusSSI (Creating and deploying 2012 package) http://www.codeproject.com/Articles/547311/ProgrammaticallyplusCreateplus andplusDeployplusSSI http://blogs.msdn.com/b/dataaccesstechnologies/ (Search for SSIS 2012 automation) http://blogs.msdn.com/b/dataaccesstechnologies/ http://blogs.msdn.com/b/dataaccesstechnologies/archive/2013/11/26/ssis- package-implementation-programmatically.aspx (Creating 2008 package) http://blogs.msdn.com/b/dataaccesstechnologies/archive/2013/11/26/ssis- package-implementation-programmatically.aspx https://msdn.microsoft.com/en- us/library/microsoft.sqlserver.dts.pipeline.wrapper.idtscomponentmetadata10 0.aspx (This is Data Flow control) https://msdn.microsoft.com/en- us/library/microsoft.sqlserver.dts.pipeline.wrapper.idtscomponentmetadata10 0.aspx

23. Questions? Can send questions to jeminijoseph@bi-datasolutions.com