What’s New in SQL Server 2016 Integration Services Microsoft C+E Technology Training Solution Area Data Analytics Solution Business Intelligence Technology SQL Server EIM What’s New in SQL Server 2016 Integration Services In this video I will demonstration What’s New in SQL Server 2016 Integration Services. Peter Myers

Presentation Outline Integration Services Overview New SSIS Features Deprecated SSIS Features Discontinued SSIS Features Upgrading to SSIS 2016 Supported SSIS Features by Edition

Integration Services Overview Integration Services is a platform for building enterprise-level data integration and data transformation solutions Primarily designed to implement ETL processes, and its capabilities are useful in many other scenarios: Assessing data quality Cleansing and standardizing data Merging data from heterogeneous data stores Implementing ad hoc data transfers Automating administrative tasks Orchestrating Azure data processes

New SSIS Features The new SSIS features are grouped by category: Manageability Connectivity Usability and Productivity Other

New SSIS Features Manageability The new Manageability features are further grouped by sub- category: Improved Deployment Improved Debugging Improved Package Management

New SSIS Features Manageability ► Improved Deployment Support for Always On in the SSIS Catalog Support for Always Encrypted in the SSIS Catalog Incremental package deployment Single packages can now be deployed separately

New SSIS Features Manageability ► Improved Debugging New ssis_logreader database-level role in the SSIS Catalog New RuntimeLineage logging level in the SSIS Catalog New customized logging levels in the SSIS Catalog Created from existing logging levels Can capture statistics, and log events, optionally including context Can be assigned as the server-wide default logging level Column names for redirected error rows can be programmatically retrieved

New SSIS Features Manageability ► Improved Debugging [Continued] New IDTSComponentMetaData130 interface in the API Includes the GetIdentificationStringByID method New IDTSInputColumn130 and IDTSOutputColumn130 interfaces Both provide the LineageIdentificationString property

New SSIS Features Manageability ► Improved Package Management Improved experience for project upgrade Project-level connection managers continue to work as expected, and package layout and annotations are retained New data flow task AutoAdjustBufferSize property to automatically calculate bigger size for the data flow If the property is set to true, the data flow engine uses the calculated value as the buffer size, and the value of DefaultBufferSize is ignored By default, this property is set to false Reusable control flow parts

New SSIS Features Control Flow Package Parts A commonly used control flow executable (task or container) can be developed as a standalone part Parts are developed in an Integration Services project, by using the new package part editor Parts are stored with the .dtsxp file extension Parts within a project are available in the SSIS Toolbox, and external parts can be referenced Each instance of a part can be edited by using the Package Part Configuration dialog, allowing overrides to properties and connection managers

New SSIS Features Control Flow Package Parts [Continued] Parts are a design-time feature When a package using parts is saved, the package designer generates a .dtsxdesigner file containing part- related information The content of the .dtsx.designer file is merged with the .dtsx file, and is never deployed with the package An update to a part will require the package re-merging and redeployment

New SSIS Features Connectivity The new Connectivity features are further grouped by sub- category: Expanded On-Premises Connectivity Expanded Cloud Connectivity

New SSIS Features Connectivity ► Expanded On-Premises Connectivity Out-of-the-box support for new data sources: OData v4 Excel 2013, and 2016 Hadoop Distributed File System (HDFS) The SQL Server 2016 Feature Pack includes: Connector for SAP BW Updates to the Oracle and Teradata components by Attunity

New SSIS Features Connectivity ► Expanded On-Premises Connectivity [Continued] Updated ODBC components for SQL Server 2016 The ODBC Source and Destination components have been updated to provide full compatibility with SQL Server 2016 There is no new functionality, and there are no changes in behavior

New SSIS Features Connectivity ► Expanded Cloud Connectivity Download and install the Azure Feature Pack for new: Connection Managers: Azure Storage Azure Subscription Enumerators: Foreach Azure Blob Enumerator Tasks: Azure Blob Upload Task Azure Blob Download Task Azure HDInsight Hive Task Azure HDInsight Pig Task Azure HDInsight Create Cluster Task Azure HDInsight Delete Cluster Task Data Flow Components: Azure Blob Source Azure Blob Destination

New SSIS Features Connectivity ► Expanded Cloud Connectivity ► Example Orchestrate Azure HDInsight big data processes:

New SSIS Features Usability and Productivity Improved Installation Experience Upgrade is blocked if the SSISDB database belongs to an Availability Group Improved Design Experience SSIS designer creates and maintains packages for SQL Server 2012, 2014 and 2016 Improved SSMS experience Improved performance for SSIS Catalog views

New SSIS Features Other The Balanced Data Distributor Transformation is now part of SSIS The Data Feed Publishing Components are now part of SSIS Support for Azure Blob Storage in the SQL Server Import and Export Wizard Updated CDC components provide full compatibility with SQL Server 2016 CDC Designer and Service for Oracle by Attunity now supports Oracle 12c (SQL Server Feature Pack) The Analysis Services Execute DDL Task has been updated to accept Tabular Model Scripting Language commands (models at compatibility level 1200 only) The Analysis Services Processing Task and Partition Processing Destination are configured by using tabular objects—instead of multidimensional objects (models at compatibility level 1200 only)

New SSIS Features Other [Continued] Support for Built-In R Services SSIS already supports the built-in SQL Server 2016 R Services It is possible to use SSIS to extract data and load the output of analysis, and also to build, run and periodically re-train R models For further information, see the following blog post: Operationalize your machine learning project by using SSIS 2016 and R Services http://blogs.msdn.com/b/ssis/archive/2016/01/12/operationalize-your-machine-learning-project-using-sql-server- 2016-ssis-and-r-services.aspx

Deprecated SSIS Features There are no deprecated features in SSIS 2016 Deprecated features are those which are marked for removal in future releases https://msdn.microsoft.com/en-us/library/ms403408.aspx

Discontinued SSIS Features SSIS 2005 custom components will no longer work Discontinued features are those which are no longer supported, and which is likely to have been removed in the SQL Server 2016 release https://msdn.microsoft.com/en-us/library/bb500429.aspx

Upgrading to SSIS 2016 Upgrading to SSIS 2016 can involve: Upgrading SSIS Upgrading the SSISDB database Upgrading SSIS packages

Upgrading to SSIS 2016 Upgrading SSIS SSIS 2008, or later, can be upgraded The upgrade process will install SSIS 2016 side-by-side with the earlier version The SSISDB database should also be upgraded—unless you rely solely on the file system for package storage Before upgrading, Microsoft recommends that you run the Upgrade Advisor to report issues that you may encounter https://msdn.microsoft.com/en-us/library/mt668459.aspx

Upgrading to SSIS 2016 Upgrading the SSISDB Database In SSMS for SQL Server 2016, use the SSISDB Upgrade Wizard to upgrade the SSISDB database The wizard can only upgrade the database on the local instance Upgrade is blocked if the SSISDB database belongs to an Availability Group https://msdn.microsoft.com/en-us/library/mt668459.aspx

Upgrading to SSIS 2016 Upgrading SSIS Packages SSIS packages are not automatically upgraded Upgrade methods include: Executing a package with SSIS 2016 dtexec—this will perform a temporary upgrade Opening a package in SSDT, and saving the upgraded package Add a package to SQL Server 2016 SSDT project (with target server set to SSIS 2016) Open the project in SQL Server 2016 SSDT, and then use the SSIS Package Upgrade Wizard to upgrade all project packages Programmatically: Use the Application.Upgrade method of the Microsoft.SqlServer.ManagedDTS assembly https://msdn.microsoft.com/en-us/library/mt668459.aspx

Upgrading to SSIS 2016 Upgrading SSIS Packages [Continued] Script Task and Script Component scripts that explicitly reference ADODB.dll may not upgrade or run on machines without SSMS or SSDT installed In order to upgrade these scripts, it is recommended that you remove the dependency on ADODB.dll ADO.NET is the recommended alternative for managed code, such as VB and C# scripts https://msdn.microsoft.com/en-us/library/cc280546.aspx

Supported SSIS Features by Edition SSIS 2016 is supported by Enterprise, Standard and Developer editions Import/export Wizard, Built-in and Azure data source connectors and tasks are also supported by Web and Express editions Hadoop/HDFS connectors and tasks are also supported by Web edition Supported only by Enterprise edition: CDC Service and Designer for Oracle by Attunity Advanced adapters Advanced transforms https://msdn.microsoft.com/en-us/library/cc645993.aspx?f=255&MSPPError=-2147217396#IS

Supported SSIS Features by Edition Supported Only by Enterprise Edition Advanced adapters: Advanced transforms: High performance Oracle destination Persistent (high performance) lookups High performance Teradata Destination Data mining query transformation SAP BW source and destination Fuzzy grouping and lookup transformations Data mining model training destination adapter Term extractions and lookup transformations Dimension processing destination adapter Partition processing destination adapter Change Data Capture components by Attunity Connector for Open Database Connectivity (ODBC) by Attunity https://msdn.microsoft.com/en-us/library/cc645993.aspx?f=255&MSPPError=-2147217396#IS

Summary Sixth release of Integration Services Numerous new features and enhancements deliver improved manageability, connectivity, usability and productivity Upgrade to SQL Server 2016 is easily accomplished, it is possible to continue managing SSIS packages developed for SSIS 2012

Resources MSDN: SQL Server Integration Services https://msdn.microsoft.com/en-us/library/ms141026 MSDN: What’s New in Integration Services https://msdn.microsoft.com/en-us/library/bb522534.aspx MSDN: Features Supported by the Editions of SQL Server 2016 https://msdn.microsoft.com/en-us/library/cc645993.aspx?#IS SQL Server Integration Services (SSIS) Official Blog https://blogs.msdn.microsoft.com/ssis/

Resources [Continued] MSDN: Reuse Control Flow Across Packages by Using Control Flow Parts https://msdn.microsoft.com/en-au/library/mt620034.aspx MSDN: Azure Feature Pack for Integration Services (SSIS) https://msdn.microsoft.com/en-us/library/mt146770.aspx Downloads: Microsoft SQL Server 2016 Feature Pack Includes the SSIS Azure Feature Pack, SAP BW connector, and Attunity CDC components https://www.microsoft.com/en-us/download/details.aspx?id=52676

Resources [Continued] TechNet Virtual Lab: Exploring What's New in SQL Server 2016 Integration Services https://technet.microsoft.com/en-us/virtuallabs Use the search pane to locate the lab

© 2016 Microsoft Corporation. All rights reserved © 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, Microsoft Azure, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION