1. Please note that the session topic has changed
Please note that the session topic has changed. Please visit My Ignite for current session description.

3. What´s new in SQL Server 2016 Integration & Master Data Services
Jimmy Wong
DAT 325

4. Overview
This session covers the major improvements in SQL Server Integration Services (SSIS) & Master Data Services (MDS) in the SQL Server 2016 release.
There will be slides, and demos.
There will be happy faces (if we did it right).

5. System Center Marketing
4/27/2017
Focus Areas
Performance
Manageability
Connectivity
Customer Feedback
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. SQL Server 2016 Master Data Services (MDS)

7. Master Data Services in SQL 2016
System Center Marketing
4/27/2017
Master Data Services in SQL 2016
Performance and Scale
Modeling and Management
Security and Administration
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. Do more. Faster. Massive improvements to performance and scalability
Microsoft Ignite 2015
4/27/2017 7:58 AM
Do more. Faster.
Massive improvements to performance and scalability
Capability
Heavy performance optimizations for the MDS backend system. Scale entities to 100 million members (and beyond).
Benefits
15x performance increase for Excel
Faster entity based staging
Reduce the overhead by optimizing security check
15x
© 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. System Center Marketing
4/27/2017
Do more. Achieve more.
Transaction log retention
Configurable settings for retaining the MDS transaction history table to enable automatic truncation.
Improved hierarchy management
Simplified the various hierarchy types, and made it easier to find and manage unused members. Manage hierarchies from Excel.
Improved business rule experience
Create and manage business rules from Excel, and use custom SQL logic as part of your business rules. Addition of NOT operator, and conditional ELSE logic.
Historical subscription views
See attribute change history in an easy to consume format.
Custom and compound indexes
Add a custom index on commonly used attributes to improve overall performance. Also allows combination of multiple attributes to create a compound index.
Modeling and Management
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

10. Entity Sync Share common entities between models Capability Benefits
Allows entities to be shared across models on the same MDS instance. You can configure one time, on-demand, or continuous sync of members.
Benefits
Reuse of common entities

11. Many-to-Many Relationships
Create M2M relationships in an derived hierarchy
Capability
Goes beyond the One-to-Many relationship mapping normally implied by derived hierarchies, allowing you to model Many-to- Many relationships.
Benefits
Allows modeling of M2M relationships
Replaces Collections

12. Simple Approval Flows
Require approval before changes are made to an entity
Capability
This feature allows the model administrator to specify that changes to a give entity require approval. Changes will be batched (Change Set), and only committed once they are approved.
Benefits
Create (single level) approval flows
Review changes before an entity is modified

13. Do more. With confidence.
System Center Marketing
4/27/2017
Do more. With confidence.
Security and Administration
Granular security permissions
Allows permissions to be set around read, write, create, and delete.
Multiple administrator roles
Support for Super User and Model Admin roles allows for multiple system administrators, and model level admins.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. User Experience Improvements
Capability
Redesign of the MDS web portal.
Benefits
Modern look and feel
Cross Platform
Usability improvement for administration tasks
Hierarchy managements improvements

15. Updated Excel Add-in Capability Benefits
General usability and performance improvements. More functionality moving from the Web Portal to the Excel add-in.
Benefits
Manage and edit business rules from Excel
Change display names for Domain Based Attributes
Filter attribute values in Domain Based Attributes based on another attribute (using hierarchies)

16. MDS SQL 2016 at a glance Performance and Scale Modeling and Mgmt
Security and Admin
Excel add-in
4-8x perf increase for SQL Server 2012 and SQL Server Up to 15x in SQL 2016.
Entity based staging
Loading data into MDS is now % faster than SQL Server 2014.
Model deployment
Faster deployment, and removed size limitations when deploying models with data.
Custom indexes
Create non-clustered indexes on one or more attributes to improve query performance.
Transaction log retention
Configurable settings for retaining the MDS transaction history table to enable automatic truncation.
Entity sync relationship
Share entities and data between models.
Many to many relationships
Create derived hierarchies that support many-to-many relationships.
Attribute filters
Filter domain based attribute values based on another attribute.
Business rule extensions
Apply user defined SQL scripts as business rule conditions and actions.
Granular security permissions
Allows permissions to be set around read, write, create, and delete.
Multiple administrator roles
Support for Super User and Model Admin roles allows for multiple system administrators, and model level admins.
Smarter caching of permissions
Reduces the overhead of adding member and attribute level security settings.
Member revision history
Track change history of members (type 2) through specialized subscription view.
Change sets and approval
Use change sets to save pending changes.

17. SQL Server 2016 Integration Services (SSIS)

18. System Center Marketing
4/27/2017
Three Focus Areas
Manageability
Connectivity
Usability & Productivity
BIG DATA
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. System Center Marketing
4/27/2017
Do more. Achieve more.
Better deployment
Incremental project deployment
Always On support for the SSIS Catalog
Better debugging
Column names included in error outputs
Customized logging levels in the SSIS Catalog
Better package management
Package templates
Automatic MaxBufferSize for Data Flow Task
Manageability
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. Always On support for SSISDB
Server & Tools Business
4/27/2017
Always On support for SSISDB
High Availability for the SSIS Catalog
Capability
DBA can set up Always On availability groups for the SSIS Catalog
Benefits
Consistent with SQL Server High Availability setup flow
Works for any user database which contains encrypted data based on database master key
Support SSISDB upgrade even when it has been added to the HA group
Easy to configure
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. Error Column Name Support
Server & Tools Business
4/27/2017
Error Column Name Support
No need to manually figure out the column name
Capability
Developer can see the error column name the data viewer
Developer can also see the IntToString lineage ID mapping in the log
Developer can also programmatically get the column name using the lineage ID
Benefits
No more manual work to figure it out
Speed up and simplify debugging
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. Custom Logging Level & Log Reader Role
Server & Tools Business
4/27/2017
Custom Logging Level & Log Reader Role
Customize your SSIS log with exactly what you need
Capability
Define custom logging levels for package executions within the SSIS Catalog
Additional ssis_logreader role to see the SSIS logs (no need to be DBA / Admin)
Benefits
Capture the exact information you need
Flexibility of roles and reporting
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Server & Tools Business
4/27/2017
Package Template
Give you code reusability
Capability
Save part of the package as template and reuse it in the design of other packages
Benefits
Reuse control flow logic across packages
Simplify package maintenance
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

24. Automatic MaxBufferSize in Data Flow Task
Server & Tools Business
4/27/2017
Automatic MaxBufferSize in Data Flow Task
Suitable max buffer size is calculated for you
Capability
Set AutoAdjustBufferSize to TRUE to automatically calculate the buffer size for your data flow task:
BufferSize = DefaultBufferMaxRows (design time) * RowSize (runtime)
Benefits
Max Buffer Size can go up to 2GB-1
Easier to size your buffers
Improved performance when working with ColumnStore in SQL Server
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. Do more. With more sources.
System Center Marketing
4/27/2017
Do more. With more sources.
Expanded Connectivity
More on-premises connectivity
HDFS Connectors
Hadoop HIVE/PIG Tasks
Excel 2013 / 2016 support
OData V4 Support
Teradata / Oracle updates
Execute SQL Task to run analytics with R in SQL Server 2016
More cloud connectivity
Azure Storage Connectors
HDInsight HIVE/PIG Task
BIG DATA
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

26. Azure Storage Connectors
Server & Tools Business
4/27/2017
Azure Storage Connectors
Enables hybrid ETL
Capability
Move files or read and write data Azure Blob storage
Benefits
Extend existing ETL pipeline with cloud storage, or cloud based SSIS execution through Azure VMs
Data preparation for cloud compute services such as HDInsight or Azure Machine Learning
Data archival to cloud storage
Easy enumeration of files in blob storage
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

27. Server & Tools Business
4/27/2017
HDFS Connectors
Enables data movement between Hadoop
BIG DATA
Capability
Move data to and from HDFS as source or destination
Benefits
Extend existing ETL pipeline with on-premises Hadoop and other big data processing
Data preparation for big data compute services such as Hadoop or Spark
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

28. Server & Tools Business
4/27/2017
HDInsight Tasks
Orchestrate and manage HDI clusters
Capability
Trigger HDInsight jobs and manage your HDInsight cluster life cycle directly from SSIS
Benefits
Integrate big data processing into your existing ETL flows
Filter and process raw cloud-born data using HDI before moving it to your data warehouse
Dynamically create your HDI cluster on demand, and remove it once processing is complete
Combine with Azure Storage connectors to extend your ETL environments to the cloud
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

29. Server & Tools Business
4/27/2017
Hadoop Tasks
Orchestrate and manage on-premises Hadoop
BIG DATA
Capability
Submit Hive and Pig jobs to your Hadoop cluster directly from SSIS
Benefits
Integrate big data processing into your traditional ETL flows
Combine with HDFS connectors to extend your ETL environments with big data processing power
Leverage the power of Hadoop as part of your Data Warehouse loads
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

30. Server & Tools Business
4/27/2017
OData V4 Support
Connect more with applications out of the box
Capability
Use the same OData source data flow task to connect to OData V4 feed
Benefits
Same OData source UI, no change
Automatic detection of OData version
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

31. System Center Marketing
4/27/2017
Do more. Faster.
Usability & Productivity
Faster Design Experience
Multi-version support in SSDT
Many designer improvements
Faster Install Experience
IS designer is now part of SSDT install
Faster Management Studio Experience
Performance improvement on SSIS Catalog views
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

32. Multi-Version Support in SSDT
Server & Tools Business
4/27/2017
Multi-Version Support in SSDT
Handle different SSIS package versions with a single designer
Capability
Author, execute, deploy and debug multiple versions of SSIS package from a single version of SSDT Designer.
Benefits
No need to maintain multiple version of Visual Studio, SSDT designer to manage different versions of SSIS packages
Enjoy the benefits from the latest visual studio shell and SSDT designer even when they are working on older version of SSIS package
Support SSIS 2012, SSIS 2014 and SSIS 2016
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

33. Usability & Productivity
SSIS SQL 2016 at a glance
Manageability
Connectivity
Usability & Productivity
Incremental Project Deployment
Incrementally deploy projects to the SSIS Catalog.
Error Column Name Support
See lineage identification string from the log / data viewer
Custom Logging Level
Customize your own logging level for more flexibility
Package Template
Achieve code usability by creating template part
Always On Support
Build your SSIS server farm with high availability
Automatic MaxBufferSize
Automatically calculate the suitable max buffer size for you
Azure storage connectors
Move data to/from Azure storage for hybrid data scenarios.
HDInsight Tasks
Orchestrate HDInsight jobs and manage your HDInsight cluster life cycle directly from SSIS.
HDFS Connectors
Mover data to/from Windows File system and HDFS
Hadoop tasks
Orchestrate HIVE jobs and manage the lifecycle of your on- premises Hadoop from the SSIS Control Flow.
Expanded Connectivity
OData V4, Excel 2013/2016, Teradata/Oracle
Execute SQL Task to run analytics with R Prepare data for model training and trigger R prediction
Multi-Version Support in SSDT
Ability to author, execute, deploy and debug multiple versions of SSIS packages from a single version of SSDT.
Designer Improvement
A number of enhancement and fixes on the designer capabilities such as drag and drop, resizing, etc
Unified Install for SSDT
Faster and easier to install
Catalog View Perf Improvement
Make viewing the SSIS catalog view faster

34. Demo - SSIS 2016

35. Complete your session evaluation on My Ignite for your chance to win one of many daily prizes.

36. Continue your Ignite learning path
Microsoft Ignite 2015
4/27/2017 7:58 AM
Continue your Ignite learning path
Visit Microsoft Virtual Academy for free online training visit
Visit Channel 9 to access a wide range of Microsoft training and event recordings
Head to the TechNet Eval Centre to download trials of the latest Microsoft products
© 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.