Brian

 Brian D. Garraty  SQL Server DBA, Va Beach Public Schools  HRSSUG Leadership Team  Background in C++, VB, ASP, C#  NULLgarity.wordpress.com  >10 years experience with SSIS & DTS

 SSIS is often  Misunderstood  Disliked  Considered difficult to learn  Subject to ridicule

 SSIS is actually  Fairly simple (mostly)  Pretty good at a lot of things  Fun  The right set of tools for the job (sometimes)

 A desire to learn new things  The time to learn new things  Willingness to work through frustrations  Willingness to start over  The right project

 Brian D. Garraty, SSIS Defender  Brian D. Garraty, SSIS Apologist

 Type of Work You Do  Experience with SSIS  Rank your love of SSIS on Scale of 1 to 10  10 = God, Family, Country, SSIS  5 = Like it like a friend  1 = I want to fight it, really

 Starting from scratch – 100 Level  Control Flow  Data Flow  Configurations  Dynamic SSIS  What you can do with SSIS  What I have done with SSIS

 Defense Will Rest  If I knew then...  Resources  Questions

 SSIS in SQL Server Denali  How to do this Stuff  Demos

SQL Server Integration Services

 Bundled Extract, Transform, Load (ETL) Platform  Rewrite of Data Transformation Services (DTS)  Released with SQL Server 2005

 Developed in Visual Studio (err…”Business Intelligence Development Studio”)  Solutions, Projects, Packages

 Packages are  Deployed as files or to msdb  Executed via dtexec Utility (or wrapper to it)  SSIS Service  Runs on the server  Doesn’t do much

 Control Flow  Ordered workflow  Isolated tasks  Precedence Constraints  Data flow tasks  Data Flow  Move data from Point A to Point B  Manipulate data along the way

Where You Build Your Workflow

 Group related tasks  Can be enabled/disabled  Can serve as source for Precedence Constraints, even when empty

 Execute Process Task  Send Mail Task  Execute Package Task  Data Flow Task…

Where You Manipulate Data

What You See Isn’t Always What You Get

 Object property values  Variable values

 Determined Dynamic Values  Expressions evaluate at runtime  Example: Use current date to build unique log file name

 Instructed Dynamic Values  XML file  Example: Connection Strings

 What you see may not be what you get…  But it is what you asked for!

with SQL Server Integration Services

 Join data from Oracle with data from SQL Server?  No problem.  No linked server.  No OPENROWSET  No xp_cmdshell  No bcp

 Have two SQL Servers with 1500 databases, give or take, and need to pull data from all?  No problem.  No linked server.  No OPENROWSET  No xp_cmdshell  No bcp

 Multiple lines == Multiple threads  Often equals, at least

 Pull data off OLTP systems  Crunch it to your heart’s content  Location agnostic  OLTP = Online Transactional Processing

 Seemless integration with your comfort zone .NET  Stored Procedures  XML

 Can you have it me?  Can you only when X and Y but not Z?  This program needs to run when the extract finishes but only if…

with SQL Server Integration Services

 Extract data from numerous sources  Stage to database  Apply transforms and business logic  Write final data to files  Zip files and ftp to host  Track each run

 How will the data get from these 100 databases on these two servers to this one?  Can you not send any dups?  Can you us the dups?  Can you track the dups that have been fixed?

 Extract users, groups, & membership  Load data into SQL Server  Ensure it always works

The Dark Side of SSIS

 If you aren’t comfortable in a GUI-intensive IDE, SSIS will be a challenge

 But you do get what you asked for!  WYSIWYAF

 Inconsistent  Difficult to Master  Flying solo (No Intellisense)

 SSIS binds itself to your meta data and holds on tight  SSIS continually checks that things have not changed  Single column change in data source must trickle all the way down your data source

 Source Control? In my experience you can check things in, label them, and do “gets”.  Trying to do a Diff might induce panic  If needed, seek a third party

 The more dynamic you get, the less parallel you tend to get

 Just viewing details of SSIS components often leads SSIS to want to make changes to the file  Code review often leads to check out the entire project

 The server is not your machine  Early on, budget some time to work through issues upon deployment  Drivers, rights, O/S, platform, etc. may cause problems  Be patient!

 No Undo (yet)  Variable Scope is read-only  Variable Scope is context-default  Audience?

Best Practices and Words of Caution

 Benefits:  Encapsulation  Readability  Collaboration

 Packages in development must be files  You will have multiple file projects  To minimize changes on deployment, deploy as files  More info: bit.ly/lSniJSbit.ly/lSniJS

 String variable to store location of package  Use to build relative file paths  Input files (xsd, raw, etc)  Output files (xml, raw, etc)  Connection strings (packages, log files, etc)  More info: bit.ly/iuLctDbit.ly/iuLctD

 Store location of configuration file in environment variable  More info: bit.ly/ksAQOGbit.ly/ksAQOG

 In response to certain changes, delete  Take advantage of ease of creating a new item rather than suffer through unpleasant editing experience  Particularly true for file connections

 The New ETL Paradigm, Jamie Thomson  bit.ly/e17DUR bit.ly/e17DUR  Jamie Thomson’s Blogs  sqlblog.com/blogs/jamie_thomson sqlblog.com/blogs/jamie_thomson  consultingblogs.emc.com/jamiethomson consultingblogs.emc.com/jamiethomson

 SSIS Community Tasks and Components  ssisctc.codeplex.com ssisctc.codeplex.com  Locally grown SSIS Training  andyleonard.net andyleonard.net