INTEGRATION SERVICES IN THE REAL WORLD Brian Garraty, Database Administrator Virginia Beach City Public Schools

Who I Am  DBA for Virginia Beach City Public Schools  10 years working with SQL Server  5 years working with SSIS  MCITP: DBA Integration Services in the Real World, Brian Garraty

Who You Are  DBAs?  Developers?  DBAs with Development Background?  SSIS experience? DTS experience?  SSIS fans? SSIS haters? Integration Services in the Real World, Brian Garraty

Itinerary Integration Services in the Real World, Brian Garraty  Introduction  Real World SSIS Case Studies  Wrap Up  Q&A  Resources

Can you farm?” –Mitch Hedberg “You sure are a great cook! Integration Services in the Real World, Brian Garraty

What SSIS Buys Me Integration Services in the Real World, Brian Garraty  Flexibility  Performance  Rapid development  Dependability  Security

SSIS Rather Than… Integration Services in the Real World, Brian Garraty  xp_cmdshell  bcp/BULK INSERT  osql/SQLCMD  Linked Servers  Stand-alone.NET Projects

How I Learned Integration Services in the Real World, Brian Garraty  Read “Paradigm Shift” article  Rewrote monster, unwieldy stored procedure  True user documentation (blogs)  Rewrote first SSIS project

A lot of problems have started looking like nails.” – Me “Since SSIS has been my hammer, Integration Services in the Real World, Brian Garraty

Storing Snapshot of Active Directory in SQL Server Tables Case Study #1 Integration Services in the Real World, Brian Garraty

AD Snapshot: Requirements Integration Services in the Real World, Brian Garraty  Query Active Directory for:  User Accounts  Groups  Group Memberships  QA the results  Load data into SQL Server tables

AD Snapshot: Linked Server Approach Integration Services in the Real World, Brian Garraty  Create linked server to ADSI  Query linked server to temp tables  QA temp tables (2 nd iteration)  Replace data in destination tables from temp table

AD Snapshot: Linked Server Problems Integration Services in the Real World, Brian Garraty  Partial results (limited to 2000 results)  Timeouts without error message  “An error occurred” Errors  Not Configurable

AD Snapshot: SSIS Approach Integration Services in the Real World, Brian Garraty  Script tasks in Control Flow  Query AD via DirectoryServices.DirectorySearcher  Stage results in XML  QA  Execute SQL Task for row counts  Script task to check threshold and throw error

AD Snapshot: SSIS Approach (con’t) Integration Services in the Real World, Brian Garraty  Expression Precedence Constraint  For each item  Truncate via Execute SQL Task  Data Flow Task loads table from XML

Index Defragmentation Case Study #2 Integration Services in the Real World, Brian Garraty

Index Defrag: Requirements Integration Services in the Real World, Brian Garraty  Targeted defragmentation  Configurable thresholds  Support for 2005 syntax and features  Report-only mode

Index Defrag: Without SSIS Integration Services in the Real World, Brian Garraty  Many choices  Powershell  Stored procedure  SMO  Valid options - No problems slide here

Index Defrag: SSIS Approach Integration Services in the Real World, Brian Garraty  Generic SSIS package – can run on any server  Store thresholds in configuration  ForEach with SMO to loop dbs

Index Defrag: SSIS (con’t) Integration Services in the Real World, Brian Garraty  Dynamic SQL to query indexes  sys.dm_db_index_physical_stats  sys.dm_db_index_usage_stats  ForEach with NodeList to loop indexes  Script Task to build defrag statement

Other Projects Integration Services in the Real World, Brian Garraty  Monster Stored Procedure  Queries by Committee  Rendered Report Compare  Nightly Data Extracts via Secure-FTP

Wrap Up Integration Services in the Real World, Brian Garraty

Praise for SSIS Integration Services in the Real World, Brian Garraty  Parallelism  Expressions  Configurations .NET

Praise for SSIS (con’t) Integration Services in the Real World, Brian Garraty  Debugging  Data Viewers  Logging  Security  No linked servers  Not necessarily dependent on SQL Service Account

Gripes: Dark Side of SSIS Integration Services in the Real World, Brian Garraty  GUI intensive  Visual Studio outside DBA comfort zone  Requires.NET Skills Helpful  What you see != What you get  Expressions  Configurations

Gripes: Dark Side of SSIS (con’t) Integration Services in the Real World, Brian Garraty  Source Control Integration  Expressions – Syntax and Editor  Copy/Paste and ID’s  Sorting – DB vs. SSIS

My* Top 10 Tips and Best Practices Integration Services in the Real World, Brian Garraty 1. Store packages as files - always 2. ROOT_FOLDER and common directory structure 3. Naming Conventions - Project, Package, and Task 4. Use indirect configurations 5. Break project into multiple packages

My* Top 10 Tips and Best Practices Integration Services in the Real World, Brian Garraty 6. Do not sort in T-SQL 7. Use logging – expression can timestamp file name 8. Learn to use ForEach container with NodeList 9. Use CmdExec steps in SQL Agent with Proxy Accounts to run packages 10. Create separate SSMS solution to manage source control of all T-SQL code

Questions and Answers? Integration Services in the Real World, Brian Garraty

Resources Integration Services in the Real World, Brian Garraty  Jamie Thomson’s old blog (now on SQLBLOG)   Paradigm Shift Article on SSC   SQL Lunch   My Blog   #SSISHELP

Brian Garraty NULLgarity.wordpress.com Thanks! Integration Services in the Real World, Brian Garraty