SQL Saturday #662 - Sioux Falls, SD Hosted by (605) SQL

Want More Networking and Training? Please be sure to visit the sponsors during breaks and enter their end-of-day raffles! Remember to complete session surveys! You will be emailed a link after the event or you visit http://www.sqlsaturday.com/662/Sessions/SessionEvaluation.aspx Event After Party At Will’s Training Table (Formally Beef O’Bradys) near the Pentagon starting at 5:45 PM Want More Networking and Training? (605) SQL meets the 2nd Tuesday of every month. https://605sqlusergroup.sqlpass.org

THANK YOU SPONSORS

Remote SQL Server Troubleshooting and Monitoring Using SSIS Don St. Cyr Don.StCyr@Gmail.Com @Machiavelli_too

Resources I Find Helpful Professional Microsoft SQL Server 2014 Integration Services (Brian Knight, Devin Knight, Jessica M. Moss, Mike Davis, Chris Rock) published by Wrox http://www.wrox.com/WileyCDA/WroxTitle/Professional-Microsoft-SQL-Server-2014-Integration-Services.productCd-1118850874.html The StackOverflow community https://stackoverflow.com/

All About Me! DBA at ScriptPro Formerly worked as a contractor to the U.S Army and the DoD on their Force Management database. Also was a technical trainer at a hospital and a systems administrator for a school district.

Why SSIS for Troubleshooting? A project to learn on Portability Security Automation Graphical interface Simple to work with Great documentation and community Preparation for future ETL/data mining projects Easily interact with other tools (i.e. Powershell & CMD line) And #1, why not?

Project Infancy Rpt Name Query 2 Query 3 Query 7 Query 6 Query 5 Glenn Berry’s “SQL Server Diagnostic Information Queries” (https://www.sqlskills.com/blogs/glenn/category/dmv-queries/). One Package to Rule Them All! Rpt Name Query 2 Query 3 Query 7 Query 6 Query 5 Query 4 Query 8 Query 9 Query 10 Query 11 The first version had 25-30 separate queries

2nd and 3rd Iterations Rpt Name Query 2 Query 3 Query 4 Query 5 Part 2 Query 8 Query 9 Query 10 Query 11 Query 12 Query 13 Query 14 The next few versions had 40-50 separate queries

Modular Stats The current version has 25-30 separate queries Improvements Over the Old Simple construction Flexible arrangement Individual self contained pieces can be exchanged/removed Base system runs with or without any of the individual packages *NOTE: Package name and text output file need to be named the same (see step 4 of Master Package slide) The current version has 25-30 separate queries

SPDBStats (SQL Performance Database Stats) We create a Data_Feed table with the important information that we will need to run the packages. The PACKAGE names, the CLASS and SUBCLASS (to create the order), the SAVEDAYS (how many days to save the output), the DIRECTORY where the files get staged, a VALID_TIME and an INVALID_TIME to turn the packages off and on as needed.

Master Package SP Counts the # of packages in the Data_Feed table Selects top 1 based on Class then Subclass and then executes the package Package exports to text files and saves them in the (Directory) folder Deletes all the text files (using the Package name) older than X (SaveDays) days in the (Directory) folder & subfolders If SaveDays = 0, the SP deletes all the txt files that match the Package name dot txt Selects the next top 1 & cycles through steps 2-5 until the count is reached Last package to run is Rollup which creates the dated folder and moves the files into it Master Package SP finishes by deleting any empty folders

SSIS Packages Make the individual SSIS packages however you would like I strongly encourage encrypting the packages and using the password in the Master Package SP to execute This prevents anyone else from adding in a package of their own for automatic execution Ensure the SSIS package name is exactly the same as the out file name It is important to consider data privacy and other confidential information, in our case HIPAA We avoid all PHI & PII in our reports, this info is about server health only!

Reports Each days information is saved as individual text files and saved into a dated folder The folders can be pulled back to a system that has Excel installed and full use of Powershell I also have a report in HTML for sites where getting the data back is more difficult, but this report is currently in beta

The Rollup Package Name Scope Data Type Value Expression The Rollup package uses these variables: Name Scope Data Type Value Expression Arg Rollup String /C robocopy C:\Temp C:\Temp\20170716 *.txt /mov /is "/C robocopy C:\\Temp " + @[User::Path] + " *.txt /mov /is" Date 20170716 (DT_STR,4,1252)DATEPART( "yyyy" , getdate() ) + RIGHT("0" + (DT_STR,4,1252)DATEPART( "mm" , getdate() ), 2) + RIGHT("0" + (DT_STR,4,1252)DATEPART( "dd" , getdate() ), 2) Folder C:\Temp @[User::Folder] Path C:\Temp\20170716 @[User::Folder]+ @[User::Date]

The Rollup Package (continued) The Rollup package uses two tasks: The File System Task configuration as shown here

The Rollup Package (continued) And the Execute Process Task configuration here:

Demos Data Feed table Master Package SP Invalid Time Save Days part 1 Save Days part 2 Save Days part 3 Report Generation

My Next Steps (or, How I’ll do it Differently) In the Master Package, use CMD line only (for DoD and other Gov clients) [currently uses a mix of CMD line & Powershell] - Currently in beta Create the report as a web page for viewing on site, where pulling the info back is difficult (DoD & other Gov clients) Export the same data to tables rather than text files and then use a new report to query the desired data from the tables Use the report (or a separate package) to create a single text file for offsite review

And thank YOU for coming out today! THANK YOU SPONSORS And thank YOU for coming out today!