1. Stop Wasting Time & Resources: Performance Tune Your Jobs
Presented By:
Jeff Prom
BI Data Architect
Bridgepoint Education
MCTS - Business Intelligence, Admin, Developer
Stop Wasting Time & Resources: Performance Tune Your Jobs

2. Please Support Our Sponsors

3. Orange County User Group
Local User Groups
Orange County User Group
2rd Thursday of each month
bigpass.pass.org
Los Angeles User Group
San Diego User Group
3rd Thursday of each odd month
1st & 3rd Thursday of each month
meetup.com/sdsqlug
sqlla.pass.org
meetup.com/sdsqlbig
Malibu User Group
Los Angeles - Korean
3rd Wednesday of each month
Every Other Tuesday
sqlmalibu.pass.org
sqlangeles.pass.org
SQLSaturday Los Angeles
June 9th
SQLSaturday San Diego
September 15th

4. Discount Code: SSDISODNS
SQL Summit
Annual International Conference
November 6 -9 | Seattle, WA
2 Days of Pre-Cons
200+ sessions over 3 days
Over 5,000 SQL Professionals
Evening Networking Activities
Discount Code: SSDISODNS

5. What’s at Stake? Time Money Confidence
Operational support
On-call support
Money
Support Personnel
Hardware resources
Do more with less
Able to downsize server and save money?
Confidence
Deadlocks & Failures
Jobs Run Long
Delayed reports
Employee Morale / Loss of Employees
Marriages ?

6. Why Haven’t these been tuned yet?
If it aint broke don’t touch it!
Lack of qualified resources
Too complex
Older code. Nobody on the team is familiar with it anymore.
Mission critical process. Nobody want’s to work on it.
Lack of time

7. Which Job(s)? Where Exactly? Why is it slow? Fix it!

8. Which Job(s)? Which jobs are running long? Check Job History
Job Activity Monitor
Using Queries
Monitoring Tools

9. Where Exactly? Identify Job Step Package Where in the Package exactly
An SSIS Object?
A Query?

10. Where Exactly? Pinpoint the Pain Points Find the root cause
Check the logs
Agent History
SSIS Catalog Reports
SSIS Logs
Custom Logging
Stored Procedure logging

11. You Can’t Fix it if You Don’t Know What’s Broke

12. Strategies & Examples

13. May Not Always Be Where You Think
Think outside the box!
MySQL Example
The problem was because of activity happening on the source server
Backups
Jobs
Queries
Reports
Time Saved: 2 hours per day

14. Indexes Add indexes if needed!! Check for/add missing indexes
Easy to add but may not always be faster.
May need to break the query down
Test after you add!!
Clustered
Non-Clustered
Filtered

15. T-SQL Execution Plans Cut down the amount of records whenever possible
Estimated – Check for missing indexes
Actual – Check for missing indexes
Live Query Statistics
Cut down the amount of records whenever possible
If you need a single value, set it as a variable and assign it first. Then re-use it throughout the code.

16. T-SQL – Using Tables If there are a lot of joins, consider using:
Temp Tables
Derived Tables
Work Tables
Updates
Breaks the work down into chunks
Execution plans are much simpler
Can make a huge difference in performance

17. T-SQL – Things to Avoid Subqueries Cursors (RBAR)
Try to use joins instead
Cursors (RBAR)
Scalar-Valued Functions
Complex Derived Table joins
Use temp tables, etc instead
Recursive CTE’s if possible
They can be expensive operations

18. Scalar-Valued Functions
Executes for every record processed
RBAR
Kills the execution plan
Consider moving the logic directly into the code/stored procedure instead

19. T-SQL - Troubleshooting
Take original query
Turn the select <column names> into select count(*)
Break the query down into chunks to identify which part is slow
Note: This is also a great strategy for testing to make sure record counts are accurate

20. T-SQL – Troubleshooting (continued)
Try to get some sort of execution plan to start with
Start with a minimum set of code and keep adding joins and/or indexes to see where the breaking point is
May need to re-work the code to use temp tables, or some other strategy

21. SSIS – Cached Connections
Remove cached connections
It may be loading millions of records into memory to process a small amount of records
Multiple cached connections may strain the server
Add as a join in the source query instead if possible
Time Saved: 1 hour per day

22. SSIS – Data Conversion
When source data types are slightly different than destination data types
Seems like a good idea to convert in SSIS, but may actually be a really bad idea
Just let it go to the source table
Kills performance
Especially bad with text values
Time Saved: 3 hours per day

23. SSIS - Lookups Full cache
Loads all records into memory first before processing data
May be loading millions of records into cache to process a small amount of records
If doing many large lookups, consider doing lookups in the source stored procedure instead
Time Saved: 1 hour per day

24. Delta Processing Truncate / Reload Delta By Easy Slow
Not a good long-term solution with a significant amount of data
Delta By
Incremental ID value (inserts only)
Date/Time
Checksum
Use a control table or get max from data already loaded

25. Stage Filtering
Filter records out BEFORE even writing it to a stage table
Identify delta method
Conditional Split
Then add data transformations to only incoming records
Time Saved: 2+ hours per day
Went from 10 hours 40 minutes to 4 minutes 42 seconds in one case

26. Task Factory – SCD Component
Be cautious of hitting the same tables at the same time
Blocking
Deadlocks
Slow Performance
Consider using a stored procedure to do the SCD instead

27. Task Factory – Upsert Component
Blocking
Deadlocks
Consider Instead:
Using a stored procedure
Creating two Data Flow Tasks
One to do Updates
One to do Inserts
Result: No more deadlocks

28. Partition Swapping Works great with 2014 and below
Select into is much faster than insert because of parallelism
Select into (to load data)
Then swap partitions
Time Saved: over 13 hours*
*When combined with removing a scalar function
Demo

29. Parameter Sniffing Symptoms: Stored Procedure runs really slow
The code runs fast outside of the procedure
A recompile doesn’t fix it
Fix: Re-declare the incoming parameters
Time Saved: 3 hours

30. Thank You! Questions? Jeff Prom
Blog:
LinkedIn: