1. Parallel Database Maintenance with 24/7 Systems and Huge DBs
SQLBits 2018

2. Grzegorz Łyp MS SQL Architect & Optimizer
MS SQL, .NET & Java Developer
Product and Project Manager
15 years in professional IT
10 years in MS SQL
Member of Data Community Poland
Blog:
Mail:
LinkedIn:
Facebook:

3. Agenda Background Database Maintenance Speedup maintenance tasks
Locking during database maintenance
Creating specific statistics

4. DEMOS WorldWideImporters
world-importers-v1.0

5. Background Once upon a time there was… a database
At the moment of start it worked for 8 hours a day…
DBA had rest of the time for maintenance

6. Background As the database owner company grew database usage grew
After certain amount of time the maintenace time went down to „0”
DBA lost his job as he could not do the database maintenance in such conditions
Did he try hard ?
Let’s try to satisfy 24/7 system maintenance needs!

7. Database Maintenance
Indexes
Statistics

8. Indexes Index fragmentation
What is the actual value of fragmentation ?
Is fragmentation bad or good ?
DEMO
1.sql

9. Indexes

10. Indexes Page splitting does not directly depend on index fragmentation
Read ahead is off when reading data from pages that are not neighbours
Data size necessary on physical storage and in memory higher for higher fragmentation
avg_fragmentation_in_percent is not enough for saving space - avg_page_space_user_in_percent gives additional details
Typically we assume that at 5-20% of fragmentation REORGANIZE index and over 20% we should REBUILD

11. REORGANIZE VS REBUILD REORGANIZE PROS REBUILD PROS Non-blocking
Faster than reorganize
No extra space required
Rebuilds not only leaf part of index
Non-blocking when ONLINE (Enterprise Only)
Can run on multiple processors
REORGANIZE CONS
REBUILD CONS
Slow
Blocking (ONLINE OFF)
Only leaf level part of the index
Slow ONLINE mode
Needs a lot of space

12. Conslusions Gathering information about fragmentation can be slow
REBUILD can only run on Enterprise systems with ONLINE = ON and need a lot of free space
REOGRANIZE is slow but it is not blocking even in standard edition
It can block on SQL Server 2014 SP2 (fixed in CU)
How to speed up index fragmentation gathering process ?
How to speed up REORGANIZE ?

13. Statistics Settings Manual statistics maintenance DEMO
Auto Create Statistics – very dangerous in case of unexpected statistics
You can’t see which statistics are used in query plan before SQL 2017
Auto Update Statistics – unpredictible moment of statistics update
Causes recompilations and possible changes in execution plans
For big tables refresh is too rare – especially for estimator before SQL 2014
Same for Auto Update Statistics Asynchronously
Manual statistics maintenance
Periodical
Updating statistics can take some time
Default sample for update can be not enough for rare records
DEMO
2.sql

14. Conclusions Auto Create Statistics Off Auto Update Statistics Off
Periodical statistics update (even everyday)
How to get proper sample ?
How to reduce time of statistics update ?

15. How to get proper update statistics sample ?
Important for seeking with direct values
All seek values should be in the sample
For optimizer 120 or higher values out of statistics are not of 0 cardinality when no direct value is used, but it might be not enough

16. Speedup Maintenance Tasks

17. Typical maintenance process
Iterate across indexes
For each index
Get index fragmenation
Defragment if necessary (REORGANIZE or REBUILD)
Update statistics if necessary

18. Proposed maintenace process
Iterate across indexes and make queue
Spread across number of threads each index in queue
Single thread task
Get index fragmenation
Defragment if necessary
Update statistics if necessary

19. Proposed maintenace process1 2 3 4 5 6 7 8 9 10 11 12 13
TIME

20. Proposed maintenace proces - queue1 2 3 4 5 6 7 8 9 10 11 12 13

21. Proposed maintenace proces 4 threads timing1 5 9 13
Thread 2 2 6 10
Thread 3 3 7 8 12
Thread 4 4 11
TIME

22. Proposed maintenace proces 4 threads timing vs Single thread1 5 9 13
Thread 2 2 6 10
Thread 3 3 7 8 12
Thread 4 4 11
TIME
Single thread 1 2 3 4 5 6 7 8 9 10 11 12 13
TIME

23. Proposed maintenace proces – queue – order by length13 4 11 3 5 6 9 10 1 2 12 7 8

24. Proposed maintenace proces 4 threads timing – queue ordered by length13 10 7
Thread 2 4 9 12
Thread 3 11 6 2
Thread 4 3 5 1 8
TIME

25. Proposed maintenace proces 4 threads timing – comparison13 10 7
Thread 2 4 9 12
Thread 3 11 6 2
Thread 4 3 5 1 8
TIME
Thread 1 1 5 9 13
Thread 2 2 6 10
Thread 3 3 7 8 12
Thread 4 4 11
TIME

26. Proposed maintenace proces
DEMO
3.sql, 4.sql, 5.sql, 6.sql, 7.sql, 8.sql

27. Proposed maintenance proces profit

28. Proposed maintenance proces
Optimal number of threads can be calculated based on real index operation times
The shortest possible time is the time of the longest index to maintain
More threads are applicable as each index mainenance is uniform to each other

29. Proposed maintenace process – v2
Iterate across indexes and make queue
Spread across number of threads each index in queue
Single thread perform
Get index fragmenation
Store index fragmentation in the table
Remove indexes below fragmentation limits
Spread across numer of treads each index in queue
Do index maintenance (fragmentation & statistics)

30. Proposed maintenace process – v2
Have to measure time for index fragmentation gathering and index maintenance separately
Should we split index defragmentation and statistics update ?
This might be version 3 – try yourself 
Is it better ?

31. Split maintenance task into parts
Read current fragmentation
Do index defragmentation
Update statistics 1 2 3 4 5 6 7 8 9 10 11 12 13
TIME

32. Three stages – better decisions
Thread 1 1 5 12 1 6 12 2 9
Thread 2 2 6 10 2 7 9 3 10
Thread 3 3 9 13 4 13 5 13
Thread 4 4 7 8 11 5 11 7 8 11
Fragmentation analysis
Index defragmentation
Statistics update

33. Proposed maintenace process – summary
Parallel maintenance shortens processing time
Each maintenance task split to 2 or 3 stages for better resources use
Operations time gather for better thread tasks allocation
Process can be fully automated

34. Parallel tasks - implementation

35. Parallel tasks – how to implement ?
Jobs
Need to create jobs
No performance balance built-in
Service broker
Need to create service broker objects (queues, services)
Number of parallel executions configurable and performance dependent
Created requests treated as system requests (cannot kill )
CLR
Unsafe assembly with multithread implementation
Need to implement breaking method (killing does not work)

36. Service Broker – parallel tasks implementation
T-SQL
Source Queue
Destination Queue
T-SQL
T-SQL
T-SQL
9.sql
DEMO

37. Service Broker - conclusions
Easy to implement
Some difficulties in error handling
Parallelism controlled by server with MAX_QUEUE_READERS suggestion
Faster thread allocation can be obtained with CLR

38. Conclusions

39. Conclusions Database index maintenance can consume time and resources
For 24/7 systems there are no technical breaks and every time we expect plan stability
Typical maintenance proces is single-threaded
Making maintenance multi-thread reduce time and make better use from resources
The final solution can be parametrized according to specific hardware abilities (number of cores, storage type)
We have non-blocking, adaptive, multi-threaded maintenace process that can work anytime

40. If we have more time…
BONUS DEMO

41. Just like Jimi Hendrix …
We love to get feedback
Please complete the session feedback forms

42. SQLBits - It's all about the community...
Please visit Community Corner, we are trying this year to get more people to learn about the SQL Community, equally if you would be happy to visit the community corner we’d really appreciate it.

43. Q&A
Thank You for the attention