1. Turbo-Charged Transaction Logs
12/2/2018 8:57 AM
Turbo-Charged Transaction Logs
Welcome!
First SQL Saturday?
Please visit sponsors.
Please thank the volunteers.
Please consider speaking.
Demographics. DBA / Dev? Versions?
Housekeeping – Presentation materials will be (are?) available for download at my site, and SQL Saturday site.
Any and all feedback is welcome.
David Maxwell
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. David M Maxwell twitter.com/dmmaxwell or twitter.com/upsearchsql
12/2/2018 8:57 AM
Who am I?
David M Maxwell
twitter.com/dmmaxwell or twitter.com/upsearchsql or
dmmaxwell.wordpress.com
Scripts / Slides available here.
Visit and find your ideal way to stay in touch.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. What’s really in the transaction log?
12/2/2018 8:57 AM
What we will cover…
What’s really in the transaction log?
How does the log manage transactions?
Potential problems, and solutions.
Performance tuning for the log.
What’s in the log? How does the log store transaction information?
How does the log manage transaction information that it stores? How does it control space usage?
What problems can we run into with the log?
How can we get better performance with the transaction log?
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4. What’s in the Transaction Log?
12/2/2018 8:57 AM
What’s in the Transaction Log?
The log is a record of all changes in a database.
Example: An UPDATE statement…
The 8k page with the record to be changed is read into memory.
A record of the change is written to the log.
Known as “Write-Ahead Logging”
ACID compliance
The page in memory is modified.
On COMMIT:
Commit Transaction is written to the log.
On ROLLBACK:
Compensation record is written to the log.
Page in memory is modified again.
Any change, whether it’s an insert, update, delete to creating or altering objects, or modifying database files, is considered a transaction, and recorded.
Much of what is recorded is not obvious. – Behind the scenes changes.
SQL Server stores data in 8k pages.
Tran starts, page is read into buffer pool memory.
A record of the change is written to the log first.
Then, the page is modified in memory.
On commit: transaction completion is recorded.
On rollback: compensation record is written.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. A word about CHECKPOINT
12/2/2018 8:57 AM
A word about CHECKPOINT
CHECKPOINT writes changed or “dirty” pages in memory to disk.
Happens independently of state of transactions.
Does not require COMMIT or ROLLBACK
Checkpoint writes modified pages back to disk.
Doesn’t matter whether or not the transaction has been committed or rolled back.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. DEMO: Viewing transactions with fn_dblog(). DEMO 12/2/2018 8:57 AM
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7. Example: 1GB log file, with 100MB growth.
How is the log managed?
Virtual Log Files
Number depends on create / grow size.
If running SQL 2014, growth less than 1/8 of current size = 1 VLF.
Example: 1GB log file, with 100MB growth.
SQL Server 2012 and lower: 8 VLFs, 12.5 MB each.
SQL Server 2014: 100MB, 1 VLF
Warning: Bug in SQL 2008 R2 for 4GB growth
Create / Growth Size
# of VLFs Created
< 64 MB
4 VLFs at 1/4 of the size
64MB to 1GB
8 VLFs at 1/8 of the size.
> 1GB
16 VLFs at 1/16 of the size.
SQL manages log file space usage by breaking up the log into Virtual Log Files, or VLFs.
Number of VLFs depends on create or growth size. Least you can have is 2.
If less than 64MB, 4 at ¼ of the size.
If between 64MB and 1GB, 8 at 1/8 of the size.
If more than 1GB, 16 at 1/16 of the size.
(Unless you’re running SQL 2014, then if the growth is < 1/8 of the current size, you get 1 VLF.)
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. Transaction starts – VLF becomes active. VLF is Truncated when:
12/2/2018 8:57 AM
VLF Usage
Transaction starts – VLF becomes active.
Must always be at least 1 active VLF.
VLF is Truncated when:
Transaction is Committed / Rolled Back AND…
If in SIMPLE recovery, CHECKPOINT has occurred.
If in FULL or BULK_LOGGED, log backup taken.
Log is circular. VLFs can be re-used.
Log grows if no VLF is available for re-use.
How SQL Server uses the VLFs.
When a transaction starts, the VLF the record is written in becomes active. More than one transaction can use space in a VLF.
Note: At least 1 VLF is always active.
When a VLF becomes full, SQL moves on to the next VLF.
VLF stays active until the transaction completed, AND…
1. CHECKPOINT in SIMPLE recovery mode OR
2. Log backup in FULL or BULK_LOGGED recovery mode.
Once a VLF has been marked inactive, it can be used again, making the log circular.
If no VLFs are available, then the log must grow to accommodate new transactions.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. DEMO: Examining VLFs with DBCC LOGINFO and fn_dblog. DEMO
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

10. Troubleshooting the Log
12/2/2018 8:57 AM
Troubleshooting the Log
Scenario 1: Log runs out of space.
Cause
Long-running or open transaction.
Secondary Truncation Point is involved.
Log backups not keeping up. (Or being done at all.)
Solution
Increase log backup frequency.
Kill open transaction.
Add another log file on a separate drive.
Caveats
Rollback is logged as well.
And could be single-threaded.
Determine what the transaction is doing.
Remember to remove additional log file when done.
Log runs out of space. Common problem.
Usually caused by a long-running heavily logged transaction, or insufficient log backups.
You could increase (or start) log backups, add an additional log file to let the transaction finish, or kill the open transaction.
Know that if you kill the transaction, the compensation records must be logged as well.
Determine what transactions are holding on to the log before killing anything. You may be able to kill smaller transactions to make room for the larger one to finish.
Remember to remove additional log files later.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Troubleshooting the Log
12/2/2018 8:57 AM
Troubleshooting the Log
Scenario 2: VLF Fragmentation.
Cause
Improper growth settings
Solution
Determine good growth setting.
Shrink log.
Pre-size log appropriately
Caveats
Watch out for ‘rubber-band’ logs.
Delays while growing.
VLF Fragmentation can cause slow startup or recovery times.
Usually caused by improper growth settings.
You can determine how big the log should be, shrink the log and pre-size it.
Watch out for rubber-band logs, that grow and shrink repeatedly, as this leads to file fragmentation.
Keep in mind that the DB stops during log growth, so try to manage proactively.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12. Identifying transactions using log space.
12/2/2018 8:57 AM
DEMO x2
DOUBLE DEMO:
Identifying transactions
using log space.
Cleaning up VLF Fragmentation.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13. Improving Log Performance
12/2/2018 8:57 AM
Improving Log Performance
Improve overall IO.
Faster Disk
File Fragmentation
Random vs. Sequential IO
Many log files on one volume = Random IO.
Pre-Allocate Log
Watch VLF count
Auto-grow as fail-safe
Do less work
Getting better throughput.
Improve IO.
Get faster disk overall. SSDs or arrays with good write cache are good choices for logs.
Make sure file fragmentation doesn’t get out of hand.
Keep log files on a volume to a minimum.
Pre-allocate space on the disk for the log file.
Watch that VLF count.
Leave auto-grow on as a fail-safe.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. Recovery times with different VLF counts.
12/2/2018 8:57 AM
DEMO
DOUBLE DEMO:
Recovery times with different VLF counts.
How index fill factor affects log performance.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15. Resources: SQL Skills blogs Transaction Log Management Thank You!
12/2/2018 8:57 AM
Thank You!
Resources:
SQL Skills blogs
Transaction Log Management
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16. David M Maxwell twitter.com/dmmaxwell or twitter.com/upsearchsql
12/2/2018 8:57 AM
Thank you!
David M Maxwell
twitter.com/dmmaxwell or twitter.com/upsearchsql or
dmmaxwell.wordpress.com
Scripts / Slides available here.
Visit and find your ideal way to stay in touch.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.