Troubleshooting Service Broker Brian Gale Troubleshooting Service Broker

Troubleshooting Service Broker Brian Gale Troubleshooting Service Broker

SQL Summit Annual International Conference November 6 -9 | Seattle, WA 2 Days of Pre-Cons 200+ sessions over 3 days Over 5,000 SQL Professionals Evening Networking Activities

Please Support Our Sponsors

Let’s Not Distract Others Please silent your phone We can chat later Please don’t snore

WHO AM I Brian Gale DBA for the past 7 years at Vecima Networks Inc First time SQL Saturday presenter and 3rd time doing a SQL presentation Contact info: Twitter - @gh0ce Linkedin – https://www.linkedin.com/in/brianga

WHAT IS SERVICE BROKER? Microsoft definition – SQL Server Service Broker provides native support for messaging and queuing applications in the SQL Server Database Engine. This makes it easier for developers to create sophisticated applications that use the Database Engine components to communicate between databases. Developers can use Service Broker to easily build distributed and reliable applications.

WHAT IS SERVICE BROKER? Service Broker allows: - Asynchronous queries - Cross database communication - Cross instance communication - Cross server communication - Ordered delivery of messages - Ensured delivery of messages

DEFINITIONS Message – a piece of data you want to move across databases Message Type – definition of what a message may look like Contract – controls who can send each message type Queue – a place to store messages Endpoint – a listener for conversations Route – define the path to the remote SQL instance Service – binds a contract to a queue Conversation – what ties all of the above together

DEFINITIONS - ANALOGY Message – the information you want to get across Message Type – conversation topic Contract – set of rules for what each participant is allowed to talk about Queue – your minds Endpoint – your telephone Route – phone number Service – keeps you on topic Conversation – the talking

DEMO 1 Set up service broker, explain the setup steps, show that it works

WHERE WOULD YOU USE THIS? In-house CDC like system ETL system that needs some near real-time data Offsite data duplication

WHERE THINGS CAN GO WRONG Can’t decrypt certificates Endpoint not listening Route pointing to wrong endpoint Queue disabled Poisoned messaging

DEMO 2 Bad certificate due to database restore

CAN’T DECRYPT CERTIFICATE Happens when you restore from LIVE to TEST How to resolve: Regenerate the master key Alternate method: 1 – drop all certificates 2 – drop master key 3 – create master key 4 – re-create all certificates

WHERE THINGS CAN GO WRONG Can’t decrypt certificates Endpoint not listening Route pointing to wrong endpoint Queue disabled Poisoned messaging

POISON MESSAGE Definition – a message that cannot be processed Causes: message contains data that cannot be processed deadlock Resolution: get the bad message and discard it fix the data so the message can be processed Deadlock is rarely the cause of poison message unless you have retry set to 1

DEMO 3 Queue disabled and poison messaging

DISABLED QUEUE Cause: poison message intentionally disabled Fix: fix poison message and turn queue back on turn queue back on if appropriate

DATABASE OWNERSHIP Rarely happen Difficult to troubleshoot using profiler Fix – Change the database owner

EASY WAY The easy way to check things is: SQL Server Event Log DMV’s This is also a “Demo” slide

DMV DMV’s: sys.transmission_queue – holds potential errors sys.conversation_endpoints – status of conversations sys.dm_broker_connections – information about network connections

WHICH METHOD TO USE SQL LOG METHOD: - use this if production systems are down and downtime is expensive - if you don’t care about what is happening in the background - you are pretty sure you know what is wrong but want to verify PROFILER METHOD: - if you want to see what is actually going on - Log didn’t give you accurate or enough information - presentations

MY CHECKLIST Depends on the issue, but normally a process similar to: SQL Event Log on all affected servers sys.transmission_queue on all affected servers sys.conversation_endpoints on all affected servers sys.dm_broker_connections on all affected servers profiler Other times I may do that in reverse order.

Q&A

USEFUL LINKS Troubleshoot Service Broker Problems: http://www.sqlteam.com/article/how-to-troubleshoot-service-broker-problems Service Broker Troubleshooting: https://www.mssqltips.com/sqlservertip/1197/service-broker-troubleshooting/ Troubleshooting Service Broker (script): http://www.sqlservercentral.com/scripts/Maintenance+and+Management/31867/ My blog: https://sqlserverinablog.wordpress.com/