1. Troubleshooting Service Broker
Brian Gale
Troubleshooting Service Broker

2. Troubleshooting Service Broker
Brian Gale
Troubleshooting Service Broker

3. SQL Summit Annual International Conference November 6 -9 | Seattle, WA
2 Days of Pre-Cons
200+ sessions over 3 days
Over 5,000 SQL Professionals
Evening Networking Activities

4. Please Support Our Sponsors

5. Let’s Not Distract Others
Please silent your phone
We can chat later
Please don’t snore

6. WHO AM I
Brian Gale DBA for the past 7 years at Vecima Networks Inc First time SQL Saturday presenter and 3rd time doing a SQL presentation Contact info: Twitter Linkedin –

7. WHAT IS SERVICE BROKER?
Microsoft definition – SQL Server Service Broker provides native support for messaging and queuing applications in the SQL Server Database Engine. This makes it easier for developers to create sophisticated applications that use the Database Engine components to communicate between databases. Developers can use Service Broker to easily build distributed and reliable applications.

8. WHAT IS SERVICE BROKER?
Service Broker allows: - Asynchronous queries - Cross database communication - Cross instance communication - Cross server communication - Ordered delivery of messages - Ensured delivery of messages

9. DEFINITIONS
Message – a piece of data you want to move across databases Message Type – definition of what a message may look like Contract – controls who can send each message type Queue – a place to store messages Endpoint – a listener for conversations Route – define the path to the remote SQL instance Service – binds a contract to a queue Conversation – what ties all of the above together

10. DEFINITIONS - ANALOGY
Message – the information you want to get across Message Type – conversation topic Contract – set of rules for what each participant is allowed to talk about Queue – your minds Endpoint – your telephone Route – phone number Service – keeps you on topic Conversation – the talking

11. DEMO 1
Set up service broker, explain the setup steps, show that it works

12. WHERE WOULD YOU USE THIS?
In-house CDC like system ETL system that needs some near real-time data Offsite data duplication

13. WHERE THINGS CAN GO WRONG
Can’t decrypt certificates Endpoint not listening Route pointing to wrong endpoint Queue disabled Poisoned messaging

14. DEMO 2
Bad certificate due to database restore

15. CAN’T DECRYPT CERTIFICATE
Happens when you restore from LIVE to TEST How to resolve: Regenerate the master key Alternate method: 1 – drop all certificates 2 – drop master key 3 – create master key 4 – re-create all certificates

16. WHERE THINGS CAN GO WRONG
Can’t decrypt certificates Endpoint not listening Route pointing to wrong endpoint Queue disabled Poisoned messaging

17. POISON MESSAGE
Definition – a message that cannot be processed Causes: message contains data that cannot be processed deadlock Resolution: get the bad message and discard it fix the data so the message can be processed
Deadlock is rarely the cause of poison message unless you have retry set to 1

18. DEMO 3
Queue disabled and poison messaging

19. DISABLED QUEUE
Cause: poison message intentionally disabled Fix: fix poison message and turn queue back on turn queue back on if appropriate

20. DATABASE OWNERSHIP
Rarely happen Difficult to troubleshoot using profiler Fix – Change the database owner

21. EASY WAY The easy way to check things is: SQL Server Event Log DMV’s
This is also a “Demo” slide

22. DMV
DMV’s: sys.transmission_queue – holds potential errors sys.conversation_endpoints – status of conversations sys.dm_broker_connections – information about network connections

23. WHICH METHOD TO USE
SQL LOG METHOD: - use this if production systems are down and downtime is expensive - if you don’t care about what is happening in the background - you are pretty sure you know what is wrong but want to verify PROFILER METHOD: - if you want to see what is actually going on - Log didn’t give you accurate or enough information - presentations

24. MY CHECKLIST
Depends on the issue, but normally a process similar to: SQL Event Log on all affected servers sys.transmission_queue on all affected servers sys.conversation_endpoints on all affected servers sys.dm_broker_connections on all affected servers profiler Other times I may do that in reverse order.

25. Q&A

26. USEFUL LINKS
Troubleshoot Service Broker Problems: Service Broker Troubleshooting: Troubleshooting Service Broker (script): My blog: