But It Sounded So Simple! Building a Messaging System in Service Broker Matt Wigdahl, ScriptPro LLC

Introduction  Matt Wigdahl, Director of Application and Database Development at ScriptPro LLC  Over 20 years of development experience, over 15 years of SQL Server experience    LinkedIn:

What the Heck is Service Broker?  Lightweight Enterprise Service Bus  Powerful  Mature – 10 years old (!)  Obscure

But What Does It Do?  Asynchronous messaging  Persistence  Ordering  Full transactional support  Useful for apps that require inter-database connectivity  Single- or multi-instance

Aren’t There Other Options? YYes! MMSMQ – Microsoft Message Queueing OOld (NT 4.0) UUses file-based queues and MSDTC SService Bus AAzure or Windows Server NNew FFull ESB architecture HHarder

So Why SSSB?  Happy medium LESB  Powerful but relatively simple  Plays to strengths of DB  Persistence  Transactional processing  Existing connectivity

But Mostly… AActivation! EEvent-driven processing IInternal (proc) or External (app)

What Would I Use It For?  Workload processing for distributed apps  Wigxpedia.com customers reserve hotel rooms, cars, and airline flights  Asynchronous triggers  Replacement for SQL Agent for Express  Messaging system for data and app events

Getting to Know Service Broker  Many types of components that work together  Message types  Empty  Free-form  “Well-formed” XML  Schema-bound XML

Getting to Know Service Broker Some More  Services  Logical endpoints of communication  Contracts and Conversations  Patterns of message exchange between two services  Reference message types and services  Queues  Physical repositories for messages  Specialized tables

Service Broker Overview Service A Queue A App A Service B Queue B Message (XML) Contract* *(Initator to Target using Message, followed by Target to Initator using Message) InitiatorTarget App B

The Project: Basic Messaging System  Desktop applications need to update proactively when data changes.  Interaction is via a single SQL Server instance.  Messaging is one-way (publish- subscribe).  Multiple receivers can subscribe to a given message.

The Message Type  Could include several pieces of data:  Table name  Type of modification  Key data  Routing information  WELL_FORMED_XML  XPath / XQuery to process data

Message Code

The Contract

The Services and Queues SSituation is slightly more complex here 11->N with routing means we need: SSomething to process messages SSomething to perform routing and dispatch PPerfect case for internal activation TTwo stages – pre- and post-dispatch

Service and Queue Layout Initiator Start Service Initiator Start Queue Initiator End Service Initiator End Queue Activation Procedure Receiver Start Service Receiver Start Queue Initiator End Service Initiator End Queue Initiator End Service Initiator End Queue Receiver End Service Receiver End Queue App A App B

Queue Code

Service Code

Routing Infrastructure  Need something to drive message routing  Map apps to services  Determine the messages each app wants  Use tables:

The Activated Procedure  Plain old stored procedure  Reads messages off queue and processes them  Figures out where it needs to send each message  Fires off new copies when there is more to do

Sending Messages  Format up some XML:  Generate conversation between services using proper contract  Send XML as the message payload

Receiving Messages  Use RECEIVE FROM (usually with WAITFOR):  Can embed in procedure – general procedure will need dynamic SQL.

Putting It All Together

The Devil in the Details  Error Processing  Security  Scaling and Optimization  Diagnostics

Errors and Error Handling TTwo major types of errors: TT-SQL errors during message processing EErrors received as messages FFirst Rule of Message Processing PProcess in a transaction IIf they fail, you can roll back and try again SSome errors (XML) are unrecoverable

Poison Messages  What happens when a message keeps failing?  Could spin forever  Would waste resources  Auto deactivate queue after 5 failures  Can turn off in 2008 R2+, but don’t  Better: Detect up front / Remove manually

Security  2 types of security  Dialog  At conversation level  Master key required  Remote service bindings / certificates  Transport  At instance level  Negotiated between endpoints

Scaling and Optimization  Standard guidelines:  Touch as little data as possible  Do as little work as possible  Small messages, minimal processing  Bottleneck analysis (activated procedures)

Service Broker Specific Tips SService Broker specific: RReuse conversations DDon’t just use one conversation RReceive multiple messages at once TThe “150 trick” (look it up)

Diagnostics RRoll your own LLog at various stages QQuery queues, Service Broker tables, DMVs sssbdiagnose CConfiguration and error analysis CCan handle more complex setups

More Resources MMSDN SStack Overflow RRemus Rusanu WWrote large amounts of Service Broker rrusanu.org