Common Transport Rafael Schloming

Objectives

Scaling Engineering Time ● N experts in protocol & language -> 1 protocol expert & N language experts ● Protocol features implemented in 1 place rather than N – flow control, transports, addressing, etc ● Simplified interop ● Support Clients, Brokers, & Others ● Leverage 3 rd party contributions

Adoption ● Simple and useful Abstraction ● Broad platform support – Windows, Solaris, Posix, Mobile Devices,... ● Broad environment support – C, C++, Python, Perl, Ruby, Java,.Net, Lua,... ● Embedability ● Narrow & Stable ABI ● Minimal Dependencies

Architecture

Ubiquitous Block Diagram

Message Transport ● Send & Receive high level messages – includes data binding into the host environment ● structured headers ● structured messages (map message) ● Must not constrain the programming model – blocking, non blocking, threaded,... ● Transport, not monolithic client/server – proxies, concentrators, services,...

Message Transport ● Symmetric – used producer to broker & broker to consumer ● Point to point – direct vs brokered vs federated is a deploy time decision ● depends on global addressing ● Acknowledgment ● Flow Control

Ubiquitous Block Diagram ● Useful high level illustration ● Naive translation into call stack yields undesirable dependencies

Core Components ● Arrows depict formal dependencies ● Event Driver serves as Nexus of Dependencies – allows protocol, I/O and app to be decoupled

Event Driver ● Feeds protocol engine with events from: – I/O modules – Timers – Application ● Depends on well defined value objects – Message & Data abstractions ● Different threading models might use different drivers

Component Detail

Protocol Engine ● Interface is subdivided into top and bottom half ● Top half – defined in terms of links and messages – non blocking form of the messaging API ● Bottom half – defined in terms of bytes, frames & ticks

Protocol Engine ● Pure state machine – state is strictly characterized by inputs & outputs – allows driver to know when things may have changed ● Protocol implementation doesn't interact with I/O code

I/O Module ● Provides Byte/Frame Transport ● Decoupled from protocol entirely ● Easy to define narrow & stable plugin API

Module Diagram

Multi Protocol ● Public protocol plugin API ● Supports multiple versions of AMQP ● Could support other messaging protocols – XMPP, Stomp, SMTP

Multiple I/O Modules ● Decoupling allows – easy 3 rd party contributions – public plugin API

C Stack ● Enables Binding – Swig, JNI ● Easily integrated into standard libraries

Parallel Java Stack ● Java as a Platform vs Java as an Environment ● Java as a Platform requires pure Java Stack ● Java as a Language can make do with JNI

Testing Strategy ● Drive both stack implementations from a single python test suite ● Jython for Java Stack ● Swig for C Stack

Build Requirements ● Support bindings for multiple environments ● Support multiple platforms ● Transport and/or Clients available as independent artifacts