Spiral 1 Design Joshua Horwitz Mackenzie Sweeney

Architecture Components Server Communication: AMQP Server-side Application: Python Tornado or NodeJS RabbitMQ as message broker Client-Server Communication: SockJS & RESTful API Client-side Application: HTML, CSS, JS with jQuery Browser Communication: PubSubJS

Spiral 1 Proposal Design/Prototype client pub/sub environment with PubSubJS Design/Prototype Map abstraction using CMAPI specification Either modify Mapstraction or implement our own Maps to support for spiral 1: LeafletJS (at minimum) Google Maps V3 OpenLayers Design/Prototype UI (for maps) Design/Prototype Overlay Widget Extremely useful to test interoperability with other widgets Can incorporate this as an add-on to our widget Keep access control in mind, but don’t prioritize

Client Pub/Sub Implement using global pub/sub library that meets criteria: JS-native Widely used Actively maintained/developed Our recommendation is PubSubJS // Basic pub/sub operations PubSub.subscribe(‘map.overlay.create’, callback); PubSub.publish(‘map.overlay.create’, payload); // Can also use jQuery syntax $.pubsub(‘subscribe’, ‘map.overlay.create’, callback); $.pubsub(‘publish’, ‘map.overlay.create’, payload)

Map Abstraction Layer Goal: 0% code-rewriting, 100% interoperability Possible starting point is Mapstraction (BSD License) Fork and extend Or create our own: take what’s useful and improve Implement CMAPI over all supported maps At a minimum we should support LeafletJS (uses OpenStreetMaps) Google Maps V3 OpenLayers Should also support multiple data formats KML GeoJSON

How Does it Work? Create a map template interface For all maps, implement the same interface (think Java) User selects an underlying map to use (LeafletJS, Google, etc) Based on user choice, load in appropriate module Object guaranteed to have defined template Can subscribe to appropriate channels with known callbacks Essentially, its polymorphism Benefits No shim required: no unnecessary parsing One handler, one time Does not rule out subsequent reloads (swap underlying map)

User Interface Much of the UI will be provided by the maps provided Custom widgets (like overlay) should leverage: HTML5 CSS3 (responsiveness for different clients) JS/jQuery Priorities Modern feel! High degree of usability (learn from JC2CUI widget mistakes) Persistent storage in browser Get fancy later Could create our own Grid widget (industry folks hate JC2CUI’s) Provide cool analytics on mapped data (Jake/David )

Server Pub/Sub (looking forward) Leverage RabbitMQ Single exchange routes published messages to rooms All messages for one room end up at one queue Provides a pure FIFO structure to receive messages in order Producers (users): publish directly to the exchange via SockJS and the RabbitMQ Web-STOMP plugin Consumer (web server): handles the user subscriptions and sends the data to users via SockJS

Consumer Model 2-layer consumer model Web server Browser (user) Walkthrough 1.Web server consumes messages from room queues 2.Gets list of users in room from DB 3.Broadcasts message to all users over SockJS connections 4.Browsers (users) then receive messages 5.Channel parsed from message; appropriate callback executed

Benefits of Consumer Model Obvious benefits Establishes central authority on room state Update new users who join room Recover from disconnects New features Collect data on publishing patterns Save existing session state for quick restore later This approach has been proven in the commercial world It works! No need to modify current CMAPI channel payloads! HuffingtonPost Live pumps out over 20 million messages/week

AMQP Developed to solve the messaging interoperability problem in the financial industry Used now by 100s of critical systems Open standard Highly interoperable Use-cases Transient pub/sub distribution Reliable request-reply transactions

RabbitMQ Market leader MOM for AMQP Most popular choice on EC2 Bindings for multiple languages Java, Ruby, Python, C, C#, Erlang, Node, Perl Multi-protocol Support AMP 0-9-1, 0-9 and 0-8 AMQP 1.0 MQTT, XMPP, SMTP, HTTP, STOMP

RabbitMQ Built-in RESTful API & Web-STOMP plugin The creators have web messaging in mind! Reliability – can persist data and recover from failures Built-in clustering and transparent scaling Self-healing capabilities and extensive tooling/configuration Built-in access control techniques

RabbitMQ Web-stomp A simple bridge exposing STOMP protocol over emulated HTML5 websockets Makes it possible to use RabbitMQ from web browsers Comes with RabbitMQ Used in industry (see previous huffington post example) They handle over 20 million messages/week with web-stomp EXTREMELY SIMPLE to use

Web-STOMP Code // Stomp.js boilerplate var ws = new SockJS(' + window.location.hostname + ':15674/stomp'); var client = Stomp.over(ws); send = function(data) { client.send('/topic/bunny', {}, data); }; var on_connect = function(x) { id = client.subscribe('/topic/bunny', function(d) { // do some stuff }); }; var on_error = function() { console.log('error'); }; client.connect('guest', 'guest', on_connect, on_error, /');

So what is this SockJS? A ws polyfill that provides a ws-like JS object in ANY browser Automatic fallback if the channel can’t support websockets SocketIO does not do this Also works in browsers behind ws-unfriendly proxies If it doesn’t work, your network broke the Internet Well documented scaling and load balancing techniques Can utilize sticky sessions (JSESSIONID) or prefix-based LB Used in critical industry systems

Server Application Must support SockJS Our recommendation is Python/Tornado 2 nd option recommendation is NodeJS/Express The approach we are outlining has been proven in the commercial industry as a viable solution factory.com/upload/presentations/791/HuffingtonPost.pdf factory.com/upload/presentations/791/HuffingtonPost.pdf Extremely easy to implement simple prototype has been developed to proof feasibility