1. Devising your Data Movement Strategy for IoT
Dave McAllister
September, 2016

2. Data Movement Between Elements of IoT
Top level system considerations Architectures for different types of data movement Which standards to use; where and why

3. Relevant System Requirements
Scale of Devices Capabilities of Devices Message Exchange Patterns Messaging QoS
Traffic Characteristics
Security
IT Application Architecture & Technologies

4. Problem Statement ?
Firewall
Load Balancer

5. - Producer / Consumer

7. Message Bus

8. IoT Interlude

9. Typical IoT Cloud Deployment Architecture
Enterprise Bus
Firewall
CORE
Firewall
Connection Tier
Load Balancer
EDGE
FOG

10. System Considerations
Enterprise Bus
Connection Tier
System Considerations
Connections, queues, subscrptns
MEPs
Device-initiated pub/sub
Device-initiated request/reply
Cloud app-initiated pub/sub
Cloud app-initiated request/reply
QoS – MQTT QoS0, QoS1
Single point of contact for apps
Load Balancer

11. System Considerations
Enterprise Bus
Connection Tier
System Considerations
Connections, queues, topics
MEPs
Device-initiated pub/sub
Device-initiated request/reply
Cloud app-initiated pub/sub
Cloud app-initiated request/reply
QoS – MQTT QoS0, QoS1
Single point of contact for apps
Load Balancer

12. Challenges with Stateless Connect Layer
Enterprise Bus
Connection Tier
Challenges with Stateless Connect Layer
Device Queues
Where is the queue for each device for QoS1 subscriptions?
How does Connection layer find its queue/state? (among the millions….)
App to device MEP
How does Core layer find Connection layer server for given MQTT device for QoS0?
Where is the Q for a given device for QoS1?
Need massive subscription scalability updated in real time
Load Balancer

13. Functional Infrastructure Digital River Fog Cloud Edge Core Security
Capture
Analyze
Store
React
Predict
Things & Apps
Fog Computing
Legacy Integration
Real Time Analytics
GUIs and Dashboards
Legacy Gateways
Business Logic & Integration
[Big] Data Analytics
[Big] Data Store
Security
CICD & Device Management
Management and Monitoring
Event Messaging Bus
Multi Protocol Standards
Aggregation
Throttling
Connection Scaling
Filtering & Routing
Fan in -Burst Handling
Persistence
Slow Consumer Handling
Fanout – Lambda Stream
Infrastructure
Fog/ Gateway Layer
Cloud/ Datacenter Entry Layer
Cloud/ Datacenter Core Layer
Network
Talk about Data River – Messaging and Networking
Talk about Data Lake.
Digital River

14. Relevant Protocols & APIs Proprietary Proprietary/ JMS/JCA HTTP/REST
In Runtime
AMQP 1.0
Qpid / JMS
MQTT
Paho
 JMS is an API specification (part of the Java EE specification) that defines how message producers and consumers are implemented. JMS does not guarantee interoperability between implementations, and the JMS-compliant messaging system in use may need to be deployed on both client and server. On the other hand, AMQP is a wire-level protocol specification. In theory AMQP provides interoperability as different AMQP-compliant software can be deployed on the client and server sides. Note that, like HTTP and XMPP, AMQP does not have a standard API.
MQTT (Message Queue Telemetry Transport)
Open : created by IBM & Eurotech and donated to Eclipse “Paho” M2M project (OASIS standard in 2014)
Lightweight : smallest packet size 2 bytes (header), reduced clients footprint (C# M2Mqtt library 30 KB)
Reliable : three QoS and patterns to avoid packet loss on client disconnection
Simple :
TCP based
Asynchronous
Publish/Subscribe
Few verbs
Payload agnostic
CoAP
Several OSS

15. AMQP Pros Cons Supports most MEPs
High(er) power, processing and memory requirement
Supports all classes of service
Header fields are long
Supports detailed header fields
Enables portable encoding
Supports TCP and UDP
detailed header fields such as TTL, replyTo and user properties.

16. MQTT Pros Cons Lightweight API – low power consumption
No section for message properties
Message header as small as 2 bytes
No support for header fields
Support for major IoT MEPs
No message queue support
MQTT-SN for other connections

17. JMS Pros Cons Widely used in the datacenter
Power, processing and memory requirements for a device are relatively high.
easy integration among server applications.
No interoperability between suppliers’ JMS stacks (unlike REST, MQTT and AMQP).
Support for JMS APIs is embedded
Support for all classes of service

18. REST Pros Cons No device client library
Header fields are long (impacting bandwidth)
Simple is data loss is acceptable
No innate QoS
Lowest common connectivity (POST and GET)
No varied MEP
Reliability is app dependent

19. CoAP Pros Cons Like REST but no TCP
Same as REST (except QoS) Confirmable and non-confirmable QoS
Very fast communication in UDP
Subnet only without special handling
Very low power requirements
Only request/reply MEP
(pub/sub on futures)

20. MQTT versus CoAP MQTT CoAP Many to many (but Pub-sub) One to one
Event based
State based
No discovery (Pub-sub)
Supports content negotiation
OASIS / ISO
IETF
3 levels QoS
2 levels QoS

21. Fog Device Constraints Message Exchange Patterns Messaging QoS
Traffic Characteristics
Security

22. Fog Proprietary Proprietary/ JMS/JCA HTTP/REST In Runtime AMQP 1.0
PROTOCOL
API
Proprietary
Proprietary/ JMS/JCA
HTTP/REST
In Runtime
AMQP 1.0
Qpid / JMS
MQTT
Paho
CoAP
Several OSS

23. Edge # Concurrent Connections “Always on” Connections? Burstiness
Congestion / Prioritization
Multi-protocol?
QoS
Security

24. Edge Proprietary Proprietary/ JMS/JCA HTTP/REST In Runtime AMQP 1.0
PROTOCOL
API
Proprietary
Proprietary/ JMS/JCA
HTTP/REST
In Runtime
AMQP 1.0
Qpid / JMS
MQTT
Paho
CoAP
Several OSS

25. Core Less scale, more concentration Need enterprise features
Enterprise grade, popular APIs
Support streaming & microservices

26. Core Proprietary Proprietary/ JMS/JCA HTTP/REST In Runtime AMQP 1.0
PROTOCOL
API
Proprietary
Proprietary/ JMS/JCA
HTTP/REST
In Runtime
AMQP 1.0
Qpid / JMS
MQTT
Paho
CoAP
Several OSS

27. IoT is all about the data
Capacity & Availability Limits
Big Data RIVER Shock Absorber
Network
Processing
Storage
Outages
Upgrades
Inconsistent Aggregate
Input Stream

28. Big Data Big Data River Public Cloud Public Clouds Public Cloud PaaS
App
Big Data
Public Cloud
App
App
App
Big Data River
PaaS
App
App
App
IaaS
Private Cloud
App
App
App
On Premise

29. Addressing Scalability
Enterprise Bus
Connection Tier
Addressing Scalability
Command & Control sending message to a specific device

30. Addressing Scalability
Enterprise Bus
Connection Tier
Addressing Scalability
Command & Control sending message to a specific device
Device-initiated request/reply
Among millions of devices
Across 2 middleware tiers
Need Integrated addressing between Enterprise Bus & Connection Tier at massive scale

31. Addressing Scalability
Enterprise Bus
Connection Tier
Addressing Scalability
Load Balancer

32. Addressing Scalability
Enterprise Bus
Connection Tier
Addressing Scalability
Connection Tier Addressing
MQTT Topics
Enterprise Bus Addressing
Other topics & queues
Need to scale
In an integrated manner
Update in real-time
Handling bursts of changes

33. Singapore Land Transport Authority – ERP2 http://www. straitstimes
Next-Gen Electronic Road Pricing
Connect 1.5M vehicles (200K at a time) in real-time for pay per use road tolls
Suggest better routes, traffic management – real time and long term and value-added services/analytics
Routing
Server
MHI OBU
Payments
JMS
MQTT
200K Concurrent Connections
MQTT Termination
Request/reply for firmware upgrades
Push Notifications
Persistent Messaging
Shock Absorber
High Throughput
Fanout
Slow Consumer Handling
Analytics
0 0 0

34. Bus Monitoring
Collect real-time location, loading information and more thousands of buses
Optimize bus movement, fleet management, avoid bunching, forecast arrival time and more
Legacy REST
Legacy Gateway
REST/MQTT
Regulator’s Dashboard
JMS
ESB (if needed)
Analytics
CEP
DWH
REST/MQTT
Commuter’s Apps and RIAs

35. IoT Connectivity Concerns
Overall Architecture Requirements
Scalability: connections, queues, subscriptions
“single bus” connectivity for both apps & devices – solve the whole problem!
Unified addressing between cloud apps & devices
Single point of application connectivity
Support necessary QoS
Support all MEPs(?)
Device-initiated pub/sub
Device-initiated request/reply
Cloud app-initiated pub/sub
Cloud app-initiated request/reply

36. IoT Connectivity Concerns(cont)
At the Edge
Scalability: connections, queues, subscriptions
Protocols: MQTT, REST, CoAP(?)
Security:
Authentication & Encryption
Per-device authorization
deviceID in topics
/in/<devID>/<system>  ACL on /in/<devID>/>
In the Core
Single, simple point of contact to talk to any device
APIs & protocols to cloud apps
JMS, AMQP, Node.js, REST
App-to-device and app-to-app
Message filtering
Application horizontal scaling with non-exclusive queues
Integration with Hadoop

37. Can’t predict coming evolution of IoT, and changing infrastructure isn’t cheap. Need flexible data movement platform

38. Strong Architecture Smart Use of Standards Decoupling of concerns
Scales to meet needs of clients
Right features at each layer
Smart Use of Standards
Right standards
Features & exchange patterns
Optimized for needs of users
Security
Supported by key products and open source projects

39. Thanks! Dwmcallister dave.mcallister@solace.com
Download the free VMR Community Edition at