Message Queuing Telemetry Transport Lightweight Messaging for IoT MQTT Message Queuing Telemetry Transport Lightweight Messaging for IoT Some slides from Peter R. Egli’s presentation.

Outline Overview Functional Overview Topics Methods Protocol APIs

What is it? Lightweight connection protocol specifically designed for connecting devices “at the edge of the network” Runs over TCP/IP Pub/Sub architecture It is delay tolerant (disruption tolerant), i.e., works on intermittently connected links. Invented in 1999 Considered especially well suited for IoT applications Is the gateway into IBM’s Bluemix IoT backend, but it is an open standard. http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html

Pub/Sub Implementation The players: Publisher, Broker, Consumer(s) Consumer Data MQTT Broker Publisher Data Consumer Data Data Consumer

Example Drone 1 Position Operator Drone 1 Position MQTT Broker Collision Avoidance

Advantages Decouples consumers and producers NAT traversal! Unknown/variable number of producers Unknown/variable number of consumers Promotes interoperability NAT traversal!

Topics /home/rooms/kitchen/temperature Each published data specifies a topic Each subscriber subscribed to that topic will receive it (as a first approximation) Topic format: separator /home/rooms/kitchen/temperature sub-topic sub-topic sub-topic sub-topic

Topics Example OwnerType/DroneOwner/DroneID/DataType Two wild cards: Hobby/Mihai/Drone1/Position Hobby/Mihai/Drone1/BatteryCharge Hobby/Mihai/Drone1/GPSSatellites Hobby/Mihai/Drone2/Position Hobby/Rudra/Drone1/Position Business/Amazon/Drone20375/Position Two wild cards: “+” replaces a single level in the hierarchy: /Hobby/Mihai/+/Position -> The position of all of Mihai’s drones Can use more than one: /Hobby/+/+/Position -> positions of all hobbyist drones “#” replaces multiple levels in the hierarchy (must be last): /Hobby/Mihai/# -> All data from all of Mihai’s drones Neither wild card can be used for publishing

Durable/Transient Subscriptions If the subscriber disconnect messages are buffered at the broker and delivered upon reconnection Non-durable Connection lifetime gives subscription lifetime M1 M2 M3 M4 M5 M6 Subscription Durable Connection Connected Connected

State Retention Publications Retained The subscriber upon first connection receives the last good publication (i.e., does not have to wait for new publication) One flag set both in the publish packet to the broker and in the published packet to the subscribers.

Session Aware Last Will and Testament (LWT) – topic published upon disconnecting a connection Anybody subscribing to the LWT topic will know when a certain device disconnected

Stack Application MQTT TCP IP SSL optional TCP/IP Port: 1883 When running over SSL, TCP/IP port 8883

Publishing “QoS” (reliability) Three classes: 0 – unreliable (aka “at most once”) OK for continuous streams, least overhead (1 message) 1 – delivery “at least once” (duplicates possible) Used for alarms – more overhead (2 messages) 2 – delivery “exactly once” Utmost reliability is important – most overhead (4 messages) Reliability maintained even if the TCP connection breaks (intermittent connections) Separate QoS for publishing and for subscribing

MQTT Message Format Shortest Message is Two Bytes

Message Types

Message type

Remaining Length Total number of bytes in the message (after the “remaining length”). If <127 bytes, then encode it in one byte. For example 64 is 0x41. If more, in more bytes. Use byte 1 MSB to show that there is a byte 2. For example 321 = 65+2*128 is encoded as two bytes, least significant first. The first byte is 65+128 = 193. Note that the top bit is set to indicate at least one following byte. The second byte is 2. If more than two bytes needed, use more. Max message size is 256 MB = 0xFF, 0xFF, 0xFF, 0x7F Remaining Length = a*1280+b*1281+c*1282+d*1283

Packet Identifier (optional) Most messages (if reliability is desired) include a packet identifier which is then used to ACK the packet (e.g., SUBSCRIBE, Packet identifier 137, SUBACK 137)

Payload (optional) Required for some messages, forbidden for others, and optional for publish

CONNECT Protocol Name (MQTT), with two bytes length (4) Protocol Version (4) for 3.1.1

CONNECT Protocol Name (MQTT), with two bytes length (4) Protocol Version (4) for 3.1.1

CONNACK

PUBLISH

PUBACK and PUBREC (received) For QoS =1 For QoS =2

PUBREL (release) and PUBCOMP (complete) For QoS =2 For QoS =2

QoS - implementation At most once At least once Publishing of the topic is repeated if PUBACK is not received “in time”. Receiver does not need to store the data Exactly once Publishing of the topics is never repeated after PUBREC, receiver can delete topic data and reference after PUBREL

SUBSCRIBE

SUBACK

UNSUBSCRIBE

UNSUBACK, DISCONNECT, PINGREQ, PINGRESP

Keep Alive Timer If more than 1.5 * KeepAlive timer elapses with no message from the client, the server terminates the connection and publishes LTW topic

Last Will Example

MQTT Methods Connect Disconnect Subscribe UnSubscribe Publish Client waits for a connection to be established with the broker Disconnect Waits for the MQTT client to finish any work it must do, and for the TCP/IP session to disconnect. Subscribe Client requests the broker to subscribe to one or more topics UnSubscribe Client requests the broker to unsubscribe from one or more topics. Publish Send message to the broker

Example Java https://gist.github.com/m2mIO-gister/5275324 MqttClient myClient; MqttConnectOptions connOpt; static final String BROKER_URL = "tcp://q.m2m.io:1883"; static final String M2MIO_DOMAIN = "<Insert m2m.io domain here>"; static final String M2MIO_STUFF = "things"; static final String M2MIO_THING = "<Unique device ID>"; static final String M2MIO_USERNAME = "<m2m.io username>"; static final String M2MIO_PASSWORD_MD5 = "<m2m.io password (MD5 sum of password)>"; // the following two flags control whether this example is a publisher, a subscriber or both static final Boolean subscriber = true; static final Boolean publisher = true;

Example Java (connect) https://gist.github.com/m2mIO-gister/5275324 public void runClient() { // setup MQTT Client String clientID = M2MIO_THING; connOpt = new MqttConnectOptions(); connOpt.setCleanSession(true); connOpt.setKeepAliveInterval(30); connOpt.setUserName(M2MIO_USERNAME); connOpt.setPassword(M2MIO_PASSWORD_MD5.toCharArray()); // Connect to Broker try { myClient = new MqttClient(BROKER_URL, clientID); myClient.setCallback(this); myClient.connect(connOpt); } catch (MqttException e) { e.printStackTrace(); System.exit(-1); } System.out.println("Connected to " + BROKER_URL);

Example Java (subscribe) https://gist.github.com/m2mIO-gister/5275324 // setup topic // topics on m2m.io are in the form <domain>/<stuff>/<thing> String myTopic = M2MIO_DOMAIN + "/" + M2MIO_STUFF + "/" + M2MIO_THING; MqttTopic topic = myClient.getTopic(myTopic); // subscribe to topic if subscriber if (subscriber) { try { int subQoS = 0; myClient.subscribe(myTopic, subQoS); } catch (Exception e) { e.printStackTrace(); }

Example Java (publish) https://gist.github.com/m2mIO-gister/5275324 if (publisher) { for (int i=1; i<=10; i++) { String pubMsg = "{\"pubmsg\":" + i + "}"; int pubQoS = 0; MqttMessage message = new MqttMessage(pubMsg.getBytes()); message.setQos(pubQoS); message.setRetained(false); // Publish the message System.out.println("Publishing to topic \"" + topic + "\" qos " + pubQoS); MqttDeliveryToken token = null; try { // publish message to broker token = topic.publish(message); // Wait until the message has been delivered to the broker token.waitForCompletion(); Thread.sleep(100); } catch (Exception e) { e.printStackTrace(); }

Example Python (publish) https://pypi.python.org/pypi/paho-mqtt/1.1 import paho.mqtt.publish as publish publish.single("paho/test/single", "boo", hostname="test.mosquitto.org") single(topic, payload=None, qos=0, retain=False, hostname="localhost", port=1883, client_id="", keepalive=60, will=None, auth=None, tls=None, protocol=mqtt.MQTTv311)

Example Python (subscribe) https://pypi.python.org/pypi/paho-mqtt/1.1 def on_connect(mqttc, obj, flags, rc): print("rc: "+str(rc)) def on_message(mqttc, obj, msg): print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload)) def on_publish(mqttc, obj, mid): print("mid: "+str(mid)) def on_subscribe(mqttc, obj, mid, granted_qos): print("Subscribed: "+str(mid)+" "+str(granted_qos)) def on_log(mqttc, obj, level, string): print(string) mqttc = mqtt.Client() mqttc.on_message = on_message mqttc.on_connect = on_connect mqttc.on_publish = on_publish mqttc.on_subscribe = on_subscribe # Uncomment to enable debug messages #mqttc.on_log = on_log mqttc.connect("m2m.eclipse.org", 1883, 60) mqttc.subscribe("$SYS/#", 0)