Overview

MQ Telemetry Transport is a Client – Server Publish/ Subscribe messaging transport protocol and was initially named after the MQ series of IBM where the protocol was first developed for proprietary embedded systems in the late 90’s.

Nowadays, MQTT is considered as the protocol name, as opposed to be an acronym previously, and is the de facto standard by OASIS and later ISO.

Throughout the development of MQTT, ease of use was always the key concern and thus the protocol are said to be easy to implement on the client side and excel at transferring data over the wire since it is lightweight, simple, and a widely adapted open standard.

Prerequisite

1. A MQTT Client (for Desktop), e.g. MQTT.fx, a cross-platform client (Windows, macOS, Linux) built under the Eclipse Paho Project. Another option is Mosquito MQTT.
2. (Optional) Setup Mathwork free account for ThingSpeak & MATLAB Online

NOTE: From this point onwards, the term MQTT is referring to the MQTT Version 3.1.1.

Understand how MQTT works

The Pub/ Sub pattern of MQTT provides a decoupling mechanism between the Client and Server where the Broker acting as a medium between them (this can be extrapolated from the preview image).

• Space Decoupling: Publisher and Subscriber do not need to know each other, i.e. no exchange of IP address and port.
• Time Decoupling: Publisher and Subscriber do not need to run at the same time, e.g. sensor update at specific interval and only retrieved to end-user when needed.
• Synchronization Decoupling: Operations on both components do not need to be interrupted during sending or receiving.

The Fixed Header

There are three different aspects of MQTT include basic concepts (Publish/ Subscribe, Client/ Broker), basic functionalities (Connect, Publish, Subscribe) of MQTT and basic features such as Quality of Service, Retained Messages, Persistent Session, Last Will and Testament, Keep Alive and more.

In general, the protocol has three different headers vary by the package type.

Side note: Full list of Control Packet Types.

The Variable Header

A Variable Header includes a 2 byte Packet Identifier field and resides between the Fixed Header and the Payload. Although, not every Control Package requires the identifier field.

For the above example, every bits encoded in hexadecimal format, e.g. Keep Alive field of 60 seconds ($60_{DEC} = 3C_{HEX}$).

The Payload

A Payload is the final part of the packet and also optional depends on the Control Package type. In the case of the PUBLISH packet this is optional and commonly named as the Application Message, whereas it’s required in the SUBSCRIBE package and contain Topic Filter, Quality of Service.

Quality of Service

MQTT Protocol ensures high delivery guarantees with 3 levels of QoS serve to the constraint of embedded system communication in previous discussion.

• QoS 0: Guarantees a best effort delivery but not result.

• QoS 1: Guarantees that a message will be delivered at least once.

• QoS 2: Guarantees that each message is received only once by the counterpart

Beside that, MQTT also provides other mechanism for quality control.

1. Last will means in case of unexpected disconnection of a client all subscribed clients will get a message from a broker.
2. Testament and Retained message means that a newly subscribed client will get an immediate status update.

Implement MQTT for practical uses

Now that you have an idea of MQTT, let’s examine it in action using MQTT.fx to establish the connection and Wireshark to capture the packet.

NOTE: If you have a MathWork ThingSpeak/MATLAB account, sign in under User Credentials sub-menu (Refer to Prerequisite). Otherwise, leave at default.

Under tab Subscribe, use the below syntax to Subscribe any Public channel

channels/<channel_ID>/subscribe/json

Once you receive the channel feed, it means that

1. The connection was requested & Server Acknowledged Request
2. Subscription requested & Server Acknowledged Subscription
3. Message Published from Server to MQTT.fx client

This can be verified in Wireshark by select “Start Capturing Packet” icon in the Top bar and re-do every steps.

Side note: Use “MQTT” (protocol) or “1883” (port) as the filter for better analysis.

Under network layer section, it can seen that MQTT indeed working on top of the TCP/IP layer. Beside that, Wireshark provides deeper look into the protocol and exactly how the packet was formed.

This can breakdown into several headers as discussed, i.e. Fixed Header with Control Packet Type, Variable Header with Protocol Length/ Name, Connect Flags, Keep Alive fields and lastly the Payload with Client ID and Authentication.

Also, these information can be interpreted in raw hexadecimal format too.

Tips: You can hover over the HEX code in Wireshark to display the MQTT field that matches in the control bar at the bottom.

By using any DEC-HEX decoder, these code will yield similar result as expected.

Grab it here.

NOTE: In Excel, the built-in function CODE() convert character to ASCII and DEX2HEX() could seamlessly achieve the task and of course modify to analyze other MQTT packets.

ThingSpeak & MATLAB Application

The great thing about ThingSpeak & MATLAB combination is that it provides an easy-to-implement approach without any restriction of further improvement.

Many tools can be found under Apps top menu on your ThingSpeak profile with two main categories: Analytics and Actions. Also, MATLAB also provides excellent MQTT Support Toolbox for both reading and writing data directly to the ThingSpeak database.

% thingSpeakRead(<channelID>, <Name>, <Value>)
[data,time] = thingSpeakRead(12397)

% Or specific Fields with Output Format flag
thingSpeakRead(12397,'Fields',[1 4],'NumPoints',10,'OutputFormat','table')

% Now try to write to your private channel
% NOTE: Example Key Value

% Update multiple fields at once 
thingSpeakWrite(<channelID>, 'WriteKey', <write-key>,...
		[field1_val, field2_val, field3_val]) 

% Or automated process for batch authentication from multiple channels
thingSpeakAuthenticate(<channelID>, <Name>, <Value>)

Tips: Use help or doc prefix in MATLAB Console Window on instruction how to implement the commands and its parameters

Troubleshooting (Extra)

• Authorization issues in MQTT.fx

Make sure that you type in User-ID and MQTT-API-key for fields <User Name> and <Password> of User Credentials Tab respectively.

Both can be found under Account $\rightarrow$ My Profile on your ThingSpeak profile.

• Decimal-Hexadecimal Conversion

In case you could not access the Excel file for reading the content of a MQTT packet, try to use directly in MATLAB Online with the Dec2Hex built-in function.