MQTT Common Questions

Common Questions

If you are familiar with the web and email then you will probably find, as I did, that MQTT is very different. These are some of the questions I had, and saw on other sites and forums that may clear things up a little.

Q, Can multiple client publish to the same topic?

A- Yes

Q- Is is possible to know the identity of the client that published a message?

A- No not unless the client includes that information in the topic or payload.

Q- What happens to messages that get published to topics that no one subscribes to?

A- They are discarded by the broker.

Q-How can I find out what topics have been published?

A- You can’t do this easily as the broker doesn’t seem to keep a list of published topics as they aren’t permanent.

Q- Can I subscribe to a topic that no one is publishing to?

A- Yes

Q- Are messages stored on the broker?

A- Yes but only temporarily. Once they have been sent to all subscribers they are then discarded. But see next question.

Q- What are retained messages?

A- When you publish a message you can have the broker store the last published message. This message will be the first message that new subscribers see when they subscribe to that topic. MQTT only retains 1 message.

MQTT (Message Queuing Telemetry Transport) is an extremely lightweight connectivity protocol designed to cater to the Internet of Things. It is a machine-to-machine (M2M) protocol which works on publish/subscribe principle of transport of message. The small code footprint for its implementation and open-endedness makes it an ideal candidate for its application in IoT.

There are two types of devices participating in the protocol – brokers and clients. Brokers are the devices which host and relays data. Clients are the devices which access and modify data by publishing topics and receive other devices’ topics by subscribing. Let us take an example of the bulletin board. On the notice board, you might go through the topics which you have interest in. If you have something interesting to put up on the notice board, you are free to do that. To get the attention of the people who are interested in the topic, it is usually done in big bold letters. This is exactly how MQTT works.
Quality of Service (QoS)

The MQTT also gives a provision for three Quality of Service (QoS) – 0, 1 and 2.

MQTT QoS

    For QoS = 0, it ensures at most one delivery of the topic. It is equivalent to a ‘fire and forget’ concept.
    For QoS = 1, it ensures at least one delivery of the topic. It receives an acknowledgment. It makes sure that the broker has received the message.
    For QoS = 2, it ensures exactly one delivery of the topic. It gives us an assurance to the delivery of the message.

Load Testing HAProxy (Part 1)

This is the first post in a 3 part series on load testing HAProxy, which is a reliable, high performant TCP/HTTP load balancer.

Load Testing? HAProxy? If all this seems greek to you, don’t worry. I will provide inline links to read up on everything I’m talking about in this blog post.

Load Balancing with HAProxy for MQTT

Need to cluster

IBM has a Redbook for its own IOT service environment. The book also described the topologies of enterprise MQTT service architectures. So, to cluster some mosquitto brokers, we need some other tools.

Mosquitto is a single thread process. To scale out, we need tools to better monitor and manage them.

Cue in load balancing

I followed the steps in this nice tutorial, others you googled might be just as good, as HAProxy is very easy to install, configure and run.

Use HAProxy to load balance 300k concurrent tcp socket connections: Port Exhaustion, Keep-alive and others

I’m trying to build up a push system recently. To increase the scalability of the system, the best practice is to make each connection as stateless as possible. Therefore when bottleneck appears, the capacity of the whole system can be easily expanded by adding more machines. Speaking of load balancing and reverse proxying, Nginx is probably the most famous and acknowledged one. However, TCP proxying is a rather recent thing. Nginx introduced TCP load balancing and reverse proxying from v1.9, which is released in late May this year with a lot of missing features. On the other hand, HAProxy, as the pioneer of TCP loading balacing, is rather mature and stable. I chose to use HAProxy to build up the system and eventually I reached a result of 300k concurrent tcp socket connections. I could have achieved a higher number if it were not for my rather outdated client PC.

Step 1. Tuning the Linux system

Unexpected DDOS: Blocking China with ipset and iptables

Set up iptables

If you already have iptables set up and in use, skip this section and go straight to the ipset section.

Create a file where we can declare some rules to use:

sudo nano /etc/iptables.firewall.rules 

Inside there you'll want to paste the following:

 

CLEAN URL REWRITES USING NGINX

This article will cover how to easily implement Clean URLs (also known as Semantic URLs, RESTful URLs, User-Friendly URLs and Search Engine-Friendly URLs) using NGINX Web Server; currently the second most popular web server platform worldwide.

Not using NGINX web server? No problem, Apache users can check out my article Clean URL Rewrites Using Apache.

A Clean URL is a URL that does not contain query strings or parameters. This makes the URL easier to read and more understandable to users. 

Clean URLs are a high ranking factor for many search engines, but there are many other reasons why having Clean URLs can be important to your website. Check out our article Are your URLs Letting Your Website Down for more information.