My key findings concerning RabbitMQ and AMQP
For my master thesis I am kind of looking into message brokers. To be specific I needed to compare MQTT in general to RabbitMQs implementation of AMQP.
The journey described here is based on nearly zero knowledge about all of these tools/technologies mentioned above. So this text is targeted for people with nearly the same knowledge base.
A little bit context first
Currently I work on a project that is located in between two message brokers. Think of it as shown in figure 1 below. On the left I have various kinds of services that are connnected to my project via RabbitMQ. My project, serves here as an controller, collector or aggregator. And on the right side I have a MQTT Broker which I am connected to as a client to send the data to another central service. The MQTT service on the right side was predefined and fix. The RabbitMQ service was choosen since I wanted to get to know RabbitMQ and needed to evaluate if it would be a good choice on the long run.
So with this layout in mind I started to evaluate if RabbitMQ was the way to go by trying to implement a simple concept and vision I had designed in before.
As stated above my project collects and aggregates data of services via RabbitMQ. The services that should get aggregated can be different over time by dancamically opping in or out. So I need to make sure they can register to opt in.
Whats the difference to MQTT
Currently I am talking about RabbitMQ and MQTT but whats the difference between them. First of all RabbitMQ is a software that uses the AMQP 0–9–1 protocol and MQTT is a communication protocol. Per definition MQTT describes itself on its website as the following below:
MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. For example, it has been used in sensors communicating to a broker via satellite link, over occasional dial-up connections with healthcare providers, and in a range of home automation and small device scenarios. […]
— http://mqtt.org
As stated above is MQTT a lightweight protocol on top of TCP/IP but can also be transported via websocket. AMQP instead is a more “bloated” or fully-fledged kind of protocol. MQTT simple has topics similar to URLs like ‘service/<service-id>’ where data can be published or which they can be subscribed to.
AMQP instead adds some intermediate layers to reach this level of “easiness” but perhaps is suited for use cases that I cannot guess.
AMQP explained by RabbitMQ
There are various important terms that are useful to understand when implementing an RabbitMQ client. First of all there are the terms Producer and Consumer. These are easy understandable as their meanings can be transcribed. A Producer produces data and puts it into RabbitMQ and a Consumer is devastated to get data and is listening for anything to appear.
When publishing or listening for data this is done via Queues. These are message buffers that hold the messages depending on their configuration (transient or durable). Durable queues retain their data after a reboot, transient queues do not. See the official RabbitMQ queue documentation for more information.
A little bit more complex is the term Exchange which I did not fully understand I think. I did interpret them as general filter for exchanging data for mainly for publishers but for consumers too. They specify a general ‘topic’ that is kinda feels like it should be named after the application name or the application ID. Exhanges need to be bound to queues to define to which queues they redirect the incoming data. This is defined by the type of an exchange.
To understand the described a little bit better look at figure 2 below.
As stated above Exchanges can have different types. These are direct, topic, headers and fanout. Let me explain them from easy to difficult. The easiest is the fanout type since its function is that it broadcasts its receiving data to all connected queues.
The direct exchange type is the default one and a little bit more complicated. Before defining a queue you need to specify an exchange first. If you do not do that the created queue will automatically be bound to a default, direct exchange. It automatically maps the queue name to the exchange routing key. Take a look at the following scenario.
I create a queue called queue-a and a queue-b. If I want to publish to queue-a I need to give its name as routing key. And only queue-a would receive the message.
The next one in the queue is the topic type. It is an interim between direct and fanout since it allows us to do multicast messages. The concept itself is the same as in the direct exchange, with all the exchange name and routing key mashup.
As the last one the headers type is a little bit more different than the others. As far as I understood it its purpose its to route packages based on metadata information inside the headers. Exchanges with this type ignore the routing key and route the packages based on the information of the attributes provided by the headers field itself.
Recap
I personally think of AMQP as extended version of MQTT. Due to Exchanges and Queues it is possible to make various ways of publishing and receiving messages possible. Therefore it is possible to configure the flow of data as you wish it. For me and my project it was a bit to complex so I used MQTT for it.
