Case study: EMQ's EMQX MQTT Platform

Case study: EMQ’s EMQX MQTT Platform

In this case study, we delve into EMQ’s flagship product EMQX, an ultra-scalable distributed MQTT platform written in Erlang/OTP. This article sheds light on the team’s structure, their approach to development, the technologies they used, their contributions to the open-source community, and the benefits of choosing the Erlang ecosystem for building the EMQX solution.

EMQ is a leading software provider of modern IoT data infrastructure. As a company, they have created multiple products. At the forefront stands EMQX that supports 100M concurrent IoT device connections per cluster while maintaining 1M message per second throughput and sub-millisecond latency. It also boasts more than 20K+ enterprise users, connecting 100M+ IoT devices, and is trusted by over 400 customers in mission-critical IoT scenarios, including well-known brands like HPE, VMware, Verifone, SAIC Volkswagen and Ericsson.

The Birth of EMQX: A Response to Digital Transformation

The emergence of digital transformation in different sectors has made the MQTT broker a crucial element in numerous digital solutions such as IoT, Connected Cars, Industry 4.0, smart energy, gaming, mobile applications, and more. This significant trend has motivated them to develop the EMQX MQTT broker.

EMQX started with a small and effective core. As the project grew, the biggest challenge was how to continue keeping the core small and effective while extending the project to support many different features. EMQX today supports around a dozen integrations with different data and stream platforms including relational databases such as Postgres and MySQL and streaming platforms such as Kafka and Pulsar.

To create a robust and innovative MQTT broker like EMQX, collaboration is key. EMQ boasts a globally distributed team with members from various corners of the world, representing major time-zones, including Brazil, China, Sweden, Ukraine, and the USA.

Programming languages, tools and libraries

Beyond the Erlang ecosystem, EMQX draws on various programming languages to create a suite of products, such as Haskell to build HStreamDB, Javascript for MQTT X, C for NanoQM and Neuron, GoLang for eKuiper and Java for XMeter.

In addition to programming languages, EMQX relies on a selection of tools, libraries, and frameworks to optimize performance and maintainability. Noteworthy mentions include Mria, HOCON, Esockd, Ecpool, and Quicer, as well as third-party dependencies such as Cowboy, cowboy-swagger, and Observer-CLI.

In EMQX project, they use rebar3/mix for Erlang/Elixir packaging release and created in house libraries, such as:

1. Mria to make Mnesia more scalable
2. HOCON to provide type-safe configs
3. Esockd as TCP/TLS socket server
4. Ecpool (making use of gproc) for generic pooling
5. Quicer for MQTT over QUIC

The umbrella project also pulls in around 100 third-party dependencies such as Cowboy and cowboy-swagger for REST API and specification, Erlang-rocksdb as their next generation storage engine and Observer-CLI for great observability.

The features are organized as independent Erlang applications in the mono-repo, and most of the features are developed by making use of the powerful callback(hook) mechanism provided by the core app “emqx”.

Benefits of using Erlang

Erlang is a language well-suited to developing distributed, fault-tolerant, and highly concurrent systems. The choice of the Erlang programming language for building EMQX offers numerous benefits, including:

1. Concurrency and Parallelism: Erlang has first-class support for concurrency and parallelism. It achieves this through lightweight processes that do not share state. This makes it possible to create systems with hundreds of thousands, or even millions, of concurrent processes. In EMQX, one Erlang process serves one MQTT client for state isolation, and MQTT message broking is as simple as an Erlang message sending.

2. Fault Tolerance: Erlang was designed for fault-tolerant systems. It provides mechanisms for error recovery and process isolation, which allows individual processes to fail without affecting the entire system. This means if an MQTT’s client runs into an error which caused it to crash, the exception is confined to only affect this one single client, but not other clients nor other components in the system.

3. Distributed Computing: Erlang provides robust primitives for distributed computing. Messages can be transparently sent between processes, regardless of whether they are running on the same machine or different machines in a network. This makes it well-suited to building distributed systems.

4. Hot Code Swapping: Erlang supports hot code swapping, allowing developers to upgrade a running system without downtime. This is particularly useful for systems that require high availability.

In conclusion, EMQX stands as a shining example of the power of the Erlang ecosystem in building scalable, fault-tolerant, and highly concurrent IoT infrastructure. With its global team and innovative approach to development, EMQX continues to play a vital role in enabling IoT solutions worldwide.