P2P Implementation: https://github.com/forgoes/bifrost
Abstract—The Message Queuing Telemetry Transport (MQTT [1]) protocol is a lightweight, publish-subscribe network protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks. An MQTT broker is a central component in the Message Queuing Telemetry Transport (MQTT) protocol, responsible for managing and facilitating communication between clients in a publish-subscribe architecture. However, designing a distributed MQTT broker to handle massive connections involves addressing a wide range of challenges related to scalability, reliability, latency, consistency, security, resource management, and operational complexity. Each of these aspects requires careful planning, robust architectural solutions, and advanced algorithms to ensure the system meets the demands of large-scale IoT applications. The proposed design is structured into distinct layers, incorporating the Raft consensus protocol [2] and consistent hashing, offers a robust solution for distributed MQTT brokers.
Keywords—MQTT, broker, distributed, cluster, cloud, consistent hash, LSM tree
Introduction
As the number of IoT devices continues to grow exponentially, a single-server MQTT broker faces inherent limitations in CPU, memory, and network capacity, making it inadequate for handling massive connections and high message throughput. One approach is a peer-to-peer architecture, within which packets must be routed through multiple nodes before reaching their destination, leading to increased latency and decreased message throughput as the number of nodes grows.
This paper presents an in-depth examination of a proposed design’s architecture, as depicted in Figure 2, comprising three distinct layers. Each layer encapsulates specific functionalities and contributes synergistically towards the overarching objectives of the system.
Leave a Reply