MQTT vs Kafka Notes

MQTT (Message Queue Telemetry Transport)

• MQ Telemetry Transport
• The choice for wireless networks
• Publish / Subscribe system

Key Concepts

• MQTT Session - Connection, Authentication, Communication and Termination
• Client Operations - Publish, Subscribe, Unsubscribe, ping
• Multiple implementations of client libraries and brokers (Mosquitto, JoramMQ...) exist and are virtually compatible
• MQTT just specifies the transport, and vaguely the application part (i.e. how data is handled and possibly stored, how clients are authorized...)
• Standard pub/sub protocol (with multiple implementations)
• MQTT as a communication protocol between several applications. It was designed to be extremely low light to fit into IoT and resource-constrained environment

Competing Tools

• Constrained Application Protocol (CoAP) 
• Simple Media Control Protocol (SMCP) 

MQTT Recommendations

• Machine-to-Machine (M2M) communication
• MQTT is designed for low-power devices
• MQTT purpose is to hold a communication channel alive on client-side without draining battery and to have a reliable messaging
• The edge devices speak MQTT protocol (for the benefits it has in edge environments). 
• Very easy to configure and use with open source tools, Lightweight with a relatively small data footprint, Varying levels of Quality of Service to fit a range of

Kafka

• The main motive behind Kafka is scalability.
• Apache Kafka is a message broker based on an internal "commit log": its focus is storing massive amounts of data on disk, and allowing consumption in real-time or later (as long as data is still available on disk)
• It's designed to be deployable as cluster of multiple nodes, with good scalability properties. Kafka uses its own network protocol.
• Kafka has no built in msg priority, poor security, heavy protocol
• Apache Kafka may deal with high-velocity data ingestion
• Kafka depends on Zookeeper in order to work properly
• Kafka is better suited for microservices
• Kafka is a messaging broker with transient store which consumers can subscribe and listen to. It's an append only log, which consumers can pull from.
• Specific message storing/distributing software, vaguley of the same family with its own protocol.
• Kafka is broker that can store large volume of data and for long time (or for ever). It was designed to be scalable and provide the best performances. 
• High-throughput, Distributed, Scalable, High-Performance, Durable, Publish-Subscribe, Simple-to-use
• Advantage of Kafka's strengths (replayability, based on an even sourcing architecture)

IoT environments combine both MQTT and Apache Kafka.

IoT Integration with MQTT and Apache Kafka from Kai Wähner

Apache Kafka is the New Black at the Edge in Industrial IoT, Logistics and Retailing
MQTT Overview
IoT Data Platform
MQTT vs Kafka Stackoverflow
MQTT vs Kafka Cloudera
MQTT vs Kafka Stack Share

Happy Learning!!!