NATS vs Kafka: The Main Differences

In this post, we’ll examine the three main differences that, from Dhiria’s perspective, distinguish NATS from Kafka (and its derivatives, such as Redpanda).

NATS is a high-performance pub-sub (publish-subscribe) system, designed to be simple, lightweight, and fast. Unlike Kafka, which was built as a distributed log system with a strong focus on persistence and stream processing, NATS is primarily intended for service-to-service communication, offering low latency and high throughput. In its base version, NATS does not provide message persistence: messages are delivered to active subscribers and then discarded. However, thanks to the JetStream extension, advanced features such as persistence, durable consumers, replay, configurable retention, and more are available — enabling it to cover many use cases typically handled by Kafka or Redpanda.

One of the most interesting aspects of NATS is the ability to use subject wildcarding — that is, matching on the subject name to dynamically filter messages without the need to statically define topics.

In fact, subject names are hierarchical and can be composed of multiple segments separated by .:

mainname.<par1>.<par2>.<par3>.<par4>...

For example, I can publish a message to the subject mainname.impianto1.finito, and different consumers will behave differently. A consumer subscribed to mainname.> will receive all messages on subjects starting with mainname. A consumer subscribed to mainname.impianto1.* will receive all messages related to impianto1. A consumer subscribed to mainname.*.finito will receive messages from all plants, but only when finito is the second parameter.

This reduces load on clients (as they only receive messages matching the selected subject) and makes it easier to observe all messages related to a specific event or entity.

If a subject is meant to be temporary — for example, used for data transfer — message persistence can be configured on the client side using several options (maximum number of messages, message age, etc.):

from nats.js.api import StreamConfig await js.add_stream( name="file_data", subjects=["file.broadcast.*"], config=StreamConfig( max_age=60 # 60 seconds ) )

For clarity: a JetStream stream is the structure that manages the persistence of data for a collection of subjects. In the example above, the file_data stream will include data for all subjects matching file.broadcast.*.

In addition to the fact that Kafka doesn’t natively support this feature (though pattern subscriptions do exist), it’s important to note that Kafka is not designed to rapidly create and destroy large numbers of topics (especially when using ZooKeeper), as this leads to constant rebalancing in the cluster. NATS, on the other hand, supports tens of thousands of distinct subjects with ease.

The concept of a share group was recently introduced in Kafka, allowing many queue-like use cases to be addressed — though it still doesn't implement a true queue. This capability is supported natively in NATS.

A queue in a streaming system allows multiple consumers to receive messages in a balanced manner, all belonging to the same “consumer group”. Think of a queue of tasks to be executed and a pool of workers. These workers want to:

1. Retrieve tasks from the queue when available;
2. Do so continuously: while one consumer processes task N, others should be able to fetch task N+1;
3. Optionally ack individual messages — a key difference from Kafka’s typical ack mechanism.

Both Kafka and NATS support the concept of consumer groups, but with one important distinction. In Kafka, only one consumer per group can be assigned to a given partition. Therefore, to parallelize consumption (e.g., for task distribution) on a topic, you must:

1. Create multiple partitions, with all the operational complexity that entails;
2. Use a workaround: call .unsubscribe() while processing a message. This triggers a rebalance, and until .subscribe() is called again, the client is effectively offline.

This approach is not optimal: choosing the number of partitions in advance is non-trivial, and frequent .subscribe() and .unsubscribe() operations stress the Kafka cluster.

NATS simply doesn’t have this problem: consumers can receive a message and, while processing it, other consumers can fetch subsequent messages (without waiting for the ack). If a message is acked within a configurable time window (e.g., a few seconds), it’s considered processed; otherwise, it will be redelivered to another consumer.

For pull consumers, similar considerations apply, with a few differences:

• Pull mode is preferred over push mode;
• In push mode, NATS may “ignore” a consumer if it’s too slow to respond. If many messages arrive at once, consumers MUST implement concurrency to stay in the loop — otherwise, they’ll be skipped. Pull mode allows consumers to control their own pace.

To implement queue behavior in NATS:

• In push mode, simply use the same queue parameter for all consumers in the group;
• In pull mode, use a common name when creating each subscriber (i.e., create a durable consumer by specifying the durable parameter).

Again, unlike Kafka, there are no partitions or locks: when a consumer requests a message, it is directly assigned.

Transferring files composed of multiple chunks in Kafka is not straightforward. In fact, using a topic for data transport requires both the producer and the consumer to manage synchronization.

For example, a producer writing a file to a topic must keep track of the partition, the start_offset, and the end_offset of the messages that make up the file. This can be done via callbacks. The producer then communicates these parameters to the consumer, who will position itself on the correct partition and read messages from start_offset to end_offset, filtering only those related to that specific file.

With NATS, none of this is necessary: you can simply create a dedicated subject, such as data.<TASK_ID>. The consumer that needs to retrieve the messages creates an ephemeral consumer on that subject, reads all the messages, and reconstructs the file.

If message order matters, it’s enough to enable the ordered_consumer parameter, which ensures that messages are read in the same order they were written.

Messages in NATS are always produced in order on the same subject; there are no concepts like Producer or TransactionalProducer as in Kafka or Redpanda.

In addition, NATS also supports the Request/Reply paradigm.

This is essentially syntactic sugar: it allows sending a request on a subject that includes a temporary reply subject. Subscribers receive the request, process it, and respond on the provided subject.

The entire mechanism requires just a few lines of code. For more complex scenarios (e.g., multi-chunk responses), the pattern can still be implemented manually, without limitations.

NATS also provides Micro, a module for building microservices on top of NATS. These services behave like typical REST microservices but are discoverable via NATS and implement the request-reply mechanism with syntactic sugar. The main addition is the ability to make them programmatically discoverable.

While Kafka and Redpanda are the de facto standards in the industrial streaming world, NATS represents a promising alternative. Thanks to its flexibility, it enables applications that would be, at the very least, cumbersome or overly complex to implement with Kafka.

The insights shared in this post reflect the evaluation carried out by Dhiria in the pursuit of choosing the right tool for the right problem.