Still not what I would call 'proper' queuing (IMHO!) because ultimately it's still just Kafka and just append only: you can't 'atomically acknowledge+delete' individual messages anywhere in the stream, you only dealing with offsets and that's a fundamental thing that has many consequences (e.g. you don't get ordering between batches) and also means that you will get re-deliveries in some scenarios that you would not have with a system that implements proper queuing.
Let's not forget NATS.io which does queuing (and a whole lot of other things from at least once low latency pub/sub to request/reply, streaming and all the way to KV and Object store) very well and is very light weight to run and much simpler to run than Kafka.
From the exit proposal: "Over 97% of contributions to the NATS.io server were made by employees of Synadia and its predecessor company".
Also, when they applied for graduation in 2018, they were told no because most of the contributors work for Synadia (https://github.com/cncf/toc/pull/168). As of now 7 years later, it's still not graduated by CNCF. At this point it likely never will.
Putting yourself in their shoes, are your surprised they want to take it back from CNCF?
at the same time, in the graduation application, they seemed to strongly argue that they didnt even want outside contributors - saying that having community-developed client libraries and a synadia-developed core server was both proof of broad adoption and reason for future stability.
not surprised, I agree the decision makes sense for them.
It's just an unfortunate situation for the community. We don't know what the BUSL license would entail, and an open-source fork is unlikely to receive a lot of contribution (if historical contribution data is anything to judge by)
NATS does much more than pub/sub (with any fan out), it also does queueing in Core NATS (no persistence) and streaming including queuing over streams (with ack/nack/term). Don't be fooled into thinking NATS doesn't do queueing because it doesn't have a "Q" in the name :).
Yes, one of the many differences (advantages) of NATS JetStream over Kafka: with NATS you can ack explicitly each message individually, and even better if you set your stream to be in 'work-queue' mode it will also automatically (and atomically) delete the ack'd message from the stream (i.e. like a 'proper' queue) another difference with Kafka where you can't delete individual messages in the middle of a stream (only trim the tail end).
You can also 'negative ack' messages, specify a back-off period before the message is re-delivered (because NATS automatically re-delivers un-acked (or nacked) messages) when you can't temporarily process it, or 'term' a message (don't try to re-deliver it, e.g. because the payload is bad), or even 'ask for more time before needing to ack the message (if you are temporarily too slow at processing the message).
Ohhh, this is an awesomely informative and concrete message! Extremely useful, thank you!
I like everything about this: the ability to NACK individual messages, the specifying of a backoff period, _and_ to just discard a message f.ex. if you really cannot do anything about it. Super nice. I am grateful.
