message-queue-selection

Choose a work queue when each message goes to exactly one consumer and is deleted after acknowledgment. Choose Kafka (a log) when messages must be retained and replayable, or when multiple independent consumer groups each track their own offset through the same data.

A work queue delivers each message to exactly one consumer from a pool, then deletes it. Pub/sub delivers every published message to every subscriber for fan-out, so independent consumers like billing, inventory, and email each get their own copy.

A broker redelivers when it does not receive an acknowledgment, and an ack can be lost even after successful processing. The broker cannot distinguish these cases, so duplicates are the price of not losing messages. Make consumers idempotent to handle this.

Exactly-once features cover only a specific producer-to-broker-to-consumer path within one system's transactional boundary. They do not extend to side effects your consumer performs, such as charging a card or calling a third party, so idempotent handlers remain necessary.

Use a dead-letter queue whenever messages can fail deterministically, such as malformed payloads or references to deleted entities. After a configured number of failed attempts, the message moves to the DLQ so it stops blocking the pipeline, and arrivals should trigger alerts.

Global ordering forces all messages through a single serialized path, capping throughput and preventing parallel consumption. Most requirements are per-key, so use message group ids or partition keys to preserve order per entity while parallelizing across keys.