A message broker is a transport, not a manager. Teams that keep those two roles separate build systems that scale. Teams that blur them build systems that don't. That single distinction — control versus coordination — is the one most engineering groups get wrong the first time.
They wire up a broker, ship a few producers and consumers, watch the throughput graphs climb, and then spend the next year shuffling business logic in and out of routing rules, retry policies, and dead-letter queues. The architecture works. The team can no longer explain how.
The choice sits at the boundary between what the broker owns and what your services must never hand off. Getting that boundary right has less to do with the product on the box and more to do with the discipline behind it.
Two Philosophies Pull at Every Brokered Architecture
One camp treats the broker as a smart hub. Routing rules, content-based filters, message transformations, orchestration of multi-step workflows — all of it lives inside the messaging layer. The producer fires a generic event and trusts the pipe to figure out the rest. It looks tidy on a whiteboard.
The other camp treats the broker as plumbing. It moves bytes reliably, in order where that matters, and then gets out of the way. Business decisions live in services.
This is the smart endpoints principle popularized in microservices work: the transport stays minimal, the endpoints carry the intelligence. Less clever, more durable.
Both models ship products. The difference shows up two years in, when someone new has to reason about why a particular message ended up where it did.
What the Broker Legitimately Owns
There is real work that belongs inside the messaging layer, and pretending otherwise creates its own mess. A short list of what a broker is genuinely good at:
- Durable transport. Accepting a message, persisting it, and making sure it survives a consumer crash or a network blip. This is the core promise, and it is not trivial.
- Delivery semantics. At-most-once, at-least-once, and exactly-once are properties of the transport layer. Confluent's own reference on delivery guarantees lays out how consumer offset handling determines which one you actually get — and it is the broker plus its client library that enforces it.
- Routing and partitioning. Deciding which consumer group sees which stream, and how work is spread across partitions so multiple consumers can process in parallel without stepping on each other.
- Backpressure and buffering. Absorbing bursts so a slow consumer doesn't take down a fast producer.
These are hard problems. They are also generic problems, which is why you buy a broker instead of writing one.
What Your Services Must Never Hand Off
Anything that encodes what the business means by a message belongs in a service. Meaning does not travel well through routing rules.
- Idempotency. At-least-once delivery is the common default, which means the same message will land at a consumer more than once at some point. The consumer has to be safe under repeat delivery, and that is application territory, not something the broker can do for you.
- Business validation. Whether an order is allowed, whether a customer is in good standing, whether a transaction crosses a policy line — these are service decisions. A routing rule that silently drops messages based on a business condition is a bug waiting to be discovered.
- State transitions. Owning the database write that a message implies, and reconciling it with whatever gets emitted next, is service territory. When a single operation needs both a database change and a downstream event, keeping the two consistent is the service's job, not the broker's.
- Retries with meaning. The broker can redeliver. Only the service knows whether a failure was transient, whether the payload was malformed, or whether the downstream system is simply down and a retry in ninety seconds is pointless.
Draw the Line Before You Need To
Teams that stay out of trouble tend to write the boundary down early and defend it in code review. A one-page architecture note that says what the broker is allowed to do, and what has to live in a service, is worth more than any diagram. When a shortcut tempts someone to encode a business rule in a routing key, the note is what keeps the conversation short.
For a deeper walk through the same tension — producer, broker, consumer, and where the ownership shifts at each step — listen to this this Automatic.co episode about message Brokers: Whos Actually in Charge Here episode about message Brokers: Whos Actually in Charge Here, which turns the framework into concrete decisions engineering teams can apply on Monday.
Pick your side of the line deliberately. The broker will do what you configure it to do. The question is whether, a year from now, anyone can still tell you why.
