Broker Comparison

Stem ships with multiple broker adapters and planned integrations. This page explains what a broker does, how it differs from result storage, and how to choose the right transport for your deployment.

What is a broker?

A broker is the message transport between producers and workers. When you call Stem.enqueue, the broker stores (or, for an in-memory broker, retains) the task envelope and makes it available to workers. The broker is not where task results live; it handles delivery and settlement.

In Stem:

• Producer → Broker: publishes a task envelope.
• Worker ← Broker: consumes and acknowledges deliveries.
• Result backend: stores the task result/state (separate component).

Broker vs result backend

Use the broker for delivery, and a result backend for history:

• Broker: queues, leases, priority, delayed delivery, broadcast channels, and dead letters when the adapter advertises those capabilities.
• Result backend: task status, result payloads, heartbeats, group/chord metadata.

You can mix and match (e.g. Redis broker + Postgres backend) depending on durability and performance needs.

What a Stem broker must provide

The compatibility Broker facade includes common queue operations. Optional capabilities are advertised by BrokerCapabilities; do not assume every adapter provides every control-plane operation:

• At-least-once delivery with acknowledgements.
• Visibility/leases where the adapter implements LeaseBroker.
• Priority ordering where supportsPriorityOrdering is true.
• Delayed tasks where supportsDelayedDelivery is true.
• Broadcast channels where supportsBroadcastFanout is true.

If the broker does not support a feature, it should document the limitation. Planned adapters may not support full control-plane tooling until release.

Broker feature matrix

Broker	Status	Delivery	Delays	Priority	Broadcast/Control	Notes
Redis Streams	✅ Supported	At-least-once	✅	✅	✅	Lowest latency, great default.
Postgres	✅ Supported	At-least-once	✅	✅	✅	Durable, SQL-friendly; higher latency than Redis.
SQLite	✅ Supported	At-least-once	✅	✅	✅	Single-host file broker; broadcast fan-out is in-process only.
In-memory	✅ Supported	At-least-once while the process is alive	✅	❌	✅	Single-process only; broadcast is in-process and data is not durable.
RabbitMQ	🔜 Planned	—	—	—	—	No adapter is shipped in this checkout.
Amazon SQS	🔜 Planned	—	—	—	—	No adapter is shipped in this checkout.

Broker summaries

Redis Streams

Best default for most deployments: low latency, strong support for delayed tasks and leases, and mature operations tooling. Tune persistence (AOF or RDB) based on durability needs.

Postgres

A good fit when you prefer a single database dependency or want SQL visibility into queue state. Slightly higher latency than Redis; use a connection pool aligned with worker concurrency.

SQLite

Best for single-host development and demos. The SQLite broker uses polling- based delivery and supports broadcast fan-out only for subscribers in the same process. Use separate SQLite files for broker vs. backend to reduce WAL/write contention. See the SQLite adapter guide for setup and operational notes.

In-memory

Perfect for tests and local demos. Not durable and only works inside a single process.

RabbitMQ and Amazon SQS (planned)

These integrations are not shipped in this checkout. Do not use the matrix as a claim about their eventual delivery, lease, or control semantics.

Adapter Guidance

• Redis Streams is the default. Enable persistence (AOF) and replicate to a hot standby for fault tolerance. Configure namespaces per environment with ACLs. The packages/stem/example/redis_postgres_worker sample pairs Redis with Postgres for result storage.
• Postgres integrates tightly with the existing result backend for teams already running Postgres. Delivery leases are tracked in queue rows (for example via locked_until), so ensure the connection pool matches expected concurrency.
• SQLite is ideal for single-host development and demos. Use separate DB files for broker and backend; avoid producer writes to the backend.
• In-memory adapters are safe for smoke tests, but do not enforce priority ordering and lose data when the process exits.
• RabbitMQ & SQS have no released bindings here. Keep tasks idempotent when evaluating a future adapter, and verify its advertised capabilities.

Selecting a Broker

1. Start with Redis Streams unless your platform mandates a specific broker.
2. Consider Postgres when you need transactional enqueue/dequeue or prefer a single database dependency with built-in durability.
3. Use in-memory during development to simplify onboarding.
4. If you already operate RabbitMQ/SQS at scale, wait for a released adapter and verify its capability snapshot and lease/ack semantics.

Broker configuration quick start

Set the broker URL:

export STEM_BROKER_URL=redis://localhost:6379

Then bootstrap (pass a namespace if you want logical separation per env):

final broker = await RedisStreamsBroker.connect(
  Platform.environment['STEM_BROKER_URL']!,
  namespace: 'stem',
);

Or use the broker snippet entrypoints:

brokers.dart

Future<RedisStreamsBroker> connectRedisBroker() {
  return RedisStreamsBroker.connect('redis://127.0.0.1:6379/0');
}

brokers.dart

Future<PostgresBroker> connectPostgresBroker() {
  return PostgresBroker.connect('postgres://localhost:5432/stem');
}

brokers.dart

Future<SqliteBroker> connectSqliteBroker() {
  final file = File('stem_broker.sqlite');
  return SqliteBroker.open(file);
}

brokers.dart

InMemoryBroker createInMemoryBroker() {
  return InMemoryBroker();
}

For result storage, see the Persistence guide.

Need adapter limitations and behavior details? See Broker Caveats.