Welcome to Stem

Stem is a Dart-native background work platform that gives you Celery‑level capabilities without leaving the Dart ecosystem. This onboarding path assumes you have never touched Stem before and walks you from “what is this?” to “I can ship a production deployment.”

What is a task queue?

A task queue lets you push work to background workers instead of blocking your web or API process. The core pipeline looks like:

Producer → Broker → Worker → Result Backend

• Architecture at a glance

• Producer enqueues a task (e.g. send email).
• Broker stores and delivers tasks to workers.
• Worker executes tasks and reports status.
• Result backend keeps history and outputs.

In Stem, you can mix and match brokers and backends (for example, Redis for fast delivery and Postgres for durable results).

A minimal Stem pipeline

The core objects are a task handler, a worker, and a producer. This example keeps everything in a single file so you can see the moving parts together.

Define a task handler

stem_example.dart

  static final definition = TaskDefinition<HelloArgs, void>(
    name: 'demo.hello',
    encodeArgs: (args) => {'name': args.name},
    metadata: TaskMetadata(description: 'Simple hello world example'),
  );

  @override
  String get name => 'demo.hello';

  @override
  TaskMetadata get metadata => definition.metadata;

Set up the broker, backend, and worker

stem_example.dart

  final registry = SimpleTaskRegistry()..register(HelloTask());
  final broker = await RedisStreamsBroker.connect('redis://localhost:6379');
  final backend = await RedisResultBackend.connect('redis://localhost:6379/1');

  final stem = Stem(broker: broker, registry: registry, backend: backend);
  final worker = Worker(broker: broker, registry: registry, backend: backend);

Start the worker and enqueue work

stem_example.dart

  unawaited(worker.start());
  // Map-based enqueue for quick scripts or one-off calls.
  await stem.enqueue('demo.hello', args: {'name': 'Stem'});

  // Typed helper with TaskDefinition for compile-time safety.
  await stem.enqueueCall(HelloTask.definition(const HelloArgs(name: 'Stem')));
  await Future<void>.delayed(const Duration(seconds: 1));
  await worker.shutdown();
  await broker.close();
  await backend.close();

What You’ll Unlock

• Core pipeline – Enqueue tasks with delays, priorities, retries, rate limits, and canvas compositions, backed by result stores.
• Workers & signals – Operate isolate-based workers, autoscale them, and react to lifecycle signals.
• Observability & tooling – Stream metrics, traces, heartbeats, and inspect queues, DLQs, and schedules from the CLI.
• Security & deployment – Sign payloads, enable TLS, and run Stem via systemd/SysV or the multi-worker CLI wrapper.
• Enablement & quality – Use runnable examples, runbooks, and automated quality gates to keep deployments healthy.

Prerequisites

• Dart 3.3+ installed (dart --version).
• Access to the Dart pub cache (dart pub ...).
• Optional but recommended: Docker Desktop or another container runtime for local Redis/Postgres instances.
• Optional: Node.js 18+ if you plan to run the documentation site locally.
• A text editor capable of running Dart tooling (VS Code, IntelliJ, Neovim).

Onboarding Path

1. Quick Start – Build and run your first Stem worker entirely in memory while you learn the task pipeline primitives.
2. First Steps – Use Redis to run producers and workers in separate processes, then fetch results.
3. Connect to Infrastructure – Point Stem at Redis/Postgres, run workers/Beat across processes, and try routing/canvas patterns.
4. Observe & Operate – Enable telemetry, inspect heartbeats, replay DLQ entries, and wire control commands.
5. Prepare for Production – Enable signing, TLS, daemonization, and automated quality gates before launch.

Each step includes copy-pasteable code or CLI examples and ends with pointers to deeper reference material.

Next: Jump into the Quick Start to see Stem in action.