Skip to main content

Troubleshooting

Common issues when getting started with Stem and how to resolve them.

Worker starts but no tasks are processed​

Checklist:

  • Make sure the producer and worker share the same broker URL.
  • Confirm the worker is subscribed to the queue you are enqueueing into.
  • If routing is enabled, verify the routing file and default queue.
lib/troubleshooting.dart
class EchoTask extends TaskHandler<String> {
  @override
  String get name => 'debug.echo';

  @override
  TaskOptions get options => const TaskOptions(queue: 'default');

  @override
  Future<String> call(TaskContext context, Map<String, Object?> args) async {
    final message = args['message'] as String? ?? 'hello';
    print('echo: $message');
    return message;
  }
}

Helpful commands:

stem worker stats --json
stem worker inspect
stem observe queues

Routing file fails to parse​

Checklist:

  • Validate the routing file path and format (YAML/JSON).
  • Confirm STEM_ROUTING_CONFIG points at the file you expect.
  • Confirm the registry matches the task names referenced in the file.
  • If you use queue priorities, ensure the broker supports them.
lib/routing.dart
Future<RoutingRegistry> loadRouting() async {
  final source = await File('config/routing.yaml').readAsString();
  return RoutingRegistry.fromYaml(source);
}

final registry = RoutingRegistry(
  RoutingConfig(
    defaultQueue: const DefaultQueueConfig(alias: 'default', queue: 'primary'),
    queues: {'primary': QueueDefinition(name: 'primary')},
    routes: [
      RouteDefinition(
        match: RouteMatch.fromJson(const {'task': 'reports.*'}),
        target: RouteTarget(type: 'queue', name: 'primary'),
      ),
    ],
  ),
);

Helpful commands:

stem routing dump
stem routing dump --json
stem routing dump --sample

Missing or misconfigured result backend​

Symptoms: stem observe fails or task results never appear.

Checklist:

  • Set STEM_RESULT_BACKEND_URL for any workflow that needs stored results.
  • Ensure the backend URL uses the correct scheme (redis://, postgres://).
  • Confirm the worker is configured with the same result backend.
lib/persistence.dart
Future<void> connectRedisBackend() async {
  final backend = await RedisResultBackend.connect('redis://localhost:6379/1');
  final broker = await RedisStreamsBroker.connect('redis://localhost:6379');
  final stem = Stem(
    broker: broker,
    registry: registry,
    backend: backend,
  );
  await stem.enqueue('demo', args: {});
  await backend.close();
  await broker.close();
}

Helpful commands:

stem health --backend "$STEM_RESULT_BACKEND_URL"
stem observe workers
stem observe queues

TLS or signing failures​

Symptoms: health checks fail or tasks land in the DLQ with signature errors.

Checklist:

  • Verify STEM_TLS_* variables are set on every component that connects.
  • Confirm STEM_SIGNING_KEYS/STEM_SIGNING_PUBLIC_KEYS match across producers and workers.
  • Ensure STEM_SIGNING_ACTIVE_KEY is set and present in the key list.
  • Check DLQ entries for signature-invalid reasons.
lib/producer.dart
Future<void> enqueueWithSigning() async {
  final config = StemConfig.fromEnvironment();
  final broker = await RedisStreamsBroker.connect(
    config.brokerUrl,
    tls: config.tls,
  );
  final backend = InMemoryResultBackend();
  final registry = SimpleTaskRegistry()
    ..register(
      FunctionTaskHandler<void>(
        name: 'billing.charge',
        entrypoint: (context, args) async {
          final customerId = args['customerId'] as String? ?? 'unknown';
          print('Queued charge for $customerId');
          return null;
        },
      ),
    );
  final stem = Stem(
    broker: broker,
    registry: registry,
    backend: backend,
    signer: PayloadSigner.maybe(config.signing),
  );

  await stem.enqueue(
    'billing.charge',
    args: {'customerId': 'cust_123', 'amount': 4200},
    notBefore: DateTime.now().add(const Duration(minutes: 5)),
  );
  await backend.close();
  await broker.close();
}

Helpful commands:

stem health \
  --broker "$STEM_BROKER_URL" \
  --backend "$STEM_RESULT_BACKEND_URL"

stem dlq list --queue <queue>
stem dlq show --queue <queue> --id <task-id>

Namespace mismatch​

Symptoms: CLI sees no data or control commands return empty responses.

Checklist:

  • Ensure all processes (producer, worker, CLI) use the same namespace string.
  • For workers, confirm STEM_WORKER_NAMESPACE matches your CLI --namespace.
lib/namespaces.dart
Future<void> configureNamespace() async {
  final broker = await RedisStreamsBroker.connect(
    'redis://localhost:6379/0',
    namespace: 'prod-us-east',
  );
  final backend = await RedisResultBackend.connect(
    'redis://localhost:6379/1',
    namespace: 'prod-us-east',
  );

  await broker.close();
  await backend.close();
}

Helpful commands:

stem worker stats --namespace "stem"
stem worker ping --namespace "stem"

Migrations or schema errors​

Checklist:

  • Run the migration commands shipped with the adapter (Redis/Postgres).
  • Ensure your store URLs point to the migrated database/schema.
  • Set STEM_SCHEDULE_STORE_URL before running schedule commands.

Helpful commands:

stem schedule list

DLQ stalls or poison-pill tasks​

Checklist:

  • Inspect DLQ entries and replay only after fixing the root cause.
  • For repeat failures, consider lowering retries or adding task-level guards.

Helpful commands:

stem dlq list --queue <queue>
stem dlq show --queue <queue> --id <task-id>
stem dlq replay --queue <queue> --id <task-id>

Example enqueue that will land in the DLQ:

bin/producer.dart
  for (final invoice in invoices) {
    final id = await stem.enqueue(
      taskName(),
      args: {
        'invoiceId': invoice,
      },
      meta: {
        'createdAt': DateTime.now().toIso8601String(),
      },
      options: const TaskOptions(
        queue: 'default',
        maxRetries: 2,
      ),
    );
    stdout.writeln('[producer] queued invoice=$invoice taskId=$id');
  }

Control commands return no replies​

This usually means the control broadcast channel is not being consumed.

Checklist:

  • Ensure the worker is running and connected to the same broker.
  • If you use a custom namespace, pass --namespace to CLI commands.
  • Verify that the broker supports broadcast/control channels.

Helpful commands:

stem worker stats --json
stem observe workers

Task retries instantly or too quickly​

Checklist:

  • Confirm your task’s TaskOptions (maxRetries, visibilityTimeout).
  • Ensure the broker supports delayed deliveries (notBefore).
  • Check broker clock drift if delays feel inconsistent.
lib/retry_backoff.dart
  @override
  TaskOptions get options => const TaskOptions(maxRetries: 2);

Connection refused​

Checklist:

  • Verify Redis/Postgres is running and reachable.
  • Confirm the URL scheme (redis://, postgres://).
  • Ensure Docker ports are mapped (-p 6379:6379, -p 5432:5432).

Helpful commands:

stem health \
  --broker "$STEM_BROKER_URL" \
  --backend "$STEM_RESULT_BACKEND_URL" \

Still stuck?​

  • Review the Observability & Ops guide for heartbeats, DLQ inspection, and control commands.
  • Check the runnable examples under packages/stem/example/.