Skip to main content
An application that reacts to events has five hard problems, and each is easy to get wrong:
  1. Two runs act on the same state at once.
  2. An event lands while a run is busy and is never seen.
  3. A run’s own writes wake it again.
  4. A burst of writes starts a burst of runs.
  5. A cursor kept somewhere drifts from the truth.
A declared listener takes all five on the platform’s side, and leaves its workflow one job: look at what changed, read the current state, write what should follow.

What a run can count on

  • It is woken, not fed. An event says something changed here. A run reads the current state and acts on it; the batch tells it where to look (which aliases, items and executions) and is not the state. Two hundred writes to one alias are one entry with distinct_by.
  • One lane runs one run at a time. concurrency divides a listener’s events in a session into lanes: serial has one lane and keyed one per key value, each with at most one run in flight. parallel runs up to max_parallel at once and waits when its slots are full (below).
  • A matching event reaches a run at least once, unless its batch is consumed without one. Delivery is at least once: a rare event reaches two runs, an event that commits late can arrive in a later batch than events with larger ids, and batches are not globally ordered. A declared retry delivers a failed run’s batch again (below). A listener takes effect up to about ten seconds after it is saved, and events routed before then are not delivered to it. A batch that is skipped (depth, session_not_open, unmatched or throttled), ends in access_error, or comes due while the listener is switched off in the session is consumed and never runs; every one but the switched-off batch leaves an outcome that says why. A reconcile catches up on the next batch.
  • It keeps no cursor. The platform holds the listener’s position in the stream. A run that wants more than its batch reads the stream itself with list_session_events; batch.first_id and batch.last_id are ids it can pass as after or before.
  • It is not woken by itself. Everything the run writes, and the run’s own execution.ended, carries the listener as source.listener and never matches it.
  • It is bounded. A burst becomes one batch per window, up to max_events entries; a chain of listeners stops at workflow.listener_chain_depth; fires are throttled at workflow.listener_fires_per_minute. See Limits.

Concurrency

Events that match while their lane is busy wait in its next batch: under serial and keyed while the lane’s one run is in flight, under parallel while all max_parallel slots are taken, and under keyed also while max_parallel keys are running. A workflow run holds its slot until its execution ends (completed, failed or stopped), and the waiting batch is claimed once a slot is free. A thread_item or object action frees its slot as soon as its outcome is recorded. An event the key path does not reach falls in one shared lane.

What you write

A reconcile: given the batch and what it reads now, write the state that should follow. Running it twice on the same state changes nothing, which is what makes at-least-once delivery, a failed run and a retried batch safe. Not needed, and not to be written:
  • a cursor, since the platform holds the position;
  • a lease, since a serial or keyed lane never has two runs in flight;
  • a sleep-and-poll loop, since the listener wakes the run;
  • an item posted to remember that an event was seen, since a repeated event is reconciled to the same result.

When a run fails

Its batch is consumed unless the listener declares retry. With retry, a run whose execution ends failed has its batch delivered again, up to attempts times. The retry after a run whose batch.retry is n waits min(delay_seconds x 2^n, 900) seconds: the first retry, after the first delivery (n = 0), waits delay_seconds, the second twice that. Each repeat is a batch of its own with its own outcome and receipt, carrying retry, and the failed run’s slot is held through the wait. Nothing else is retried: not a completed or stopped run, and not a batch that started no run. The fields and the limits are in the listener body. The failure shows three ways:
  1. The execution’s status. list_executions {session_id, listener} lists the runs a listener started. A run that was started and then failed has the outcome success, since the action was accepted; its own status is the result.
  2. The outcome on the listener’s state in the session, read with list_session_listeners: invocation_error when the action or its template failed, refused when a limit refused it. A retry’s outcome carries retry, which says which repeat it is (outcomes).
  3. A receipt, when the listener declares one: a thread item with the outcome’s status, posted for success, invocation_error, refused and throttled.
A reconcile catches up on the next batch, a retry, or a sweep requested through a clock.

One writer per alias

The guarantees are per listener. Two listeners, or a listener and a person, may still write the same thing. The rule an application follows is one writer per alias: each listener owns its key space, and a person’s input arrives as items on a thread rather than as overwrites of a listener’s objects. Where two writers must share one graph element, the graph’s if_rev guards each write (Component Code). A session object is replaced by each write to it.

A clock

Time is the one thing events cannot say: “nobody answered for thirty minutes” is the absence of an event. A reactive application gets a clock from a schedule that writes an object, with invocation_target_type: "object", every fifteen minutes:
The listener lists the clock’s alias, ops/clock, in its filter, beside everything else it reacts to:
Each tick is an object.written on the stream. It enters the same batch and run slot as every other event, so it never starts a run beside one already in flight. A schedule that started the workflow directly would bypass the slot and need a lease back. A schedule’s strings are literal, so the clock writes the same value on every tick. A periodic sweep is the same schedule pointed at a sweep alias, such as Missions’ app/missions/sweep.

Use cases

A time-based rule, such as a cell idle past its deadline, needs no listener of its own: the next run that reaches the cell decides it, whatever started that run, and a sweep request starts one on demand. No second listener catches a run’s end either, because the application starts no workflow but its own, and a run’s own events never match the listener that started it. The orchestrator’s batch stays lossless, with no distinct_by: keeping only the newest event per alias would drop the cell ids an earlier graph.changed in the window named. Its retry: {attempts: 2} means a failed run’s batch comes again after 30 s, then after 60 s; a batch whose third run fails is consumed, and an operator can request a sweep.