Skip to main content

Async

The tasks a running service holds, the way one delivery passes through them, and the order a shutdown stops them in.

The tasks of each dispatch form on the app runtime, and the dedicated threads beside itApp runtime: Tokio, multi-thread by defaultSubscription streams of the connected brokersdefaultone task runsevery stepworkers(n)one loop taskhands them outthreads(n)one feeder taskfills the ringsn worker tasksan inbox of onen ringsof eight slotsTracked tasks: and_after,hooks, retry_after timersn OS threadsa Tokio runtimeon each
The diagram as text
  1. App runtime: Tokio, multi-thread by default: default one task runs every step; workers(n) one loop task hands them out; threads(n) one feeder task fills the rings; n worker tasks an inbox of one; n rings of eight slots; Tracked tasks: and_after, hooks, retry_after timers
  2. Subscription streams of the connected brokers → default one task runs every step: poll_next
  3. Subscription streams of the connected brokers → workers(n) one loop task hands them out
  4. Subscription streams of the connected brokers → threads(n) one feeder task fills the rings
  5. workers(n) one loop task hands them out → n worker tasks an inbox of one: send
  6. threads(n) one feeder task fills the rings → n rings of eight slots: push
  7. n rings of eight slots → n OS threads a Tokio runtime on each: pop
  8. default one task runs every step → Tracked tasks: and_after, hooks, retry_after timers: spawn
  9. n worker tasks an inbox of one → Tracked tasks: and_after, hooks, retry_after timers

The service runs on Tokio. #[ruststream::app] builds a multi-thread runtime. flavor = "current_thread" picks a single-threaded one, and worker_threads = n sets the number of threads.

A subscriber is a Stream of deliveries, and back-pressure comes from polling it. The default loop polls for the next delivery once the previous one is settled. workers(n) keeps at most n deliveries in flight. workers(n, by_key) sends every delivery of one key to the same worker, so one key keeps its order. threads(n) reads ahead at most n × 9 deliveries: one in the hands of each thread and eight in its ring.

A dedicated thread runs the whole delivery, from decode to settlement, on its own current-thread runtime. Work the handler hands to MainRuntime, taken as Ctx<MainRuntime>, runs back on the app runtime.

One delivery​

One delivery, step by step, in the column of the task that runs each stepLoop taskHandling taskLater, trackedpoll thestreambuild thecontextconsumelayersdecodehandlerbodypublishthe replysettlepoll thenext onecontinuationsand timers
The diagram as text
  1. Loop task: poll the stream; poll the next one
  2. Handling task: build the context; consume layers; decode; handler body; publish the reply; settle
  3. Later, tracked: continuations and timers
  4. poll the stream → build the context
  5. build the context → consume layers
  6. consume layers → decode
  7. decode → handler body
  8. handler body → publish the reply
  9. publish the reply → settle
  10. settle → poll the next one: next turn
  11. settle → continuations and timers: spawn

The handling task is the loop task itself by default, a worker task under workers(n), and a dedicated thread under threads(n). An and_after continuation, a post-settle hook and a retry_after timer run after the settlement, as tasks the app tracks. They run at most once: the delivery is already settled.

Shutdown​

What a graceful shutdown does, in orderA signal, a fail-fastor RunningApp::shutdownon_shutdown hooks runwhile the brokers are connectedThe loops stoptaking new deliveriesDeliveries in handare handled and settledTracked continuationsand retry timers finishBrokers shut downin reverse registration orderafter_shutdown hooks runwith the brokers closed
The diagram as text
  1. A signal, a fail-fast or RunningApp::shutdown → on_shutdown hooks run while the brokers are connected
  2. on_shutdown hooks run while the brokers are connected → The loops stop taking new deliveries
  3. The loops stop taking new deliveries → Deliveries in hand are handled and settled
  4. Deliveries in hand are handled and settled → Tracked continuations and retry timers finish
  5. Tracked continuations and retry timers finish → Brokers shut down in reverse registration order
  6. Brokers shut down in reverse registration order → after_shutdown hooks run with the brokers closed

A shutdown timeout bounds the wait. When it expires, loops still running are aborted, and tracked tasks still pending are left to finish on their own. Without a timeout, a shutdown waits for all of them. A task spawned with plain tokio::spawn on a dedicated thread is not tracked and ends with its thread.