Skip to main content

Path

The way of one Pub/Sub message through a RustStream service: the streaming pull, the delivery, the handler, the reply and the ack.

The path of one Pub/Sub message that is answered, from the streaming pull to its ackStreaming pullthe client leases the messagePump taskforwards into a channel of 16PubSubMessageattributes become headers, uncopiedDispatchone loop, or workersDecodefrom the message's dataHandlerasync fn under #[subscriber]Replyencoded into a buffer it hands overPubSubPublisherawaits the message idSettlementack, nack or a held retryAcksent with the client's next batch
The diagram as text
  1. Streaming pull the client leases the message → Pump task forwards into a channel of 16
  2. Pump task forwards into a channel of 16 → PubSubMessage attributes become headers, uncopied
  3. PubSubMessage attributes become headers, uncopied → Dispatch one loop, or workers
  4. Dispatch one loop, or workers → Decode from the message's data
  5. Decode from the message's data → Handler async fn under #[subscriber]
  6. Handler async fn under #[subscriber] → Reply encoded into a buffer it hands over
  7. Reply encoded into a buffer it hands over → PubSubPublisher awaits the message id
  8. PubSubPublisher awaits the message id → Settlement ack, nack or a held retry
  9. Settlement ack, nack or a held retry → Ack sent with the client's next batch

The client takes the message off the stream and starts its lease. The pump task wraps it in a PubSubMessage without copying: the data stays the client's buffer, and the attributes move into the headers. The ordering key becomes the partition-key header, and the attempt count pubsub-delivery-attempt where the subscription has a dead-letter policy.

The handler's reply is encoded into a buffer that becomes the outgoing message's data as it is, because the client keeps the bytes it publishes. The headers become attributes; a header that is not UTF-8 fails the publish, because an attribute is text. A key named for the publish becomes the message's ordering key, as Ordered delivery draws.

The publish returns once the service has assigned the message an id. A reply that fails to publish settles the delivery with a retry, so the message comes back.

The delivery settles after the reply. An ack leaves with the client's next batch of acknowledgements, within about 100 milliseconds. What each outcome becomes is on Settlement.