Path

The way of one Kafka record through a RustStream service: the fetch queue, the delivery, the handler, the reply and the commit.

The diagram as text

1. librdkafka fetch queue records fetched ahead → KafkaMessage the record stays in the fetch buffer
2. KafkaMessage the record stays in the fetch buffer → Dispatch one loop, or a lane per key
3. Dispatch one loop, or a lane per key → Decode read in place from the buffer
4. Decode read in place from the buffer → Handler async fn under #[subscriber]
5. Handler async fn under #[subscriber] → Reply encoded into the loop's buffer
6. Reply encoded into the loop's buffer → KafkaPublisher awaits the delivery report
7. KafkaPublisher awaits the delivery report → Settlement the Commit mode decides
8. Settlement the Commit mode decides → Commit of the stored offset

A record librdkafka has already fetched is taken without waiting. The KafkaMessage holds it where librdkafka put it, together with a reference count on the consumer that owns that buffer. The body, the key and the headers are read from the fetch buffer and not copied, and the headers are converted only when something asks for them. The consumer stays open for as long as a delivery holds a record of it.

The one record that is copied is the one a waiting stream resolves itself: when the queue was empty, the record the wait returns is copied out.

The handler's reply is encoded into a buffer the dispatch loop reuses. The publisher lends those bytes to librdkafka, which copies them into its own queue, and the publish returns once the cluster has acknowledged the record. A reply that fails to publish settles the delivery with a retry.

The delivery settles after the reply. What that does to the committed offset depends on the subscription's Commit mode, which Settlement draws.