Skip to main content

Path

The way of one Pulsar message through a RustStream service: the consumer, the driver task, the handler, the reply and the acknowledgement.

The path of one Pulsar message that is answered, from the client consumer to its acknowledgementThe client consumermessages pushed under flow permitsPulsarMessage, built by the driver taskthe payload copied into BytesDispatchone loop, or a lane per keyDecoderead from the copied payloadHandlerasync fn under #[subscriber]Replyencoded into a buffer the client takesPulsarPublisherthe topic's producer, receipt awaitedSettlementsent to the driver taskAcknowledgementwritten to the connection, no answer
The diagram as text
  1. The client consumer messages pushed under flow permits → PulsarMessage, built by the driver task the payload copied into Bytes
  2. PulsarMessage, built by the driver task the payload copied into Bytes → Dispatch one loop, or a lane per key
  3. Dispatch one loop, or a lane per key → Decode read from the copied payload
  4. Decode read from the copied payload → Handler async fn under #[subscriber]
  5. Handler async fn under #[subscriber] → Reply encoded into a buffer the client takes
  6. Reply encoded into a buffer the client takes → PulsarPublisher the topic's producer, receipt awaited
  7. PulsarPublisher the topic's producer, receipt awaited → Settlement sent to the driver task
  8. Settlement sent to the driver task → Acknowledgement written to the connection, no answer

The client consumer holds the messages the broker pushed ahead, under its flow permits. The subscription's driver task takes the next one and builds a PulsarMessage: it copies the payload into a Bytes, keeps the topic name and the message id, which a handler reads through its context, and turns the message properties into headers. A message's partition key becomes the partition-key header, which the lanes read. The delivery waits in a channel of 16 until the dispatch loop takes it.

The handler's reply is encoded into a buffer, and the publisher hands that buffer to the client without a copy. A payload over max_message_size fails before it is sent: the client does not check the size, and the server drops the whole connection on an oversized message. The publish returns once the broker has stored the message and sent its receipt. A reply that fails to publish settles the delivery with a retry.

The delivery settles after the reply. The settlement goes back to the driver task, which hands it to the client, and the call returns once the client has queued it. The client writes the acknowledgement to the connection and waits for no answer.

Benchmarks counts what this path costs per message, and how much of it is the client's.