Skip to main content

Writer

One writer task per stream file frames each message and places the beacons; the crate flushes after every publish and the file is synced to the device at shutdown.

How a publish reaches a stream file, and when the file is flushed and syncedFilePublisherenvelope, then send_toThe client's producerqueues the messageWriter taskone per file, per processMessage sinkframes it, places beaconsThe stream fileflush() afterevery publish:to the OSsync_all() once,at shutdown:to the deviceThe client stops a file at 16 GiB:that publish and every later one fail
The diagram as text
  1. FilePublisher envelope, then send_to → The client's producer queues the message
  2. The client's producer queues the message → Writer task one per file, per process
  3. Writer task one per file, per process → Message sink frames it, places beacons
  4. Message sink frames it, places beacons → The stream file
  5. flush() after every publish: to the OS → Message sink frames it, places beacons
  6. sync_all() once, at shutdown: to the device → Message sink frames it, places beacons
  7. The client stops a file at 16 GiB: that publish and every later one fail

The FilePublisher hands the stream key and the payload to the client's producer. The producer queues it for the file's writer task, which numbers the message within its key, frames it and inserts a beacon wherever the next interval falls.

The bytes collect in a buffer until something asks for them. The crate asks after every publish: publish returns once a flush has handed the message to the operating system. A live reader of the same file, in this process or another one, sees the message from then on.

The file is synced to the device once, when the producer ends. The crate ends it at shutdown, after the end-of-stream mark when one is set. A message published since the last sync survives a crash of the process, not a power loss.

Every producer of one file in a process shares one writer task, which counts the producer handles. Opening a producer on a file that already holds messages scans back to the last beacon and forward to the end, to learn where each key's numbering stands. That scan grows with the file. This crate hands the writer one append at a time per file, so a failed append leaves no other publish waiting.

The client stops a file at 16 GiB unless told otherwise, and this crate leaves the limit at that default. A publish that reaches the cap fails, and every later publish to that file returns that error at once.