Skip to main content

Queues

The zeromq client keeps no queue of messages: a send writes its frames into a peer's connection, and the kernel's socket buffers hold what is in flight.

Where a message waits between two zeromq sockets, for PUSH/PULL and for PUB/SUBPUSH/PULLPUB/SUBPUSH sendtakes the peerwhose turn it isWrites the framesinto that peer'sconnectionSocket buffersof the kernel,at both endsPULL recvreads its peers'connections in turnPUB sendevery subscriberwhose filter matchesWrites into eachone's connection,one after anotherSocket buffersof the kernel,at both endsSUB recvreads what itsfilters let throughThe zeromq client keeps no queue of messages:a send ends when its frames are writtenlibzmq keeps 1000 messages per connection instead(SNDHWM, RCVHWM): at the mark PUB drops, PUSH blocks
The diagram as text
  1. PUSH/PULL: PUSH send takes the peer whose turn it is; Writes the frames into that peer's connection; Socket buffers of the kernel, at both ends; PULL recv reads its peers' connections in turn
  2. PUB/SUB: PUB send every subscriber whose filter matches; Writes into each one's connection, one after another; Socket buffers of the kernel, at both ends; SUB recv reads what its filters let through
  3. PUSH send takes the peer whose turn it is → Writes the frames into that peer's connection: in turn
  4. Writes the frames into that peer's connection → Socket buffers of the kernel, at both ends
  5. Socket buffers of the kernel, at both ends → PULL recv reads its peers' connections in turn
  6. PUB send every subscriber whose filter matches → Writes into each one's connection, one after another
  7. Writes into each one's connection, one after another → Socket buffers of the kernel, at both ends
  8. Socket buffers of the kernel, at both ends → SUB recv reads what its filters let through
  9. The zeromq client keeps no queue of messages: a send ends when its frames are written
  10. libzmq keeps 1000 messages per connection instead (SNDHWM, RCVHWM): at the mark PUB drops, PUSH blocks

A PUSH send takes the next peer in turn and writes the frames into that peer's connection. The send ends when the frames are written, so a peer that reads slowly makes the send wait.

A PULL socket reads a message from each connected peer in turn when it is asked for one. Until then the frames stay in the kernel's buffers, on this side and on the sender's.

A PUB send writes the message into the connection of every subscriber whose filter matches, one after another. So the slowest matching subscriber holds the publisher back, and nothing is lost to a slow one. A message that matches no filter is dropped without an error.

libzmq, the C library most other languages use, works differently. Each connection there has a queue of its own, 1000 messages by default (the high-water marks SNDHWM and RCVHWM). When a queue is full, PUB drops the message and PUSH blocks the send. The zeromq client writes a send straight into the connection, and a socket's options are a peer identity and a connect timeout.

What bounds how far a subscription of this crate reads ahead is the crate's own channel, read_ahead deep, and Backpressure draws it.

The libzmq figures are the zguide's ("High-Water Marks"). The client's behaviour is the source of zeromq 0.6.