Skip to main content

Queues

A reliable list moves each entry to a processing list before the handler runs, and removes it from there when the handler settles.

A reliable list: a claim moves the entry to the processing list, the outcome settles it there, and the recovery set returns orphansProducerjobsjobs.processingRecovery setsorted by claim timeHandlerack:LREMretry: LPUSHback, then LREMdrop:LREMEach read first sweeps claims idle past min_idle back to the queue
The diagram as text
  1. Producer → jobs: LPUSH
  2. jobs → jobs.processing: BLMOVE
  3. jobs.processing → Handler: delivered
  4. jobs.processing → Recovery set sorted by claim time: ZADD
  5. Recovery set sorted by claim time → jobs: orphans back
  6. Handler → ack: LREM
  7. Handler → retry: LPUSH back, then LREM
  8. Handler → drop: LREM
  9. Each read first sweeps claims idle past min_idle back to the queue

A list is a queue: LPUSH adds at one end, a pop takes from the other, and the oldest entry leaves first. A popped entry is gone from Redis. A simple RedisList pops with BRPOP, so an entry whose handler fails or whose process dies is lost: delivery is at most once.

A reliable list claims instead of popping. BLMOVE moves the entry from the queue to a processing list in one command, and the entry stays there while the handler runs. The processing list is <key>.processing unless the subscription names another. The outcome settles the entry on the processing list: LREM removes it, and a retry first pushes it back to the queue. Delivery is at least once. Redis describes the pattern under LMOVE.

A list keeps no record of who took an entry or when, so a consumer that dies after its claim leaves the entry on the processing list. recovery_zset keeps that record. Every claim is added to a sorted set, scored by its time. Each read first sweeps the claims older than min_idle and moves each of their entries back to the queue with LREM and LPUSH, at most 128 per read.

On a cluster both lists have to hash to one slot, because BLMOVE touches both keys. A subscription whose two keys land on different slots refuses to start. A hash tag such as {jobs} puts both on one slot, and the default processing key follows it.