Skip to main content

Settlement

What ack, drop, retry and retry_after become on Kinesis, which keeps one checkpoint per shard instead of acknowledging records.

The watermark on one shard: the checkpoint stops before the first delivery that has not settledOne shard, records in the order they arrived1settled2settled3running4settled5settled6runningCheckpoint: record 2the last record beforethe first running oneLeaseStore::checkpointif the lease is heldWhen 3 settles,the checkpoint movesto 5: 6 is stillrunning
The diagram as text
  1. Checkpoint: record 2 the last record before the first running one → 2 settled: names
  2. Checkpoint: record 2 the last record before the first running one → LeaseStore ::checkpoint if the lease is held: writes
  3. One shard, records in the order they arrived
  4. 1 settled
  5. 3 running
  6. 4 settled
  7. 5 settled
  8. 6 running
  9. When 3 settles, the checkpoint moves to 5: 6 is still running

A checkpoint names one record of a shard and says that every record before it is handled. So the crate tracks the deliveries of each shard in the order they arrived. A settlement marks its delivery handled, and the checkpoint moves to the last record of the run of handled deliveries that starts at the oldest one. Acks from parallel lanes can arrive in any order, and the checkpoint never passes a record that is still running. A settlement writes to the lease store only when it moves the checkpoint, and the store takes the write only while this instance holds the shard.

OutcomeEffect on the shard
ackHandled: the checkpoint moves when no earlier record is still running
dropHandled, as for ack: the checkpoint passes a poison record
retry()Refused (AckError::Unsupported) and logged: the record stays unhandled and holds the checkpoint, and the shard is read again from it when its lease is next taken
retry() under max_attempts or a dead-letter streamA counted copy goes to the stream, and the original is handled
retry_after(delay)A counted copy goes to the stream once the delay is over, and the original is handled
The cap is spentThe record goes to the dead-letter stream, or without one it is handled and the checkpoint passes it
No settlement before shutdownThe checkpoint stays where it is, and the shard's next owner reads the record again

A shard is read in order, and Kinesis cannot hand one record back, so a requeue is refused. The reader goes on, and the record comes back after a restart or a lease handover, together with every record after it. Delivery is at least once.

A seek starts the watermark of the shards it moves afresh. A settlement of a delivery read before the seek changes nothing, so a late ack never moves a checkpoint back to where the seek came from. A closed shard is marked SHARD_END only after every one of its deliveries has settled, and its children start after that.

Retries​

A retry on Kinesis: the attempt cap decides between a copy to the stream, the dead-letter stream and a checkpoint past the recordA recordx-ruststream-retry-count says which attempt it isThe handler returnsretry_after(delay), or retry() under a declarationmax_attempts: is this attempt below the cap?Below the cap:the originalsettles, a copygoes to the streamAt the cap:the record goesto the dead-letter streamAt the cap,no dead letter:the checkpointpasses itThe copy is a new record at the tip,on the shard its key picksThe delayed copy is held in the process:a crash before the timer fires loses it
The diagram as text
  1. A record x-ruststream-retry-count says which attempt it is → The handler returns retry_after(delay), or retry() under a declaration: handled
  2. The handler returns retry_after(delay), or retry() under a declaration → max_attempts: is this attempt below the cap?: a retry
  3. max_attempts: is this attempt below the cap? → Below the cap: the original settles, a copy goes to the stream: below
  4. max_attempts: is this attempt below the cap? → At the cap: the record goes to the dead- letter stream: at the cap
  5. max_attempts: is this attempt below the cap? → At the cap, no dead letter: the checkpoint passes it
  6. Below the cap: the original settles, a copy goes to the stream → The copy is a new record at the tip, on the shard its key picks
  7. The delayed copy is held in the process: a crash before the timer fires loses it

Kinesis holds no record back and counts no deliveries, so the runtime does both. A retry_after outcome, and a retry() on a registration that declares max_attempts or a dead_letter stream, take this path. The attempt count travels in the copy's x-ruststream-retry-count header, and the first delivery counts as one.

The copy goes to the stream the subscription reads, the address the KinesisStream descriptor reports, so a registration names no destination. It lands at the tip, on the shard its partition key picks, behind the records written since. So a retried record loses its order against its neighbours. Where that order matters, declare no cap and let retry() hold the shard. Every consumer of the stream receives the copy, not only the one that retried.

A spent record goes to the dead-letter stream with its payload and headers. A cap without a dead-letter stream retires the record the way drop does.