Skip to main content

deduplication_id

deduplication_id names one message's idempotency key on a FIFO destination; left unnamed, every send gets a process-unique id.

A FIFO queue accepts a message whose deduplication id it has seen in the last five minutes and does not deliver it again. The repeated publish succeeds, so the sender cannot tell the two apart and does not need to. An id derived from the input makes a handler's publish idempotent: SQS delivers a message at least once, and a second run of the handler sends the same id.

The id is a step of one publish, never a setting of the policy. A constant id would collapse everything a position sends into one message. A send that names none carries an id this crate makes unique within the process, rs-<pid>-<sequence>. Two identical payloads sent on purpose therefore stay two messages, even on a queue with content-based deduplication. The step on a standard queue fails the publish with SqsError::NotFifo. A FIFO topic takes the same step through SnsPublish.

Example​

A payment SQS delivers twice leads to one charge: both runs send the charge under the payment's id.

examples/sqs_publishing_deduplication.rs
/// SQS delivers a message at least once, so this handler may run twice for one payment. Both
/// runs send the charge under the payment's id, and the FIFO queue keeps the first.
#[subscriber(SqsQueue::new("payments"))]
async fn bill(
    payment: &Payment,
    Out(charges): Out<impl Publisher<Options = SqsPublishOptions>, Charges>,
) -> HandlerOutcome {
    let charge = Charge {
        payment_id: payment.id,
        amount: payment.amount,
    };
    let sent = charges
        .message(&charge)
        .group_id(&payment.account)
        .deduplication_id(format!("payment-{}", payment.id))
        .publish()
        .await;
    if let Err(err) = sent {
        tracing::warn!(payment.id, %err, "charge not sent");
        return HandlerOutcome::retry();
    }
    HandlerOutcome::ack()
}