Skip to main content
Open source · Apache-2.0 · 100% safe Rust

Async messaging for Rust, on any broker

Write #[subscriber] handlers once, run them on Kafka, NATS, RabbitMQ and ten more brokers, and let the compiler catch misuse before it ships.

One handler, fourteen transports

Kafkaruststream-rdkafkaNATSruststream-natsRabbitMQruststream-lapinRedis / Valkeyruststream-fredAMQP 1.0ruststream-amqpGoogle Pub/Subruststream-gcp-pubsubSQS / SNSruststream-sqs-snsPulsarruststream-pulsarMQTT 5ruststream-rumqttcZeroMQruststream-zeromqKinesisruststream-kinesisFiles / stdioruststream-sea-fileSQL databasesruststream-sqlxIn-memoryruststream
The same consumer, twice

Write the handler, not the loop

Both programs read orders from Kafka, decode them and print the id. On the left the client is driven by hand; on the right the runtime drives it.

src/main.rs · rdkafkaBy hand
use rdkafka::Message;
use rdkafka::config::ClientConfig;
use rdkafka::consumer::{CommitMode, Consumer, StreamConsumer};
use serde::Deserialize;

#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let consumer: StreamConsumer = ClientConfig::new()
.set("bootstrap.servers", "localhost:9092")
.set("group.id", "orders-svc")
.set("enable.auto.commit", "false")
.create()?;
consumer.subscribe(&["orders"])?;

loop {
let msg = tokio::select! {
_ = tokio::signal::ctrl_c() => break,
msg = consumer.recv() => match msg {
Ok(msg) => msg,
Err(err) => {
eprintln!("kafka error: {err}");
continue;
}
},
};
match msg.payload().map(serde_json::from_slice::<Order>) {
Some(Ok(order)) => println!("got order {}", order.id),
Some(Err(err)) => eprintln!("bad payload: {err}"),
None => eprintln!("empty message"),
}
if let Err(err) = consumer.commit_message(&msg, CommitMode::Async) {
eprintln!("commit failed: {err}");
}
}
Ok(())
}
  • Client settings and commit mode, by hand
  • Decoding and every error branch, by hand
  • Signals and shutdown, by hand
  • All of it again for the next topic and the next broker
src/main.rs · ruststream-rdkafka
use ruststream_rdkafka::prelude::*;
use serde::Deserialize;

#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}

#[subscriber("orders")]
async fn handle(order: &Order) -> HandlerOutcome {
println!("got order {}", order.id);
HandlerOutcome::ack()
}

#[ruststream::app]
fn app() -> impl App {
RustStream::new(AppInfo::new("orders", "0.1.0")).with_broker(
KafkaBroker::new(["localhost:9092"]).default_group("orders-svc"),
|b| {
b.include(handle);
},
)
}
terminal
$ cargo run -- run
04:24:54.781 INFO ruststream::lifecycle starting service service=orders version=0.1.0 brokers=1 subscribers=1
04:24:54.886 INFO ruststream::lifecycle broker connected broker="ruststream_rdkafka::broker::KafkaBroker"
04:24:57.902 INFO ruststream::dispatch subscriber started subscriber=orders input="kafka_quickstart::Order"
04:24:57.902 INFO ruststream::lifecycle service running subscribers=1
# {"id":1} is published to orders
got order 1
04:25:03.779 INFO ruststream::lifecycle shutdown signal received
04:25:03.886 INFO ruststream::lifecycle service stopped

A real run of the kafka_quickstart example against a local Kafka, with the logging feature on.

  • Decodes each payload with the codec the build selects
  • Settles each message by the outcome the handler returns
  • Stops on Ctrl+C or SIGTERM and closes the broker
  • Prints its AsyncAPI document with asyncapi gen and runs under TestApp with no Kafka
Architecture

The path of a message

A delivery goes from the broker through the subscription and the codec to your handler, and the reply goes back the same way. Each part is a type the compiler checks.

BrokerSubscriptionCodecMiddlewareHandlerReplyPublisherAsyncAPITestApp
HandlerAn async function under #[subscriber]; what it returns settles the delivery or becomes the reply.
  1. BrokerA broker crate wraps one client and is held to the contract that the conformance suite checks; the core depends on no broker.
  2. SubscriptionA subscription names what to consume and is resolved once, at startup, against the connected broker.
  3. CodecThe codec turns the payload into the handler type: JSON, MessagePack or CBOR chosen at compile time, or raw bytes with no codec at all.
  4. MiddlewareConsume and publish layers wrap every delivery: logging, metrics, tracing or your own.
  5. HandlerAn async function under #[subscriber]; what it returns settles the delivery or becomes the reply.
  6. ReplyA reply type derives Outgoing and declares where it goes; the runtime encodes it and publishes it.
  7. PublisherA publish policy paired with the connected broker; a handler also publishes through typed Out slots.
  8. AsyncAPIThe same declarations generate an AsyncAPI 3.1 document, so the document cannot drift from the code.
  9. TestAppTests start the production app with the broker in process and drive the real dispatch, with no server.
Design

What the framework commits to

  • 01

    A core with no broker in it

    The core is traits and types. Each broker is its own crate, held to the contract by the conformance suite.

  • 02

    Broker settings stay with the broker

    Each broker crate owns its Config. A client upgrade changes that crate, not the framework.

  • 03

    Misuse does not compile

    ack(self) consumes the delivery, so a second ack is a compile error. Out-of-order lifecycle calls fail the same way.

  • 04

    Subscribers are streams

    A subscriber is a Stream, so back-pressure comes with it. The runtime builds handlers on top of it.

  • 05

    Generic, with no dyn in the contract

    The contract uses associated types and native async fn in trait. The runtime erases types only where a service needs it.

  • 06

    Capabilities, not a fat interface

    Batches, transactions, request-reply, partitions and seeking are separate traits. A broker implements the ones it supports.

Cost per message

Zero allocations per message

The framework's own code per message in the steady state, counted by the benchmarks on every change. The broker's client is not in these numbers.

  • 1,697instructions0 allocations

    Consume a message and decode its JSON into a struct

  • 862instructions0 allocations

    Consume the raw bytes, no codec

  • 1,785instructions0 allocations

    Consume through four middleware layers

  • 2,051instructions0 allocations

    Encode and publish a reply

Measured on ruststream 0.8.0, 2026-11-01, AMD Ryzen 9 7900X 12-Core Processor. The benchmarks page has every scenario and the method, including the paths that do allocate.

Start with one handler

Scaffold a service with cargo generate in a minute, or ask the documentation a question.

examples/quickstart.rs
//! The landing-page example: a one-handler service with no runtime boilerplate.
//!
//! ```text
//! cargo run --example quickstart --features macros,memory,json,logging -- run
//! ```

use ruststream::diagnostics::TracingLogger;
use ruststream::memory::prelude::*;
use serde::Deserialize;

#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}

#[subscriber("orders")]
async fn handle(order: &Order) -> HandlerOutcome {
tracing::info!(order.id, "got order");
HandlerOutcome::ack()
}

#[ruststream::app]
fn app() -> RustStream {
RustStream::new(AppInfo::new("orders", "0.1.0"))
.logger(TracingLogger)
.with_broker(MemoryBroker::new(), |b| b.include(handle))
}

#[ruststream::app] generates main with all the runtime boilerplate: cargo run -- run starts the service, and cargo run -- asyncapi gen prints its AsyncAPI document.