Async messaging for Rust, on any broker
Write #[subscriber] handlers once, run them on Kafka, NATS, RabbitMQ and ten more brokers, and let the compiler catch misuse before it ships.
One handler, fourteen transports
Kafkaruststream-rdkafkaNATSruststream-natsRabbitMQruststream-lapinRedis / Valkeyruststream-fredAMQP 1.0ruststream-amqpGoogle Pub/Subruststream-gcp-pubsubSQS / SNSruststream-sqs-snsPulsarruststream-pulsarMQTT 5ruststream-rumqttcZeroMQruststream-zeromqKinesisruststream-kinesisFiles / stdioruststream-sea-fileSQL databasesruststream-sqlxIn-memoryruststreamWrite the handler, not the loop
Both programs read orders from Kafka, decode them and print the id. On the left the client is driven by hand; on the right the runtime drives it.
use rdkafka::Message;
use rdkafka::config::ClientConfig;
use rdkafka::consumer::{CommitMode, Consumer, StreamConsumer};
use serde::Deserialize;
#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let consumer: StreamConsumer = ClientConfig::new()
.set("bootstrap.servers", "localhost:9092")
.set("group.id", "orders-svc")
.set("enable.auto.commit", "false")
.create()?;
consumer.subscribe(&["orders"])?;
loop {
let msg = tokio::select! {
_ = tokio::signal::ctrl_c() => break,
msg = consumer.recv() => match msg {
Ok(msg) => msg,
Err(err) => {
eprintln!("kafka error: {err}");
continue;
}
},
};
match msg.payload().map(serde_json::from_slice::<Order>) {
Some(Ok(order)) => println!("got order {}", order.id),
Some(Err(err)) => eprintln!("bad payload: {err}"),
None => eprintln!("empty message"),
}
if let Err(err) = consumer.commit_message(&msg, CommitMode::Async) {
eprintln!("commit failed: {err}");
}
}
Ok(())
}
- Client settings and commit mode, by hand
- Decoding and every error branch, by hand
- Signals and shutdown, by hand
- All of it again for the next topic and the next broker
use ruststream_rdkafka::prelude::*;
use serde::Deserialize;
#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}
#[subscriber("orders")]
async fn handle(order: &Order) -> HandlerOutcome {
println!("got order {}", order.id);
HandlerOutcome::ack()
}
#[ruststream::app]
fn app() -> impl App {
RustStream::new(AppInfo::new("orders", "0.1.0")).with_broker(
KafkaBroker::new(["localhost:9092"]).default_group("orders-svc"),
|b| {
b.include(handle);
},
)
}
$ cargo run -- run 04:24:54.781 INFO ruststream::lifecycle starting service service=orders version=0.1.0 brokers=1 subscribers=1 04:24:54.886 INFO ruststream::lifecycle broker connected broker="ruststream_rdkafka::broker::KafkaBroker" 04:24:57.902 INFO ruststream::dispatch subscriber started subscriber=orders input="kafka_quickstart::Order" 04:24:57.902 INFO ruststream::lifecycle service running subscribers=1 # {"id":1} is published to orders got order 1 04:25:03.779 INFO ruststream::lifecycle shutdown signal received 04:25:03.886 INFO ruststream::lifecycle service stopped
A real run of the kafka_quickstart example against a local Kafka, with the logging feature on.
- Decodes each payload with the codec the build selects
- Settles each message by the outcome the handler returns
- Stops on Ctrl+C or SIGTERM and closes the broker
- Prints its AsyncAPI document with
asyncapi genand runs underTestAppwith no Kafka
The path of a message
A delivery goes from the broker through the subscription and the codec to your handler, and the reply goes back the same way. Each part is a type the compiler checks.
- BrokerA broker crate wraps one client and is held to the contract that the conformance suite checks; the core depends on no broker.
- SubscriptionA subscription names what to consume and is resolved once, at startup, against the connected broker.
- CodecThe codec turns the payload into the handler type: JSON, MessagePack or CBOR chosen at compile time, or raw bytes with no codec at all.
- MiddlewareConsume and publish layers wrap every delivery: logging, metrics, tracing or your own.
- HandlerAn async function under #[subscriber]; what it returns settles the delivery or becomes the reply.
- ReplyA reply type derives Outgoing and declares where it goes; the runtime encodes it and publishes it.
- PublisherA publish policy paired with the connected broker; a handler also publishes through typed Out slots.
- AsyncAPIThe same declarations generate an AsyncAPI 3.1 document, so the document cannot drift from the code.
- TestAppTests start the production app with the broker in process and drive the real dispatch, with no server.
What the framework commits to
- 01
A core with no broker in it
The core is traits and types. Each broker is its own crate, held to the contract by the
conformancesuite. - 02
Broker settings stay with the broker
Each broker crate owns its
Config. A client upgrade changes that crate, not the framework. - 03
Misuse does not compile
ack(self)consumes the delivery, so a second ack is a compile error. Out-of-order lifecycle calls fail the same way. - 04
Subscribers are streams
A subscriber is a
Stream, so back-pressure comes with it. The runtime builds handlers on top of it. - 05
Generic, with no dyn in the contract
The contract uses associated types and native
async fn in trait. The runtime erases types only where a service needs it. - 06
Capabilities, not a fat interface
Batches, transactions, request-reply, partitions and seeking are separate traits. A broker implements the ones it supports.
Zero allocations per message
The framework's own code per message in the steady state, counted by the benchmarks on every change. The broker's client is not in these numbers.
- 1,697instructions0 allocations
Consume a message and decode its JSON into a struct
- 862instructions0 allocations
Consume the raw bytes, no codec
- 1,785instructions0 allocations
Consume through four middleware layers
- 2,051instructions0 allocations
Encode and publish a reply
Measured on ruststream 0.8.0, 2026-11-01, AMD Ryzen 9 7900X 12-Core Processor. The benchmarks page has every scenario and the method, including the paths that do allocate.
Start with one handler
Scaffold a service with cargo generate in a minute, or ask the documentation a question.
- Macros
- Manual
//! The landing-page example: a one-handler service with no runtime boilerplate.
//!
//! ```text
//! cargo run --example quickstart --features macros,memory,json,logging -- run
//! ```
use ruststream::diagnostics::TracingLogger;
use ruststream::memory::prelude::*;
use serde::Deserialize;
#[derive(Debug, Deserialize)]
struct Order {
id: u64,
}
#[subscriber("orders")]
async fn handle(order: &Order) -> HandlerOutcome {
tracing::info!(order.id, "got order");
HandlerOutcome::ack()
}
#[ruststream::app]
fn app() -> RustStream {
RustStream::new(AppInfo::new("orders", "0.1.0"))
.logger(TracingLogger)
.with_broker(MemoryBroker::new(), |b| b.include(handle))
}
//! The landing-page example written without the `macros` feature: the handler is a named type
//! with an `impl Handle`, mounted with the `subscriber` constructor, and `main` is the one
//! `#[ruststream::app]` would generate.
//!
//! ```text
//! cargo run --example manual_quickstart --no-default-features --features memory,json,logging -- run
//! ```
use ruststream::diagnostics::TracingLogger;
use std::future::ready;
use std::process::ExitCode;
use ruststream::memory::prelude::*;
use ruststream::runtime::cli::run_main;
use serde::Deserialize;
#[derive(Debug, Deserialize, schemars::JsonSchema)]
struct Order {
id: u64,
}
/// The handler: `#[subscriber("orders")]` generates this type and this impl. Every axis of the
/// form - the reply, the injections, the broker context, the application state - is a defaulted
/// parameter of `Handle`, so a plain body names none of them.
struct Receive;
impl Handle<Order> for Receive {
fn handle(
&self,
order: &Order,
_outs: &(),
_ctx: &mut Context<'_>,
) -> impl Future<Output = Result<(), HandlerOutcome>> {
tracing::info!(order.id, "got order");
ready(Ok(()))
}
}
fn app() -> RustStream {
RustStream::new(AppInfo::new("orders", "0.1.0"))
.logger(TracingLogger)
.with_broker(MemoryBroker::new(), |b| {
b.include(subscriber("orders", Receive).build())
})
}
fn main() -> ExitCode {
run_main(app)
}
#[ruststream::app] generates main with all the runtime boilerplate: cargo run -- run starts the service, and cargo run -- asyncapi gen prints its AsyncAPI document.
Installation
Features and crate setup.
Quick start
Scaffold a service with
cargo generate.Tutorial
Build a service step by step.
Brokers
The in-memory broker and the broker crates.
Broker authors
Implement the contract and pass conformance.
Testing
Test handlers in process, with no server.
HTTP frameworks
Run beside axum, with a transactional outbox.
AsyncAPI
The document a service generates from its own declarations.
API reference
Every type, on docs.rs.
Benchmarks
The cost of the code, and each broker against its raw client.