Ordering from the key, not a partition count: why we built Varisai

Every team we've worked with has written the same four pieces of code around their message broker. Varisai is our attempt to stop writing them.

The glue nobody wants to own

You pick Kafka or RabbitMQ. A few months later, you also own:

A retry loop with hand-rolled backoff.
A dead-letter table in Postgres that nobody looks at until something breaks.
A cron job that wakes up every minute to release delayed work.
A trace store, so someone can answer "why wasn't this order processed?"

None of this is your product, and all of it is on call at 2 a.m. It still doesn't fix the deeper problem: one bad message from one customer can stall a whole partition, and everyone else on that partition waits behind it.

We got tired of rebuilding this glue, so we built a broker where those four things are features you switch on rather than code you maintain.

What Varisai is

Varisai is a self-hosted message broker: a durable log that also tracks the delivery of every single message. The name is Tamil — வரிசை (varisai) means a queue, a row, an order. That's the whole idea.

It runs on Node.js and MongoDB, starts with one docker compose up, and the source is public under the Elastic License 2.0. Free to use, self-host and modify.

Defaults

Ordering. Strict FIFO per stream, key and subscription.
Delivery. At-least-once, with a 30 s lease per attempt.
Retries. 5 attempts, exponential backoff with jitter, from 10 s up to 1 h.
Retention. 7 days per stream, 30 days for dead-lettered messages.
Limits. 1 MB per message, 500 in-flight deliveries per subscription (configurable).
Consume API. HTTP long-poll, up to 25 s and up to 100 messages per call.
Durability. MongoDB replica set, majority write concern.

The key is the whole model

In Kafka, you decide your parallelism on day one when you pick a partition count. Guess too low and you're stuck; guess too high and you pay for it forever.

Varisai removes that decision. Every message carries a key, and the key decides everything.

Same key. Messages arrive strictly in publish order, one at a time per subscription. If one fails, its lane pauses and everything behind it waits — exactly what you want for, say, one customer's bank transactions.
Different keys. Fully independent. No shared cursor, no shared failure. Your parallelism is simply the number of active keys, so a million keys and a thousand consumers works like one key, just bigger.

In practice, publishing looks like this:

curl -X POST https://varisai.dev/v1/streams/orders/messages -H "Authorization: Bearer vk_yourkey:vs_yoursecret" -d '{"key":"tenant_42","payload":{"order_id":8891}}'

Set key = tenant_id and the noisy-neighbour problem goes away. One tenant's poison message can only ever block that tenant.

Two real problems, one primitive

Ticket booking: no overselling

Five buyers, four tickets. Use key = event_id so every booking for that event goes through one lane, one buyer at a time. Overselling becomes impossible by construction — no distributed locks, no race conditions.

Now say buyer P1's payment keeps failing. The retry policy backs off while P2 to P5 hold their exact places in line. After the maximum attempts, P1 moves to the dead-letter queue, the lane resumes, and the four tickets sell in arrival order.

Ride matching: no double-booking

Ten riders, five drivers, two keys. key = zone queues riders first-come, first-served. key = driver_id guarantees a driver never holds two ride offers at once.

When all five drivers are busy, rider C6 waits at the head of the lane — retrying, but never losing their spot. The moment a driver frees up, C6 is matched before C7. Fairness isn't a feature you wrote; the lane enforces it. This example ships in the repo under examples/ride-matching.

Declared, not coded

Remember the four pieces of glue? In Varisai, each one is a setting on a subscription. Your consumer just does its job:

varisai.consume("billing", async (message) => { await chargeCustomer(message.payload) }) — throw to retry, resolve to ack.

Retries and DLQ. Max attempts, exponential backoff with jitter, per-attempt leases. Failures land in a browsable dead-letter queue, grouped by error, with one-click redrive.
Scheduled delivery. Publish with deliver_at and the message stays invisible until then. It survives restarts and can be cancelled until it fires. No more cron jobs for delayed work.
Message tracing. Paste a message ID and see every attempt, consumer, duration and error. If it's blocked, Varisai tells you which message is blocking it.
Replay one tenant. Acking never deletes. Rewind a subscription to an offset or timestamp — for a single key if you like — without touching anyone else.
An operational UI. Not just graphs. Skip a poison message, pause a lane or redrive a failure group from the browser.

How it compares

Kafka, SQS FIFO and RabbitMQ are all good systems. They optimise for different things. Two questions separate them: where does ordering come from, and what happens when a message fails?

Ordering unit. Varisai: the message key. Kafka: the partition, sized up front. SQS FIFO: the message group ID. RabbitMQ: the queue.
A failing message blocks. Varisai: only its own key. Kafka: its whole partition. SQS FIFO: its whole message group. RabbitMQ: the queue, or the message goes to the back.
Retries and DLQ. Varisai: declared, browsable, one-call redrive. Kafka: build it yourself. SQS FIFO: redrive policy, no backoff shaping. RabbitMQ: TTL plus dead-letter exchange plumbing.
Scheduled delivery. Varisai: any future time, cancellable. Kafka: not built in. SQS FIFO: up to a 15-minute delay. RabbitMQ: plugin or TTL tricks.
Replay after ack. Varisai: by offset or time, even for one key. Kafka: by offset, whole partition. SQS FIFO and RabbitMQ: not possible.

When not to use it

If you're streaming hundreds of MB per second of telemetry to many readers, use Kafka — that's its home ground. If you're all-in on AWS and a 15-minute delay cap never hurts, SQS FIFO is fine. If you need complex routing with exchanges and bindings, RabbitMQ does it better.

Varisai tracks state for every message. That bookkeeping is the product, and it has a cost.

The sweet spot is the per-entity work queue — orders, bookings, tenant jobs, payouts — that often sits right beside a Kafka firehose.

What we promise, and what we don't

We'd rather you trust our limits than be surprised by them.

Guaranteed

Durable publishes, acknowledged with MongoDB majority writes.
Strict FIFO per stream, key and subscription.
At-least-once delivery until a message is acked, dead-lettered or expired.
Retention independent of acking, with replay by offset, time or key.
Scheduled delivery that survives restarts.

Not guaranteed

Exactly-once. Your consumer must handle a repeat. An idempotency_key on publish narrows the window but can't close it.
Ordering across different keys.
Transactions across streams.
Exact timing. Scheduled delivery means "at or after", not "exactly at".

No broker honestly delivers exactly-once side effects. We'd rather say so up front.

The 5-minute guide walks through each call, and the full comparison goes deeper on when Varisai fits.

If you've ever written a retry loop, a dead-letter table and a delay cron for the same service, give Varisai an afternoon. We'd love to hear what breaks.

Get started at varisai.dev

Varisai is built by Higglers Lab and listed on Spotrove at its product page.