Security & Messaging

doublethink, told in commits

I already wrote about why doublethink exists. This is the other half: how it actually got built, read straight off the git history, dead ends and all. At the end there is a live demo you can run in your own browser.

Ramazan Yavuz
Ramazan Yavuz ·
doublethink, told in commits

A project's git log tends to be more honest than its README. The README is what you want people to believe about the finished thing; the commit history is what actually happened, in order, including the parts you would rather skip over. doublethink is a small publish/subscribe message broker I built, as easy to stand up as ntfy but with genuinely private channels the broker itself cannot read. I have already written about why it exists. This one is the making-of, walked through commit by commit, wrong turns left in. Every short hash below links to the actual commit on GitHub, so you can read the diff for yourself.

It began with a document, not code. The very first commit initialises the repository with specs, the prior-art research, and the first milestone's design. I do this on purpose: before writing a line, I wrote down what the thing was for and then checked whether it needed to exist at all. The research asked one question. Is there already a broker that is both ntfy-easy and genuinely private? After looking at ntfy's own access control, at MQTT brokers, at NATS, and at the managed services, the answer was no: the two properties trade off against each other everywhere I looked. That is the reason the next thirty commits got written.

Those five commits are the first milestone: a working core. A message envelope with a fixed shape, the broker that fans messages out, the crypto that runs in the client, the network transport, and a command-line tool to drive it. By the end of them doublethink could carry a message. It just could not yet be trusted by anyone but me, and the way it asked to be trusted turned out to be wrong.

The dead end is right there in the log. One commit reads Keep keypair auth (deviate from mock contract); add MITM-resistant pairing. The idea was the heavyweight one: every party carries a cryptographic device keypair, the broker authenticates connections with a signed challenge, and pairing two parties means a one-time invite code plus a short string both humans read out loud to rule out a man-in-the-middle. On paper that is the strong design. I built the whole thing, and then I tried to use it. Every private channel now needed a multi-step setup, and the one quality that made ntfy worth copying, that you can point a tool at it in minutes, was gone. I had built a secure broker that was no longer easy, which meant I had rebuilt something that already existed.

So the next commit deletes it. Redesign to ntfy-easy shared-secret channels (drop admin/keypair/SAS pairing). The replacement is almost embarrassingly simpler, and the simplicity is the feature. A private channel is gated by one high-entropy shared secret. You create a channel with a single request and get the secret back; you hand it to the other party over any trusted path you already have; whoever holds the secret can join, nobody else can. From that one secret each side derives two independent keys with different labels: one the broker stores to check you are allowed in, the other, which the broker can never compute, to encrypt the actual messages. The broker knows who may join and faithfully relays ciphertext it cannot read. That deletion is my favorite commit in the project. The strong-looking design was the wrong one, and the right one was smaller.

Then comes a small cluster of commits that record me changing my mind about packaging. It first arrives as a Debian .deb with a systemd unit. A few commits later that is torn out in favor of Docker and a single native binary, and the .deb and the unit are deleted. The log keeps both the first decision and the correction rather than a clean line pretending I knew all along. A GitHub Pages site and a continuous-integration workflow go up around the same time, which is roughly when a personal experiment starts behaving like a project other people might use.

The second milestone is the unglamorous one, and it is the one that turns a toy into something you can put on the open internet. The moment a broker is reachable by anyone, "anyone can create a channel" stops being a charming ntfy trait and becomes an open relay you have handed to strangers. One dense commit, M2: accounts, SQLite retention, TTL aging, quotas, abuse control, admin key, adds all the machinery a public instance actually needs: lightweight accounts with an API key, opt-in message retention so a peer that was offline can reconnect and catch up, a time-to-live that ages old messages out, per-channel and per-account storage quotas so nobody can fill the disk, rate limits, and an operator key that lives only on the server. None of that is exciting. All of it is the difference between a demo and a service.

The two commits after it are why this article can end the way it does. One verifies that the crypto running in a browser produces exactly the same bytes as the crypto running in Go, parity-checked rather than assumed, and opens the cross-origin access a browser needs. The next adds a live, in-browser demo to the project page, so the privacy claim stops being a sentence you have to trust and becomes a thing you can watch happen.

The third milestone came from a use case the design did not yet handle: letting someone create a topic that never expires, for something like a long-lived feed. Two problems fall out of that. A topic that never expires uses storage indefinitely, so you cannot let anyone on the internet allocate permanent space on your disk; it needs the operator's say-so. But the operator must never learn the channel's secret, or the privacy promise is broken. The answer, in the M3 commits, is a grant ticket: the operator issues a single-use, time-limited ticket that authorizes one permanent, capped channel, and the user redeems it with their own secret. The operator authorizes the channel without ever seeing the secret. There is a follow-on case too: if the topic is meant to be public, paying for end-to-end encryption buys nothing, so the grant became encryption-agnostic and the choice moved to whoever owns the data, made at redeem time. Somewhere in here the storage engine was swapped from SQLite to Redis, because the load is high-throughput rather than high-volume, and I wrote down exactly how much a hard crash can lose, about a second, because "permanent" should mean the data does not age out, not a durability promise the disk cannot keep.

That second-to-last commit, the one that stops publishing internal planning docs, is a small lesson about releasing something: the repository you develop in and the repository you publish are not the same, and choosing what to leave out is part of the work. The moment in this history I think about most is not a feature at all. It is a secret scan that caught me about to commit a real admin key into a public repo as a test fixture. The core idea in doublethink is short. Most of the effort that makes it trustworthy went into the unexciting parts around it.

So, the demo. The project page carries one live demo that runs entirely in your browser, talking to a real instance on my own server. It does the encrypted round-trip in front of you: your browser generates a secret, creates a throwaway channel, sends one message, and shows you both the exact ciphertext the broker relays and the plaintext only the secret-holder recovers. Then it tries the same bytes with a wrong secret and recovers nothing. The channel is ephemeral and torn down at once, so nothing is stored. Below the demo, the page has copy-paste commands to run the broker yourself and to publish and subscribe from the terminal with plain curl. That is the whole project page: a demo you can watch and commands you can run, nothing else.

See it run: the in-browser demo and the run-it commands are on the doublethink project page. The full source, every commit linked above included, is on GitHub.

That is the project as it was built: a research note before any code, a stronger-looking design dropped for a simpler one, a packaging decision reversed, the unglamorous milestone that made it safe to deploy, a capability added for permanent topics, and a secret scan that stopped a real mistake. doublethink is open source, runs in Docker or as a single binary, and carries a plain no-warranty notice, because a tool whose job is holding other people's private traffic should be honest that it is the work of one person, reviewed but not infallible. The commit history is the most accurate account of how it got here, which is why I wrote this from it.