The plan

The work goes from the top down, in three levels. The whole client is split into milestones, each ending in something that runs end to end: first a client that can only read, then one that can write, then accounts, threads, and so on. When a milestone starts, Claude splits it into steps, and when a step starts, Claude splits it into cards. Detail is added only one level ahead: only the milestone we are working on is split into steps, and only the current step is split into cards. This is called rolling wave planning. Every step ends with a review, and the review may change the plan for the next steps, so there is no detailed plan for work that the earlier work can still change. But there is always a more or less clear picture of what lies ahead and what is still left to do. The diagram below shows the whole cycle.

flowchart TD
    M["Milestone<br/>(e.g. a client that can write)"] -->|Claude splits the current milestone| S[Steps]
    S -->|Claude splits the current step| C["Cards: signatures,<br/>frozen tests, task"]
    C --> L[["Card loop<br/>(Bonsai writes, checks, Jev decides)"]]
    L --> R[Step review by Claude]
    R -->|next step| S
    R -.->|may change the plan| M

The unit of work is a card: one small task for the executor (normally Bonsai), usually a few functions in one file. Before a card goes anywhere, Claude has already written everything around it: the types and function signatures with empty bodies, the module’s description, and the tests. The tests are frozen. Bonsai sees the task, the files it may change and the tests, and fills in the bodies. If a test file changes by a single byte, the attempt does not count, so the model cannot pass a test by rewriting it. For a model working on its own this is the most important guard.

The cards are run by an orchestrator, a separate program that Claude wrote. For each card it asks Bonsai for code and then checks the result, cheapest checks first:

1. Did the model touch only the files it was allowed to touch?
2. Does it compile, and does clippy have nothing to say?
3. Does the code break any of the project’s mechanical rules, such as which crate may depend on which?
4. Do the tests pass?
5. Only then Jev: questions that a machine cannot check, like “does the code do what the task describes” or “does every function belong to the module’s stated responsibility”.

The first four checks cost nothing and are exact, so Jev only sees code that already works. When a check fails, the errors go back to Bonsai for another attempt. Jev also looks at the failure and decides whether it is worth retrying (a typo, a wrong method name) or whether the card should go back to Claude (the task itself looks wrong, or the model is going in circles). After three failed attempts the card goes back to Claude anyway. The second diagram shows one card’s way through this loop.

flowchart TD
    B[Bonsai writes the code] --> G1{"Only the allowed<br/>files changed?"}
    G1 -->|yes| G2{"Compiles,<br/>clippy clean?"}
    G2 -->|yes| G3{"Project rules<br/>hold?"}
    G3 -->|yes| G4{"Tests pass?"}
    G4 -->|yes| J{"Jev: does the code<br/>do what the task says?"}
    J -->|yes| A[Accepted, merged into the step]
    J -.->|not sure| JD[Claude as judge]
    JD -.-> A
    G1 & G2 & G3 & G4 & J -->|no| F{"Jev: what kind<br/>of failure?"}
    F -->|"typo, local mistake:<br/>errors go back"| B
    F -->|"card looks wrong,<br/>model goes in circles,<br/>attempts used up"| E[Back to Claude]

There is one more participant for the start. When I searched for how Jev is used, I found the recommendation to calibrate it on your own tasks before relying on it fully. So Claude was added as a judge: every question Jev answers is also put to Claude, and while this calibration is on, Claude’s answer decides (the dashed arrows in the second diagram). On the first step Jev cost about 300 times less than Claude for the same questions, which is the reason to want it to take over.

Bonsai (Bonsai 2 27B by PrismML) was chosen for a simple reason: on my machine, with one 12 GB graphics card, it had a reasonable balance between speed and quality, about 56 tokens per second with 128K tokens of context. A better candidate could probably be found, but at that moment it was a model that definitely worked, had a good context size, and I did not want to spend time searching for another one.

An independent review of Bonsai said that it plans well and codes poorly in agentic tasks. Fixed signatures and frozen tests were meant to compensate for exactly that: the model does not have to design anything, only to fill in bodies until the tests pass.

The first run

The first step was the client’s model: pure view state, room list, timeline, dates, previews. Six cards, 75 frozen tests. One card of six passed on the first attempt, one more on the second, and four came back to Claude.

When I looked at why, only one of the four was Bonsai’s failure: it converged on the answer, but ran out of attempts on the way. The other three were not its fault:

• The orchestrator itself had a bug. Bonsai’s code had passed every check, and the last thing left was to ask Claude, as the judge, the same questions Jev had answered. That request failed: the orchestrator named each question after the project rule it checks, with names like R:S4, and the colon is not allowed in the format Claude’s interface expects for these answers. So a correct card came back as a failure.
• One card asked for a function that returns bool when an operation fails. The project’s own rules say that failures are typed errors. Bonsai followed the card, and Jev correctly rejected the code for breaking the rules.
• One card asked for date calculations and said “use Hinnant’s algorithm”, without writing the algorithm down. Bonsai spent its whole token budget twice trying to derive it and did not produce a single line of code.

So in this scheme the architect’s mistakes look like the executor’s mistakes. A badly written card shows up as a failed attempt, and from the outside it is hard to tell which side is wrong.

What changed

The fixes went to where the problems were. The orchestrator got its bug fixed and a check that catches it before a run. The card with bool returns a typed error now. The date card got the formulas themselves, checked against Python’s datetime for every year from 1 to 9999. And the process for writing cards got two new rules: a known algorithm is written out, not named, and every signature is checked against the project’s rules before the card is cut.

The more general lesson is that the pipeline has to tell these failures apart. That is why Jev classifies every failure, and why “the card itself looks wrong” goes straight back to Claude instead of costing Bonsai more attempts.

What Bonsai itself does with the cards it gets is the subject of the next post.

I like trying new things. At work I use AI all the time, but as an assistant: I am in the loop for every feature. I agree with the agent on what it is going to write, correct it as it goes, and keep its work in line with the bigger picture of the project. This works very well. What I had never tried is the opposite: let the model work on its own from start to finish, choose the solutions, find its own mistakes and improve its own code.

At the same time I wanted to use my own experience. A model works better when its work is structured: there is a problem statement, a breakdown into tasks, steps that end with tests and a review of the design. Not “make it good”, but a process that looks like the classic way of building software, the one I know works. There is nothing new here. It is only a formalisation of what already exists, written down precisely enough to be run without me.

And I did not want to spend thousands of dollars on it. So the third thing I wanted to see is how the models work together when each one does the job that fits its price:

• Claude is the organiser and planner: it designs the interfaces, writes the tests and splits the work into small tasks.
• Jev, a small and very cheap decision model, is the orchestrator: it checks the result against the task and decides what happens next.
• Bonsai, a 27B model running locally on my GPU, does all the grunt work.

I had heard about this kind of setup, but I had never used it in practice.

The project

Then the question was what to build. I did not want another to-do application or another proof of concept. I have made too many pet projects like that, and they led nowhere. This time it had to be something I really need and will use every day.

I chose a desktop client for Matrix. Matrix is an open, federated chat protocol: anyone can run a server (I run my own), servers talk to each other the way email servers do, rooms can be end-to-end encrypted, and any client works with any server.

This may look like a strange choice, because there are already many Matrix desktop clients. But each of them has something that does not work for me:

• Element is the closest to what I need, but it runs only on Electron, which is heavy, and on my Hyprland desktop the left Alt key opens its menu, while I switch keyboard layouts with left Alt.
• FluffyChat is poor in features: it turns message HTML into its own widgets, so formatting comes out badly, and it has no threads.
• With the others there were other unpleasant things: no custom themes in Nheko, no text selection with the mouse in Fractal.

These are all good clients, and I have no big complaints about them. I could live with their shortcomings or send pull requests. The closest to my wishes was Tesseract, but it develops fast enough that keeping a fork would be expensive, and I did not want to carry its Windows and macOS code.

So two things came together: the wish to try a process where models write the code, and the wish to have a good client made for my needs, one I can change in any direction without depending on the plans, means or taste of its maintainers. That is how Exerakt was born: a Matrix desktop client for Wayland, written in Rust.

How it is going is the subject of the next posts.