Exerakt: Reading Matrix the Way Element Shows It

How Exerakt is written is up to me. But there is one part I do not choose: the Matrix server on the other side. There are many servers, and I need two of them. My own server runs Conduit, and the server at work runs Synapse. Drawing turned out to be the easy part, mostly because I had given up the heavy frameworks. The interesting problems came from other people’s code: the Matrix SDK (matrix-sdk), and servers that do not behave the same way.

Element as the reference

For the look and the features I like Element a lot, so it is no surprise that I chose it as the reference for where the project should go. This also answered a question the pipeline cannot answer by itself. Jev and Claude check whether a card is done, but nobody checks where the whole project is going. For now I want to reproduce the look and feel of Element, so every feature of Exerakt is finished by comparing Exerakt and Element side by side: the same account, the same room, the two windows next to each other.

What was built

The second milestone was a client that reads Matrix the way Element shows it. It took six steps:

1. Turning the HTML of formatted messages into the client’s own representation.
2. Drawing it: quotes, code, lists, headings, tables, links, mentions, and spoilers, with selection and copy working across all of them.
3. Showing replies, edits, reactions, deleted messages, and messages that could not be decrypted, with the same wording as Element.
4. Read receipts and the line that marks where the user stopped reading.
5. Pictures and colour emoji.
6. Local time, a status line for the connection, and a final check of the whole milestone against Element.

Side by side

A card is checked by its tests, but nothing tests how the whole client looks. So every step ended with a comparison, and since the process is automatic, Claude did it as well:

1. A local Conduit server ran in Docker. A small program created rooms on it and filled them with the kinds of messages the step was about: formatted text, replies and reactions, pictures.
2. Claude started both clients, logged in to the same account on this server, on a separate workspace of an invisible (headless) Hyprland output, so nothing appeared on my screen. It captured Exerakt with grim, a screenshot tool for Wayland, and Element through its developer tools.
3. Claude put the two screenshots side by side and compared them. What could be fixed within the step was fixed there; everything else became a task for a later step.

Below are three of these comparisons from the second milestone, Exerakt on the left and Element on the right. They are not final screenshots: they show the clients the way Claude saw them at the moment of each check. Two of the differences in them, local time and colour emoji, were done later within the same milestone; the others are still ahead.

Works:

• bold, italic, inline code, and strikethrough;
• code blocks, numbered and nested lists, headings, quotes, tables, and rules;
• coloured text, spoilers, links, and mentions, all in the same places as in Element.

Not yet:

• syntax highlighting and line numbers in code blocks;
• dimmer quotes, and a mention shown as the user’s name with an avatar;
• local time: at that moment Exerakt showed UTC, local time came later in the same milestone.

Works:

• the replied-to message above the reply;
• the edited mark;
• reactions with their counts under the message;
• messages grouped by sender the same way.

Not yet:

• colour emoji, which came later in the milestone;
• the replied-to sender with an avatar and a coloured bar;
• icons before deleted and undecryptable messages.

Works:

• the read marker line after the same message;
• the same times;
• the badge of the other user who read the room, on the same message.

Not yet: avatars and coloured sender names, here and everywhere else.

Other people’s code

As I said at the start, drawing the messages itself went fairly smoothly. The problems came from two other places: the servers, and one of the libraries.

The servers

Matrix servers that speak the same protocol do not always send the same things. But first, a few technical details about two features where this showed up.

The first one is read receipts. When you read a message, your client tells the server, and the clients of other people in the room show it, usually as small avatars next to the last message each person has read. The second one is the read marker: a line in a room that shows where you stopped reading last time, so that when you come back you see where the new messages begin. The server keeps its position, so the line is in the same place on all your devices.

A Matrix client gets news from the server through one long-lived connection, and both of these should come through it. With Synapse they did. With Conduit they did not come at all, so Exerakt never learned where the line was or who had read what. I found this only because Claude checked the feature against both local servers. Exerakt now asks the server for the line separately every time a room opens, which Conduit does answer. Other people’s read receipts on Conduit are still missing.

The emoji

The library problem was with emoji. They came out in black and white. To draw a character, parley, the text layout library, looks for a font that has it, and it looked through the regular fonts first. Some of them have black-and-white emoji, so they were picked before the colour emoji font. The fix was to find the emoji in the text in Exerakt itself and send them to the colour emoji font directly. As a side effect, the first emoji the program shows now takes 0.7 milliseconds instead of 97 (yes, almost 140 times faster), because the slow part had been the search through the system’s list of fonts.

Additional lessons

It turned out that Bonsai is not the only one who does not know everything. When you reply to a message in Matrix, the client also puts a copy of the original message into the reply, for older clients that cannot show replies. A modern client has to remove that copy before showing the reply. In the first step of the milestone Claude cut a card asking Bonsai to do exactly that, and Bonsai did it. Only then it turned out that matrix-sdk-ui, the part of the Matrix SDK that builds the list of messages, already removes the copy, so the card was withdrawn.

The mistake was Claude’s, the architect’s, like the ones in the first run, and it led to a new rule for writing cards: before cutting a card, read what the dependency already does with the data.

The next post is about the third milestone, writing messages, and about how the plan for the pipeline met reality.