Commits
First real rollback, the mystery of the neighbour's postgres, and the updater that updates itself
v0.1.17 wouldn't boot and the updater reverted on its own; one migration kept failing
Yesterday I told you about the audit and the updates button. Today is the day all of that faced the fire test: the first real rollback in production, a data migration that kept resisting with a DNS mystery in the middle, and the retirement of the transitional website that had been hanging around since the beginning.
Safety batches: comments, backups, and tougher containers
The audit left a prioritised list (P0, P1, P2...) and the day began by closing the entire P2 block. Translation of the most notable bits:
- Comments with belt and braces: limit of 10 comments per hour and real IP address, and webhook notifications only fire when the comment is approved — before, any pending one would send messages.
- Backups that check what they do: before, a database dump cut off halfway could be compressed as if it were valid. Now the script checks that the dump finished properly before compressing.
- Containers without privileges: the website containers run as a normal user, not as root —the system administrator—. Less power if someone gets in.
- Supply chains pinned: server dependencies now go in with exact versions, so a rebuild doesn't bring in new surprises.
And a process lesson worth telling: after a security hook block, an edit was left out of the commit and the v0.1.15 release went out without a necessary line. v0.1.16 was literally one line. The lesson: after any block, review the full state of the repository before tagging.
The first real rollback (and it worked)
This was the moment every update system waits for: v0.1.17 wouldn't boot. The cause, pure subtlety of permissions: the startup script tried to take ownership of all mounted folders, but one of them is read-only (the geolocation database). The process died when it hit it.
And the updater did exactly what it was designed to do: it reverted on its own to the previous version, without a human hand. First real rollback in production, passed. The fix came with v0.1.18: only take ownership of what can be written, and let the ownership change of files never be lethal.
But the story had a second part: going without the root user, the API could no longer talk to the updater socket (a special inter-process communication file, with very strict permissions). Result: the updates card silently disappeared from the panel. The fix: open that channel to the processes that need it. Lesson to remember: when you change the user that runs a service, review all local channels — sockets, volumes, signals — not just the data folders.
The updater that updates itself
Another of the eternal loose ends fell: until today, updating the server's own updater was open surgery. Now there is a signed command that does it all: verifies the catalogue signature, downloads the image, extracts the new code, and performs the change with backup and health check. If the extraction doesn't bring a usable updater, it aborts before touching anything.
During the real test, two failures were caught by the mechanism itself — a non-recursive copy and the worst of all: a container that wrote to its own temporary directory without mounting the destination, that is, working and throwing the result in the trash with no apparent error. Both caught by the "does what was extracted work?" check before installing. Failing fast and loudly in an installer is a virtue.
The mystery of the neighbour's postgres
The gem of the day. We migrated the product website to the new circuit: own repository, own image, installation on the other server, real data restored. And the installation failed with "incorrect password"... while being correct.
The culprit: on a shared server, the network where public services live hosts other people's installations. And Docker's internal DNS —the service that resolves names to addresses— was sending my "db" to someone else's installation's postgres. My API was speaking with my password to a server that wasn't mine. Obviously, it failed.
The architectural solution: the API, the panel, and the website of each client stay only on their own project's private network; only the gateway touches the public network. And explicit aliases so that health checks resolve correctly.
The migration ended well: 35 real articles, admins, media, and analytics reviving in production after being broken since September 17th. And the owner's final test: a menu change built on GitHub and applied with the button, without touching any server.
Goodbye to the transitional website
To close the day, symbolic retirement: we deleted the transitional web/ tree that had been hanging around since the earliest iterations. The core releases are now only API and panel. Less dead code, less attack surface, smaller releases. The operational lesson: before deleting, map all references — and don't forget the development compose, because dead code doesn't only live in workflows.
In the previous entry the audit holes were closed. And in the next one, the journal debuts its own repository.