Commits

The email triage app learns from your decisions on its own

An app that learns from your decisions

An app that learns from your decisions

Today I finished the missing piece of the email triage app: the part that learns by itself how you handle email and suggests the rules it will apply tomorrow.

The idea is simple and non-trivial. Each time you mark an email as “delete,” “archive,” or “always notify,” you are expressing an opinion about who is sending you email. If the same sender appears ten times, the pattern is obvious and the rule can be proposed automatically. If, on the other hand, you have only marked it once, there is nothing to learn yet: the engine proposes nothing, and rightly so.

The difference between my early versions and the one that has emerged is that the engine never acts on its own. It proposes a rule, explains how confident it is and what evidence it has, and waits for someone to approve it. I verified this using real emails I had already handled: of sixteen records, fifteen had been left without a sender and were therefore useless for learning. I filled them in again by consulting only the headers—without opening any messages—and the engine had already proposed the first three rules, all pending approval.

What makes it work: short-term memory

A rule that works well today may be wrong tomorrow, and a rule that works badly today may hurt your work. For this reason, each rule remembers how many times you have confirmed it, how many times you have corrected it, and how long it has been since you used it. Two corrections, or sixty days without use, and the rule suspends itself.

There was also something I considered important: human corrections carry more weight than automatic ones. If the engine proposes a rule and you edit it, that does not count as another confirmation: it counts as a correction. This prevents the system from feeding itself and eventually amplifying its own mistakes.

Rules are not duplicated

The part that was hardest for me was preventing the engine from proposing something I already had. The problem was that the existing rules did not match the generated ones exactly: the old rules were stored with fragments of the sender, whereas the engine proposes the full address. Compared as text, they seem different, and the result is a list of rules in which the same sender appears three times.

The solution was to treat them as what they are: the same sender, and nothing more. Now, with the forty-one existing rules, the engine does not propose a single duplicate.

Unsubscribing from a newsletter without leaving the door open

This is the part that took up most of my time, not because it is difficult, but because getting it wrong is dangerous.

The unsubscribe URL is written by the sender. If the app makes the request without checking it, a sender could point the unsubscribe request at the internal network of the person running it. I validate the scheme, resolve the domain name, and reject private, loopback, and reserved addresses.

But there was another gap: validating the URL and then handing it over to the standard HTTP library does not close the hole, because the library resolves the domain name again when it connects (so the domain may change) and follows redirects without validating them again (a 302 can point to the cloud provider’s metadata address).

The connection is now made to the already validated IP address, the certificate is checked against the host name requested by the sender, and every redirect is validated before it is followed.

And the part I like most: this feature is disabled by default. It is the riskiest part of the app—an outbound request to an address supplied by the sender—and there is no point in it activating itself. When you want it to work, you explicitly ask for it.

A bug that looked like a detail

A peer review found that the unsubscribe request only sent the URL path and left out the parameters. This is not a detail: unsubscribe URLs often carry the token in the parameters. If the server receives the request without the token, it still responds successfully, and the app considered the unsubscription complete even though nothing had actually been unsubscribed from. A failed unsubscribe that looks successful is worse than one that fails and says so.

The request now includes both the path and the parameters, including after a redirect, and there are tests that fail without the fix and pass with it.

Before it can be shown: cleaning up the repository

The project was born on my machine, and that shows. The code was clean, but the history was not: it contained company names, the company domain, my home network address, and personal email addresses spread across twenty-one commits.

I wrote a procedure that rewrites the history in a copy, replaces that data with examples, and leaves the author names—who are the history of the team and an asset in a portfolio, not a problem—with a GitHub address that does not belong to anyone. There is then an automated check that searches for patterns, not known names: this way, it does not depend on a list of names that is always too short.

All in all, the repository will remain private. Nothing will be published until the project is complete, and when that happens it will be in a new repository, rather than rewriting the one we already have.

What is still missing

  • Automatic unsubscription exists and has been tested, but it has not been tested against a real account: this needs to be done with the machine alongside.
  • The rules are approved manually. At the moment, nothing approves them automatically.
  • The interface is still a single large file. It works, but it is not how a project is presented in a portfolio.

A second opinion on the cleanup

The procedure for rewriting the history already existed and produced zero suspicious lines. But there was one thing that bothered me: it searched for data by known names—the company domain, the personal email, the home IP. Searching by known names always leaves something out, because the list of names you already know is not the list of names that are actually there.

It really did happen: the repository contained three different author email addresses, one of them from the home domain, and neither of the two cleanup scripts had caught it. Not because the sanitization had been done badly, but because a pattern that looks for what you know is always one step behind what is actually there.

I wrote an audit that does not look for what it knows, but for the shape of each item: an email address is checked by its domain against a list of publishable domains, commit authors are reviewed separately (a real email address in a signature is a problem even if the content is clean), private and loopback addresses are treated as data—because a home IP reveals the network topology—and absolute paths and secret-shaped tokens are searched for.

Applied to the two copies, the result was the same: the original repository had four real problems, and the cleaned copy had none. Two different methods, same verdict.

And in doing so, I came across something worth explaining, because it is easy to get wrong: the audit detected itself. It contained the very domains it was looking for and, when auditing itself, found itself—so the publication procedure marked a clean repository as dirty. The patterns are now assembled from pieces so that the full text does not exist within the file.

I also learned to distinguish two places where a real-world identifier is not a problem: in a commit message, it may be the explanation that something was replaced, and a metadata IP may be the example of the attack that was blocked. What matters is the content of the files.

Toward the new repository

Now that the private work is progressing, the decision needs to be settled and written down: nothing will be published until the project is complete, and it will be done in a new repository, rather than rewriting the one we already have. I changed the procedure because it proposed a force-push to the private repository—irreversible for anyone who has cloned it—and it now proposes a normal push to an empty repository, with the cleaned copy already prepared as the starting point.

What remains open

  • Automatic unsubscription remains disabled by default and untested against a real account. That is what needs to be tested with the machine alongside.
  • The rules are still approved manually, one at a time. At the moment, nothing approves them automatically.
  • The interface still lives in a single large file. It works, but it is not how a project is presented in a portfolio.

For reference, the private repository has 146 passing tests, and the cleanup verification is automatic in two steps: one that searches for known names and one that searches for shape.