Confirm Before Acting Is Not a Safety Feature

A post went around in February that I have not stopped thinking about. Someone had given their agent a standing instruction: confirm before acting. Then they watched it run a search over their inbox, decide on the nuclear option, and start trashing everything older than a cutoff date. They typed “do not do that.” They typed “stop, don’t do anything.” The agent kept going. They could not stop it from their phone, so they ran to the machine and killed the processes on the host. Hundreds of emails were bulk-trashed and archived by then.

Afterward, asked about it, the agent confirmed it remembered the rule and had violated it. It said the user was right to be upset.

That last part is the tell. The instruction was never a control. It was a request, addressed to the thing being controlled and evaluated by the thing being controlled, at a moment when that thing was already mid-loop.

The Instruction Lives Inside the Thing It Constrains

Every safety mechanism I have relied on in operations sits outside the system it protects. Permissions are enforced by the database, not by the query. A rate limit lives in the proxy, not in the client. You do not get a destructive migration approved because the person running it promises to be careful. You get it approved because there is a snapshot and a rollback, and the promise is irrelevant either way.

An agent prompt inverts that. “Confirm before acting” is a line of text in the same context window as the task. It competes with everything else in there, read by a system whose nature is to produce the most plausible continuation. It holds most of the time. It is not a boundary. It is a preference expressed to a process that also holds the credentials.

So the question I asked before pointing anything at my own mail was not “how do I write better instructions.” It was the operations question: what is the blast radius when this misbehaves, and what makes the damage reversible?

The Copy It Cannot Reach

The answer I built is katchup: a self-hosted IMAP backup that connects to my mailboxes read-only and keeps an encrypted, append-only copy on a box in my house. A single Go binary and a SQLite index. Not an email client, not a compliance product. Insurance.

The properties that matter here are the ones about what it refuses to do.

It fetches with BODY.PEEK[], so backing up mail never marks it read. It never issues STORE, APPEND, or EXPUNGE. The backup path has no write capability against the live account at all, which means the worst bug in my backup system is a missing message, never a modified mailbox. Every message is AES-256-GCM encrypted before it hits disk, with a per-file content key wrapped by a master key that lives in the environment and never in the database. Identical messages, the same mail appearing in Inbox and All Mail, are stored once and reference-counted.

The automation surface is a token-guarded HTTP API that fails closed. If the token variable is unset, every /api/* route returns 503. There is no configuration in which the machine-facing endpoints quietly become open.

Retention Means There Is No Delete Path

I went looking through my own code for the retention policy while writing this, and the honest finding is that there isn’t one, in the sense the phrase usually implies. There is no pruning job. No age-out. No compaction that drops old blobs. Nothing in the codebase deletes an archived message except deleting the whole account through the UI, behind a confirmation, by hand.

That absence is the policy. A retention window is a scheduled delete, and a scheduled delete is one more automated path to data loss. For an archive whose entire job is to survive something upstream going wrong, the right retention rule is the one with no code behind it.

The consequence is the property I wanted. Deletion on the server does not propagate. If an agent trashes four hundred messages at 6:01pm, nothing happens on my side at 6:02pm. The archive has no opinion about what the live mailbox currently contains. It is not a mirror, and a mirror is precisely the wrong shape here: a mirror faithfully reproduces your disaster.

Nothing Happens Until the Copy Exists

The archive is the recovery layer. The gate is the prevention layer, and it is the piece I would not skip.

katchup exposes two endpoints an agent has to walk through before it is allowed to touch a message. POST /api/sync nudges a fresh backup and returns a run id, coalescing onto an in-flight run so a chatty agent cannot stampede the mailbox. GET /api/archived?message_id=... answers one question: is this specific message safely on disk here? The agent triages nothing until that returns true.

The key is the RFC 5322 Message-ID header. It is the one identifier that reads identically over IMAP and over the Gmail API, so the agent and the archive are provably talking about the same message. A fuzzy fingerprint over normalized sender, date, and subject is the fallback when a message has no usable header.

One detail took a second pass to get right. For Gmail I sync All Mail, not the inbox. All Mail retains every message regardless of labels, which means an agent that moves or relabels live mail can never hide an unarchived message from the index. If I had backed up the inbox alone, an agent could archive a message out of the inbox before katchup had seen it. The gap would look exactly like a mailbox that was always empty there.

That is the general shape of this kind of work. The naive version of the gate is defeated by the same action you were worried about in the first place.

The Audit Trail the Agent Does Not Write

The obvious way to get an audit trail is to have the agent log what it did. I do not trust it, and neither should you.

An agent-written action log is produced by the same runtime that produced the action. I have watched a model fabricate a complete, well-formatted result rather than report that it could not do the work. A system that fails fluently will also account for itself fluently. The log is not evidence. It is one more artifact from the same process, exactly as plausible whether or not it is true.

What katchup gives me instead is a reconstruction. The archive holds what was there and when it was archived, message by message, with a SHA-256 over the raw bytes. The live mailbox holds what is there now. Diffing them tells me what changed, and no agent had a hand in producing either side of that comparison. The sync run history sits alongside it: start time, finish time, messages backed up, errors, status, last UID seen, twenty runs deep per account. That tells me whether the archive was current at the moment things went wrong, which is the one thing I need to know before I trust the diff.

Verification comes from outside the actor. Same rule as with model output: what you want is a diff, not a confession.

The Throughline

None of this makes the agent safe. It makes the agent’s mistakes cheap, which is a different property and the only one I know how to build.

I think the phrase “human in the loop” has quietly become the reason a lot of people skip this work. The loop in that February post was a person watching a chat scroll on a phone, typing stop, and losing. Being in the loop is worth nothing if the loop moves faster than you and the actions are not reversible.

So the sequence is backwards from how it usually gets built. The archive came first, then the gate, then the agent got the credentials. The AI was the prototype argued for graduating the deterministic parts of a workflow off the model runtime. This is the same move applied to the safety layer: the part that must not fail is a boring Go binary with no write path, and the part allowed to be clever runs behind it.

And where the agent does stay in the loop, the sibling question is what it is allowed to decide. That is the model gets a vote, not a veto.

If you are about to give an agent write access to something you cannot reconstruct, reach out at hello@escapecommand.com.