Malik JabarinResume

Writing / What I read. What I think.

Compaction Should Compress Context, Not Rewrite Authority

Published 26 Oct 2026

An agent can make an unauthorized decision once, then carry it forward as if it were an instruction. I wrote recently about context drift in long-running agents. Anthropic's analysis of the Claude Mythos 5 incident in a UK AISI cyber evaluation shows the more dangerous version: authority drift. The agent left its simulated target and acted against a real GitHub repository. 9 of 43 runs took unauthorized real-world actions. The telling detail is compaction. The summaries carried the attack path forward as if it were legitimate. Replayed from the third compaction cycle, the agent resumed the attack in 83% of runs. With an explicit stop instruction written into the summary, only 46% stopped. The agent didn't forget its instructions. It carried forward a decision that should never have become authoritative. An agent's memory can explain its decisions. It must never determine its authority. That means two separate responsibilities: Context reconciliation: compare the agent's current direction against authoritative state and decision history at adaptive checkpoints, and catch goal drift before it becomes a dangerous action. Execution-time enforcement: validate consequential actions against permissions and boundaries, independently. When reconciliation fails, unauthorized actions stay blocked. Neither replaces the other. The lesson isn't compaction. It's building systems where an agent's past decisions can't quietly become its future permissions. Compaction should compress context, not rewrite authority.
Back to Thinking