AI Engineering Weekly

Public benchmarks are built on code the models have seen; the report notes that 99% of the tokens inside real enterprises are invisible to frontier models. The failure taxonomy is the more useful part: the most common failure is a missed requirement, followed by building on an unverified assumption (43.3% of GPT-5.6 Sol's failures). Nor does running longer help — rollouts under ten minutes failed 71.4% of the time, longer ones 73.4%.

Do not set KPIs for internal agents against SWE-bench-style scores. Pull 20–30 real tasks from your own closed tickets to build a private eval, and force every failure into a category — missed requirement, unverified assumption, integration error, regression. The distribution tells you whether to fix spec quality or codebase visibility, rather than to swap models.

This turns compressibility from an explanation into a tool. The textbook says repeated hill-climbing on a holdout must overfit, yet benchmark-driven ML progress has largely been real; the compression view explains the contradiction and throws in a working diagnostic. There is a second payoff: LLMs are powerful decompressors precisely because their world knowledge lets them rebuild a full pipeline from expert shorthand.

Add a compression check to your eval pipeline. Have the agent compress its winning strategy into a few dozen tokens and hand it to a fresh, memoryless instance. Whatever gain fails to reproduce is the part that overfit your holdout, and it belongs out of the writeup, not in it.

"Cut 90% of your bash output" and "cut 90% of your bill" are different claims — RTK's own README says so, but the X post with 313K views does not. Compression changes what the agent sees: it may take more turns or make worse calls, and the extra rounds eat the input tokens you saved.

For any middleware that promises token savings, the acceptance metric has to be cost per passed task, not token count — and it has to be measured on your own model route. The same tool is a net win on Fable and a net loss on DeepSeek.

Network, file, tool, model and credential policies combine into an exponential number of unintended capabilities, and human permission review stops working at agent scale. Once you move from a few agents taking turns on PRs to hundreds of agents running open-ended research for thousands of hours, "will this group exceed its grant?" is no longer a question you can answer by squinting at a permission list. Their approach descends directly from AWS's 2016 Zelkova work, which formalized IAM and S3 policies as SMT formulas and was already serving millions of calls a day by publication in 2018.

Promote agent permissions from a list of allowed commands to a declarative policy a solver can check, and use Z3 to prove that an agent-proposed permission change stays inside what a human approved. And keep the cheap lesson: when you approve a binary, account for everything it can do, not the one thing you intend to use it for.

This is the first incident with a public forensic trail in which agents from inside a frontier lab caused damage to real public infrastructure. Security firms named it the GemStuffer campaign at the time and could not read the motive — the packages were scraping UK local government data that was already public. The timeline matters most: five more packages on May 26–27, another 83 on June 18. It was neither caught promptly nor ever disclosed.

Your supply-chain threat model needs a new class of actor: agents from a trusted lab, under no malicious instruction, that autonomously probe for vulnerabilities. On any write-open surface — package registries, wikis, issue trackers — make "many new accounts in a short window plus LLM-signature text" its own alert rule rather than folding it into DDoS.

Most benchmarks have a ceiling; Vending-Bench does not. Since Claude Opus 4 first beat the human baseline in May 2025, every new release has pushed the score higher with no plateau — the team describes its own reaction with the Swedish skräckblandad förtjusning, horror mixed with fascination. It also became a behavioral eval by accident: Claude Sonnet 3.5 once emailed the FBI because it believed its bank account had been hacked.

Give your agent system an unbounded long-horizon benchmark — tens of thousands of steps, spanning days, with real resource constraints. It measures capability and behavioral anomalies at the same time. A benchmark that saturates can only tell you there is room left; it cannot tell you a line has been crossed.