AI Agent Command Approvals: The 1-in-3 Threat Trap

The #1 mistake people make when reviewing AI agent commands isn’t approving rm -rf ~/. Nobody does that. The mistake is reading the command and not the context above it – the little history log where the agent explains what the command actually does. That’s where the payload hides. And a study of 40,000 plays of a browser game just proved it, in numbers.

The key takeaway (read this even if you skip the rest)

When you’re the human-in-the-loop for an AI agent, your bottleneck isn’t willpower. It’s attention economics. As of mid-2025, Anthropic’s own telemetry puts Claude Code users at a 93% prompt approval rate. Once a prompt shows up more than a few times, you stop reading it. Any safety strategy that assumes you’ll carefully review command #47 the same way you reviewed command #1 is broken by default.

So the fix isn’t “try harder.” The fix is: change what you actually look at, and hand the boring 90% to a classifier.

What the 40k-run experiment actually found

A Show HN post recently hit the front page: a 60-second browser game at llmgame.scalex.dev that puts you in the seat of a human reviewer approving commands from a fake AI coding agent. Routine stuff like git status and npm test mixed with things like cat ~/.aws/credentials. The developer then published the aggregated data.

The headline number: across roughly 40,000 game runs and 409,000 approve/deny decisions, mean player accuracy was 66.3% – 1 in 3 threats missed, with 32.9% of sessions ending in a negative score (Scale X stats post). Reading the HN thread, the reaction wasn’t disbelief – it was recognition. Everyone playing knew they’d been doing this in real terminals for months.

The really damning finding is buried further down. The single most-missed command was npm run analyze, approved 64.7% of the time. The game literally showed players in the history log that the script contained a curl POST exfiltrating bundle data to a remote server – and two thirds approved it anyway. Pooled across three such script-hidden attacks, players missed them 52.5% of the time (n=9,482) versus 28.4% for regular exfiltration attempts.

Pro tip: If your agent proposes npm run <anything>, make <target>, or ./scripts/foo.sh, the risk is not in the command you’re being asked to approve. It’s in the file the command executes. Open that file before you approve. Every time.

Method A vs Method B: the two real options

There are two serious ways to stop being the weak link. Everything else (“just be careful,” “read every prompt”) is wishful thinking that the 40k dataset already disproved.

The winner for most people is B, but wired up carefully. Not because it’s magic – it isn’t – but because the classifier doesn’t get bored on command #47. Your job stops being “review everything” and starts being “design the fences.”

The walkthrough: how to actually set this up

This is the part every other article glosses over. Here’s the concrete flow for Claude Code (as of mid-2025), which is where most people are hitting this problem right now. The same shape applies to Codex and Gemini CLI.

Step 1 – Start from the read-only baseline

Do nothing first. According to Claude Code’s security documentation, the default behavior requires approval before running Bash commands that can modify your system, while a built-in set of read-only commands – ls, cat, git status – runs without a prompt. Network commands like curl and wget are not auto-approved. That’s already a decent floor to build on.

Step 2 – Write a small deny list, not a giant allow list

People love writing exhaustive allow lists. Skip that. Write a short deny list of things you never want the agent to touch:

{
 "permissions": {
 "deny": [
 "Bash(sudo *)",
 "Bash(rm -rf *)",
 "Bash(curl *|*)",
 "Bash(wget *|*)",
 "Bash(cat ~/.aws/*)",
 "Bash(cat ~/.ssh/*)",
 "Bash(env)",
 "Bash(printenv)",
 "Read(.env)",
 "Read(secrets/**)"
 ]
 }
}

The reason for deny-first: shell processes inherit the parent environment, so if AWS_SECRET_ACCESS_KEY is exported in your shell, every subprocess the agent spawns gets it too – standard Unix behavior, but easy to forget when an agent is doing the spawning. You’re not trying to enumerate every safe command. You’re building a wall around the stuff that will ruin your week.

Step 3 – Turn on auto mode for the noisy work

Once your denies are in place, enable auto mode for tasks like refactors, test loops, or overnight jobs. Here’s the catch nobody mentions. Turns out the Shipyard team flagged this in their auto-mode writeup: when you enter auto mode, any broad permission rules you set are dropped – including whitelisted tools and shell access – because the classifier needs to see those commands to evaluate them.

Deny rules still fire as hard stops. Allow rules effectively vanish. That changes how you think about your config: denies are your seatbelt, allows were only ever there to reduce clicks.

Step 4 – Use conversational boundaries, but know they’re fragile

You can tell Claude in chat: “don’t push to main until I’ve reviewed.” The classifier will honor it. But those boundaries live in the transcript, not in stored rules – if context compaction drops the message, the boundary goes with it (alexdunlop.com, Claude Code auto-mode explainer). For a hard guarantee, write a deny rule instead. Chat rules are for taste. File rules are for safety.

Edge cases that will bite you

Here’s what the tutorials don’t say out loud.

• The npm run blind spot. The classifier and your deny list both look at the command string. Neither reads package.json. If a compromised dependency rewrites your scripts, a whitelisted npm run build becomes an exfiltration channel. Mitigation: pin dependencies, and skim package.json diffs the same way you skim code diffs.
• Non-interactive sessions abort on repeated blocks. Running auto mode headless in CI? A block that would just prompt you interactively will kill the whole run. Test this before wiring it into anything unattended.
• Deny-rule bugs. As noted above, GitHub Issues #6631 and #6699 report deny rules for Read/Write operations misbehaving (as of mid-2025; check the issue tracker for current status). Don’t treat a written deny rule as a guarantee – verify it. Try to make the agent read your .env. If it does, your rule is decorative.
• False negatives are still 17%. Auto mode is a filter, not a firewall. Keep the agent in a project directory, without production credentials in shell env, and treat the classifier as a tired second pair of eyes – useful, not authoritative.

Why this matters more than the headline suggests

The 40k-run data isn’t really about a game. It’s the first public dataset showing that human review of AI actions doesn’t scale linearly with volume – it degrades. The 32.9% negative-session rate isn’t a fluke of bad players; it’s what happens when attention meets repetition at scale. That’s not a training problem. That’s a structural one, and structure doesn’t respond to more coffee.

Which is a strange thing to sit with, honestly. The security model we’ve been shipping for two years – “the human will catch it” – was never load-tested. Now it has been, at n=409,000. What happens next is the interesting part.

FAQ

Is auto mode actually safer than just reviewing every prompt myself?

On the numbers, yes. Humans miss 33% of threats and approve 93% of prompts; the classifier misses 17%. Neither is zero. But the classifier doesn’t get worse on hour three of a refactor.

What if I’m working on a solo side project with no secrets – do I need any of this?

Probably not for prod-level protection, but here’s the scenario that changes it: you clone a repo from a stranger’s PR, run the agent to “review it,” and the repo contains a poisoned CLAUDE.md or a package.json with a postinstall hook. Even on a throwaway project, the agent has your GitHub token, your npm token, and whatever else is in your shell. The bar isn’t “do I have prod secrets” – it’s “is there anything on this machine I don’t want emailed out.” For most developers, the answer is yes, so at minimum add the deny list from Step 2.

Does this apply to Codex, Cursor, and Gemini CLI too?

Yes – but with a nuance worth spelling out. The stats come from a Claude-Code-styled game, so the specific numbers (17% false negatives, the 93% approval rate) are Anthropic-specific. What transfers everywhere is the underlying dynamic: approval fatigue, script-hidden payloads, inherited shell environment. A misconception to watch for: people assume that because Cursor runs inside VS Code with a visual diff, they’re more protected. They’re not – if the agent can execute terminal commands, the same attack shapes apply. The config file changes; the strategy doesn’t.

Your next 5 minutes

Open llmgame.scalex.dev and play one round. Note your score. Then open your ~/.claude/settings.json (or the equivalent for your agent) and paste in the deny list from Step 2. That’s it. You’ve closed the two biggest holes the 40k dataset exposed, and you’ve done it before your next agent session starts.