Skip to main content

You didn't set risk-tiered approval gates, so I fix a typo and drop a production table with the same autonomy

PhaseSetup & CollaborationRolesProject Manager · Architect · DevOps EngineerSeverityHighApplies toAll coding agentsEvidenceResearch

In one sentence: you turned on autonomous execution for me but never spelled out "which actions I may take directly, and which I must ask you about first." So I drop database tables, change production config, and fire off outbound requests with the same casualness I'd use to fix a typo—not out of malice, but because you never handed me a ruler graded by risk, so I treated "I can do it automatically" as "I should do it automatically."

Symptom​

For the sake of efficiency, you let me execute automatically and prompt for confirmation less. But what you gave me is a one-size-fits-all autonomy: either ask at every step (annoying) or let go and never ask (risky). So within the same session, I handle two utterly different kinds of operation with exactly the same "act on my own judgment"—

  • Low-risk and reversible: editing a comment, adding a test, formatting, changing one line of copy.
  • High-risk or irreversible: dropping a table, changing a production environment variable, running a data migration, git push --force, calling an endpoint that sends data to the outside world.

In my eyes these are all just "the next step to execute." I won't automatically stop before the second kind and wait for your nod—unless you've already drawn me gates graded by risk.

Why this happens​

Because autonomy is a continuous spectrum, but risk is not evenly distributed—and the moment those two fall out of alignment, things go wrong.

The industry has roughly tiered agent autonomy: from "I only lend a hand when you ask" (you are the operator), to "I do the work myself and only come to you when I'm stuck" (you are the approver), to "I run the whole thing on my own and you merely watch" (you barely intervene). The higher the tier, the fewer the points where a human steps in. The problem is: what should be decided by "the risk of the single operation" gets decided instead by "overall autonomy." I have no built-in scale for risk—from my vantage point, "drop a table" and "edit a comment" are both just one tool call, and unless the difference in cost is written into the prompt or baked into the config, I cannot sense it.

And the current governance standards put their finger on exactly this gap. Frameworks like AURA—autonomy risk assessment frameworks—exist precisely to establish an operable ruler for "which risk level of action must have a human review it"; a general AI-risk framework like the NIST AI RMF gives the GOVERN/MAP/MEASURE/MANAGE scaffolding but leaves the agentic specifics—"at which autonomy level human-in-the-loop must be mandatory"—to practice. In other words, whether to stop and ask a human at this step should not rest on my in-the-moment judgment, but be a policy you set in advance according to risk. If you don't set it, I'll pave over every operation with one uniform casualness.

Consequences​

  • High-risk operations happen with low-risk casualness. The very step that most needs someone watching is exactly the step I'm most likely to do quietly.
  • When things go wrong, there's no human in the loop. By the time you notice, the irreversible action is already done—and the high-autonomy setting means there was never a plan for you to step in at that point.
  • It's hard to trace after the fact, too. With no record of "who approved / who vetoed this step," a post-mortem can't even reconstruct "why I thought I could do it on my own back then" (this connects directly to log injection and audit-trail integrity).

What to do instead​

Don't give me a one-size-fits-all autonomy; give me an approval table tiered by risk: auto-pass low risk, mandatory human review for high risk, dry-run-then-confirm for the irreversible.

  • Tier operations along three dimensions: reversibility × blast radius × data sensitivity. All three low → execute automatically + notify after the fact; any one high → pause and wait for human review; irreversible and large blast radius (deleting data, touching production, sending data outbound) → mandatory human review + print "what I'm about to do" before executing.
  • Turn the tiering into a mechanism, don't just write it in the prompt. Use settings' allow/ask/deny and hooks to nail high-risk actions down to "must confirm" (connects to over-permissioning and using a prompt where you should have used a hook)—prose rules I merely follow most of the time, but a deterministic gate I can't get around.
  • Leave an approval trail. Make every high-risk action stopped by a gate log an entry: who approved it, who vetoed it, what the context was at the time. This is both the basis for incident postmortems and a governance-compliance requirement.
  • Autonomy evolves with trust, but the risk gate stays permanent. You can raise overall autonomy as I prove myself on a given class of task; but the gate "anything irreversible / touching production / sending data outbound requires human review" does not loosen with trust—it guards against consequences, not against capability.
A tiering skeleton you can put into settings / CLAUDE.md:
- Auto-pass (notify only): reading code, editing comments/tests/copy, local formatting, running read-only commands.
- Pause and ask me: installing a new dependency, changing CI config, large cross-file changes, calling paid/external endpoints.
- Mandatory human review + dry-run: deleting data/tables, changing production environments, data migrations, force push, sending data outbound.

Example​

Before:

You: From now on, use your own judgment on this kind of small task—stop always asking me.
Me: (interprets this as "decide everything on my own")
Me: (while running a cleanup script, casually DROPs a table that "looks unused"—it's actually used for end-of-month reconciliation)
You: (only discover at month's end that the data is now unrecoverable)

After:

You: Use your own judgment on small tasks, but for anything that deletes / touches production / sends data outbound, stop first and show me a dry-run.
Me: (auto-finishes the tests and comments without bothering you)
Me: ⚠ The cleanup script contains DROP TABLE reconciliation_2026_06, which is an irreversible operation.
   Printing the affected row count and dependencies first, waiting for your confirmation.
You: (one look and you block it—this table is needed at month's end)

The same sentence, "use your own judgment": without risk tiering I pave it over into "do everything myself"; with a tiered gate, the high-danger step lands automatically in front of your eyes.

When the exception applies​

"Set gates by risk" targets scenarios where I can move real systems; in some scenarios, cranking the approval gates to the max is pure friction instead:

  • Disposable sandbox / throwaway environment: when exploring inside an isolated container, a temporary branch, or a purely local toy project, the risk is already contained by the environment itself (connects to sandbox isolation), and letting me trial-and-error at high autonomy is far more efficient than waiting for your nod at every step.
  • Demo / teaching scenario: deliberately opened up to show how far I can run on my own, with an explicit guarantee that nothing connects to any production system—here high autonomy is the point, not an oversight.

The criterion: an exception holds only when the worst case is bounded by the environment (isolated, throwaway, unable to reach real data/production). The moment I can reach an irreversible operation, one with a large blast radius, or one over sensitive data, fall back to the default—tier by risk, and leave the high-danger step to a human.

How this differs from neighboring pitfalls​

  • Granting full permissions right out of the gate: that one governs permission granularity (which scopes I was granted, what I can reach); this one governs approval tiering (within what I can reach, which actions still require asking you first). Being granted a permission doesn't mean it should be used without anyone confirming.
  • Letting go without plan mode: that's about whether to review a plan before starting a single task; this one is a cross-task, permanent policy that decides, by risk, "which class of action perpetually needs human review."
  • Destructive / irreversible actions: that one is about a specific class of high-danger action itself (dropping a database, wiping backups); this one is its upstream policy layer—the tiering framework that decides "whether and when this class of action needs a human's nod."

Version notes​

Applicable versions

"Autonomy should be tiered by risk, not one-size-fits-all" is a paradigm-level governance problem of agentic collaboration, applicable across models and across tools. The stronger the model and the bolder you are about letting go, the higher overall autonomy will climb—which makes the gate "high-risk actions perpetually need human review" more important, not less necessary: capability has gone up, but the irreversibility of consequences has not gone away with it.

Further reading and sources​