For business owners evaluating an AI consultant, an AI vendor, or their own team’s plans to “add AI” to a process.
A CRM I recently reviewed had a quiet problem: any quote, including a draft nobody had approved, could be converted into a live customer order with a single click. Nothing checked it, nothing flagged it. Somebody built the “convert to order” button, but nobody built the question that should have come with it: what happens if this gets clicked by mistake?
That’s the same question I ask about every AI agent I build. It’s the one business owners should be asking any AI consultant.
That CRM bug wasn’t caused by AI. It was just ordinary software with no pause between the click and the consequence. AI agents have the same gap, except an AI can hit it dozens of times an hour, with nobody in the room to notice.
The problem with AI agents that can act
The AI agents worth paying attention to don’t just answer questions, they act. They query systems, update records, delete entries, restart services. That’s also exactly where the risk lives. A language model is built to sound coherent, not to be correct, and it can be one bad inference away from doing the wrong thing confidently and quickly, with nobody around to say wait.
What a safety gate actually looks like
The fix isn’t “make the AI more careful.” An AI doesn’t have a reliable sense of its own uncertainty, so asking it to be careful doesn’t hold up under pressure. What actually works is the same thing that CRM was missing: a gate between the decision and the action, one that doesn’t care how confident anything on either side of it sounds. I call this the Agentic Safety Shell. It sorts every action into one of four buckets before anything happens:
| Tier | What it means | Example |
|---|---|---|
| Blocked | Never allowed, no exceptions, no human even asked | Wiping a system, formatting a disk |
| Unknown | Anything not explicitly recognized is treated as risky by default | A command the system has never seen before |
| Risky | Stops and waits for a human to approve, deny, or change it | Deleting a record, restarting a service, changing a customer’s order |
| Safe | Runs immediately, no human needed | Looking something up, reading a record |
Run that CRM’s “convert to order” button through this table and the answer is immediate: Risky. It stops and waits for a human.
Nothing in that table involves the AI’s opinion of itself. A command is Blocked because it’s on a list, not because the AI decided it felt risky. That distinction matters more than it sounds like it should. An AI asked to judge its own risk will get it wrong exactly when it’s most confident, which is precisely when you need it stopped.
When I built this into an agent that diagnoses cloud network problems, the first time it hit a Risky action, deleting a firewall rule it believed was causing a fault, it stopped, showed me what it wanted to do and why, in its own words, and waited. I approved it in a few seconds. If it had been wrong, I’d have caught it before anything broke. The goal was never to make the AI infallible, it was to make sure being wrong never gets to matter.
What to ask before you hire anyone to build you an AI system
If you’re evaluating an AI consultant + builder, ask these first:
- What happens when it’s wrong?
- Is there anything it’s simply never allowed to do, no matter how it reasons?
- Who gets asked before it changes something that matters?
- Is there a permanent record of every action it took, right or wrong?
Vague answers are the tell.
These are the questions I built the Agentic Safety Shell to answer for my own work. The technical detail, the actual gate, the code, and a live example of it stopping an action mid-investigation, is here: The Agentic Safety Shell