People keep talking about guardrails as if they're a solution to AI safety. I'll say it plainly: they are not.
Most guardrails today are special-purpose LLMs trained to detect problematic inputs or outputs. On paper that looks clever: add a judge that says yes or no. In practice it fails. You've taken a probabilistic system and asked another probabilistic system to reliably police it. Prompt injections, jailbreaks, and adversarial inputs are fundamental vulnerabilities. Using the same class of system to enforce safety is a recipe for multiplied failure.
The Guardrail Illusion
Why do teams build guardrails this way? Because it's expedient. You can't always change the base model, so you add a check layer. It feels like defense in depth. But defense only works when layers fail in different ways. When each layer shares the same failure mode, you're stacking identical weak points.
Three Structural Failures
- The judge is probabilistic. It will miss adversarial prompts.
- Prompt injection and jailbreak techniques consistently bypass prompt-based controls.
- Context length and compaction cause "forgetting"—early instructions get compressed or lost during long interactions.
You can find this theme in recent security research and public demonstrations. I listened to Lenny's podcast with an AI security researcher who put the point bluntly: many guardrail strategies don't hold up under adversarial testing. The Wall Street Journal's demo of Anthropic's vending machine highlighted how context growth can cause models to lose their earlier constraints. These are not one-off failures. They're predictable from the architecture.
Why Stacking Fails
The instinct is to add more checks. Add a judge for the judge. Add monitoring for the monitor. That sounds like layered security. It is not. Defense in depth assumes heterogeneity of failure modes. Stacking homogeneous probabilistic checks simply increases the number of points that can fail in the same way.
Guidelines Versus Fail-safes
There is a difference between soft guidance and hard enforcement. Guidelines live in prompts. Fail-safes live in code. If an LLM's answer triggers a policy violation, your safety shouldn't depend on the LLM admitting it or declining to act. Put policy enforcement into deterministic runtime checks. If an agent tries to issue a refund above a threshold, a code gate should block that action before the LLM ever gets to decide.
Supervision: A Better Frame
Treat AI like an employee, not a perfectly obedient tool. Supervision accepts the base model is a black box. It focuses on inputs and outputs. Does the agent's behavior drift? Is it taking actions outside of constraints? Supervision monitors behavior, enforces policy in deterministic systems, and creates audit trails.
Practical Steps for Implementers
- Map the blast radius. Identify the actions where the business would be materially harmed.
- Enforce policies in code. Critical decisions must pass deterministic gates.
- Instrument and log everything. Auditors need evidence.
- Run adversarial testing on your supervision layer, not just the model.
- Consider smaller, purpose-built models for sensitive audiences (education, minors). Don't detune a general-purpose model and hope for the best.
Concrete Examples
Customer success: An agent should never process refunds above the accountant-approved threshold. Enforce that limit in code. Let the agent recommend but not execute.
Coding agents: Track diffs, require human approvals for production merges, and detect anomalous code changes.
Critical infrastructure: Don't deploy agents where a network pivot might alter operational systems without air gaps and containment strategies in place.
A Parent, Engineer, and Realist
I use these systems at home. My 11-year-old uses ChatGPT for homework and study cards. Parental controls on platforms like ChatGPT are a good step. But heuristic age detection is thin protection. If we care about kids, we should validate age and provide models designed for education, not tack on more guidelines to a general-purpose model and hope they stick.
The Bottom Line
Guardrails as they're typically implemented are security theater. They offer the appearance of protection without the structural properties that actually prevent harm. If you care about safety, move enforcement into deterministic systems, build supervision layers that watch for behavior change, and treat the labs' models as black boxes you must contain.
The responsibility sits with buyers and implementers. Don't outsource your risk to someone else's hopes and prompts.
