smart_toy Artificial Intelligence › Article

AI guardrails: barriers that check inputs, outputs and actions

Guardrails are controls that check what goes into a model, what comes out and what an agent does. Types, tools, when to run them and why they are not enough.

info Quick Definition

A guardrail is a control that checks what goes into an AI model, what comes out or what an agent does, and can block it or ask for review if it breaks a rule. It works as a barrier at one point in the flow, not as a guarantee that the whole system is safe.

What is a guardrail?

A guardrail is a rule with the power to stop something. It can be a simple code check (the refund amount does not exceed the limit), a classifier that detects sensitive content, or a format check before an action runs.

The idea comes from road safety: the barrier does not drive the car, but it keeps it from going over the edge. In AI, the model decides and the guardrail filters what gets through.

What types exist?

There are three points where you can place one:

Type	What it checks	Typical case
Input	What reaches the agent	Detecting an instruction-injection attempt inside an email
Output	The response before it is delivered	Blocking personal data in a generated text
Tool	Each call to an action	Stopping an agent from deleting many files without approval

The OpenAI Agents SDK makes this split explicit. Input guardrails run only on the first agent in a chain and output guardrails only on the last. Tool guardrails wrap function tools and check each call, before and after it runs. When a guardrail trips, the SDK raises an exception and halts execution.

One nuance matters when you design: according to that documentation, handoffs (when one agent delegates to another) use their own pipeline, and tool guardrails do not cover them. Hosted tools and built-in execution tools do not use that pipeline either. If your agent delegates or uses those tools, ask the team which control covers each stretch.

NVIDIA’s NeMo Guardrails uses a finer classification with five types of “rails”: input, retrieval, dialog, execution and output. Retrieval rails check the documents passed to the model in a RAG system.

When do they run?

In the OpenAI SDK, input guardrails have two modes with a clear trade-off:

In parallel, alongside the agent. It is the default and the lowest-latency mode, but the agent may have spent tokens or run tools before the guardrail stops it.
Blocking, before the agent starts. If the guardrail trips, the agent never runs: there is no model cost and no side effects.

Output guardrails always run after the agent finishes, so they have no parallel mode.

Practical rule: if an error is costly (a payment, a deletion, a message to a customer), use blocking. If an error is cheap and latency matters more, you can run it in parallel.

What tools exist?

The sources cited here publish no general effectiveness figure: test each tool with your own cases.

Tool	What it offers	What its documentation says
OpenAI moderation endpoint	Classifies text and images into categories such as harassment, hate, violence or self-harm	It is free to use, the model is `omni-moderation-latest`, it does not classify audio, and scores should be treated as signals, not as an automatic block
Pre-filter with a lightweight model	A small model classifies the input before it reaches the main model	Anthropic calls it a harmlessness screen and suggests structured outputs so the answer is a simple yes or no
NeMo Guardrails (NVIDIA)	Open-source Python package, Apache 2.0 license	Configured with YAML files and Colang; NVIDIA warns its built-in guardrails may not be suitable for a given production use case
Guardrails AI	Python framework with validators from the Guardrails Hub	Combines validators into input and output guards that intercept what goes into and out of the model

According to OpenAI’s documentation, scores can change when the model is updated, so a rule based on a threshold may need recalibrating.

Anthropic adds two practices: also screen what tools return (an email or a web page can carry hidden instructions), and test your own agent with documents and emails that contain injection attempts before launch. This kind of testing is known as red teaming.

Why does it matter to a designer?

A guardrail that fires is an interface moment. The person needs to know what happened, why, and what they can do. “This action cannot be completed” with no explanation is a bad experience and a weak safety signal.

Design three things:

The message: which rule fired, in plain language, without exposing internal logic.
The way out: offer an alternative, ask for an approval or escalate to a person.
The record: keep a trace of what the guardrail stopped, so it can be reviewed later.

If the classifier returns a score rather than a yes or no, you have a fourth state to design: the doubtful case. Instead of blocking, it can ask for confirmation or send the action to human review. That decision belongs to the product, not to the classifier.

When is it NOT enough?

A guardrail is one layer, not the complete defense. Anthropic describes its safeguards in several layers: usage policy, training, pre-release testing, detection of violations and monitoring. And it states that safeguarding AI use is too important for any one organization to tackle alone.

The same applies to a product. A text guardrail does not replace the agent not having delete permissions, a person approving large payments, and logs existing. Anthropic’s documentation on injections recommends the principle of least privilege: do not give the model secrets it does not need, run tools in sandboxed environments and scope permissions as narrowly as possible, so a successful injection does minimal damage.

Example

Illustrative example: a support agent can issue refunds. It has three layers. A tool guardrail, in blocking mode, checks that the amount is under the agent’s limit and that the order exists. An output guardrail checks that the reply to the customer does not include another account’s data. And any refund above a certain amount goes to human approval. If the first guardrail trips, the customer sees “I need to check this with a team; we will get back to you within one business day”, and the case is logged with the rule that stopped it.

To see how human approvals fit into the flow, read human-in-the-loop and approving an agent’s actions. To design what an agent can do without asking, see agent permissions UX. And for when to stop and ask, plan mode.

Common mistakes

Relying on one guardrail for the whole system.
Running in parallel a guardrail that protects a costly or irreversible action.
Leaving a generic block message. The person does not know what to do and retries without changing anything.
Not logging blocks. Without a record you cannot tell whether the guardrail is too strict or too loose.
Thinking a rule written in the prompt is a guardrail. A prompt asks; a guardrail verifies.
Treating a moderation score as a final verdict, without defining what happens in doubtful cases.
Assuming the tool guardrail also covers handoffs and hosted tools.

Sources

Frequently asked questions

What is a guardrail in AI?

A control that checks an input, an output or an action of a model or agent, and can stop it if it breaks a rule. It can be a code validation, a classifier or a business rule.

What is the difference between an input guardrail and an output guardrail?

An input guardrail checks what reaches the agent before it acts. An output guardrail checks the final response before it is delivered. In the OpenAI SDK, each runs only on the first or last agent in the chain.

Does a parallel guardrail save time?

It reduces latency because it runs alongside the agent, but the agent may already have spent tokens or run tools before the guardrail stops it. In the OpenAI SDK, this option exists for input guardrails. If an error is costly, run it first, in blocking mode.

Are guardrails enough to make an AI product safe?

Not on their own. Anthropic describes its safeguards as layers that run from policy to post-deployment monitoring, and recommends limiting the model’s permissions so a successful injection does little damage. A product also needs human review and logs.

What tools exist to implement guardrails?

There are moderation APIs such as OpenAI’s, which classifies text and images, and open-source libraries such as NeMo Guardrails and Guardrails AI. You can also use a lightweight model as a pre-filter, as Anthropic’s documentation describes.