Guardrails for LLM Apps: Input, Output and Action Checks

Three places things go wrong

An LLM feature can fail in three distinct ways. Someone sends it something it should not process. It produces something it should not say. Or, if it can call tools, it does something it should not do.

Most guardrail discussions blur these together into 'make the AI safe', which leads to one enormous system prompt full of instructions and a false sense of security. Separating the layers makes it clear which risks each check addresses, and which checks are guarantees versus probabilities. If your system is an agent with wide permissions, our guide to agent security and permissions goes deeper on that side. This post covers the everyday LLM app: assistants, drafting tools, extraction and support features.

Input guardrails

Input checks run before the model sees anything. They are cheap, fast and mostly deterministic.

• Length and format limits. Reject a 200-page paste into a field meant for a customer question.
• Personal data detection. Redact or block card numbers, national insurance numbers and similar where they have no business being sent.
• Topic scope. A lightweight classifier that spots requests far outside the product's purpose.
• Prompt injection signals. Heuristics and classifiers flagging text that tries to override instructions, especially in retrieved documents and emails, not only user messages.
• Rate limits per user, which stop both abuse and runaway costs.

Be honest about the limits here. Injection detection reduces risk; it does not eliminate it. Any text the model reads can contain instructions, so the design has to assume some attempts get through. Our prompt injection defence guide covers why.

Output guardrails

Output checks run on the model's response before a user or system receives it.

Decide what happens on failure for each check. Options include regenerating once, falling back to a safe canned response, handing to a human, or blocking outright. A support bot that simply goes silent on failure frustrates customers more than one that says it is passing the question to a colleague.

Action guardrails: the layer that actually guarantees

Once a model can call tools, such as issuing refunds, updating records or sending emails, input and output checks are no longer enough. The boundary has to live in the code that executes the action, where the model cannot talk its way past it.

1. Give the model narrow tools with explicit parameters, never generic database or API access
2. Run every tool call with the permissions of the user it acts for, not a superuser account
3. Enforce hard limits in code: refund ceilings, record counts, allowed recipients
4. Require human approval for anything that spends money, deletes, or reaches a customer
5. Log every proposed and executed action with the reasoning and inputs

An instruction in a prompt is a request the model usually honours. A check in the tool's code is a rule it cannot break.

Avoiding guardrails that ruin the product

Over-cautious guardrails are a real failure mode. An internal HR assistant that refuses every question containing the word 'sick' is not safe, it is broken, and staff will go back to emailing HR or pasting questions into a consumer chatbot with no controls at all.

• Measure false positive rates on real traffic, not only whether bad inputs are blocked
• Scale strictness to the risk: a public chatbot needs more than an internal drafting tool
• Prefer 'redact and continue' or 'route to human' over 'refuse' where possible
• Review blocked requests weekly for the first months and loosen rules that catch legitimate use

A layered example: an order support assistant

Picture an online retailer handling a few thousand customer chats a week. Illustratively, the guardrails might look like this. Inputs are length-limited and screened for card numbers, which are redacted before reaching the model. The assistant can look up orders only for the authenticated customer, enforced in the lookup tool rather than the prompt. Replies are checked for delivery promises and refund amounts, and anything mentioning compensation is drafted for an agent to approve.

None of these checks is sophisticated. Together they mean the worst realistic outcome is an unhelpful reply, not a refund to the wrong person.

How SpiderHunts approaches guardrails

At SpiderHunts we start from a list of concrete harms for the specific feature, written with the client, then place each one in the layer that can best contain it. The deterministic checks come first because they are testable. Model-based checks fill the gaps, and each one gets its own small evaluation set so we know its miss rate and false alarm rate.

It is part of every build in our AI agent development and assistant work. The goal is not a system that can never misbehave, which does not exist, but one where misbehaviour is bounded, visible and cheap to correct.

Frequently asked questions