Guardrails

See also

For creating custom guardrails, see Creating Custom Guardrails. For the underlying safety primitives, see Safety Primitives.

Guardrails are safety mechanisms that intercept and evaluate content before it enters or exits the AgentOS pipeline. They enable content filtering, PII redaction, policy enforcement, and mid-stream decision overrides.

Overview

Guardrails intercept content at two points:

1. Input Guardrails - Evaluate user messages before orchestration
2. Output Guardrails - Evaluate agent responses before streaming to client

User Input → [Input Guardrails] → Orchestration → [Output Guardrails] → Client

When multiple guardrails are active, AgentOS uses a two-phase dispatcher:

1. Phase 1 (sequential sanitizers) - Guardrails with config.canSanitize === true run in registration order so each sanitizer sees the cumulative sanitized text
2. Phase 2 (parallel classifiers) - All remaining guardrails run concurrently via Promise.allSettled, with worst-action aggregation (BLOCK > FLAG > ALLOW)

This keeps redaction deterministic while still allowing heavyweight classifiers and grounding checks to run in parallel.

Built-in Guardrail Packs

The extensions registry has six guardrail packs, each a standalone package:

Pack	Package	What It Does
PII Redaction	@framers/agentos-ext-pii-redaction	Four-tier PII detection (regex patterns from openredaction, an NLP prefilter, an NER model, an LLM judge) on input and output; a Phase 1 sanitizer. Tools: pii_scan, pii_redact
ML Classifiers	@framers/agentos-ext-ml-classifiers	Toxicity, prompt injection, NSFW and threat classification (ONNX models, with LLM and keyword fallbacks). Tool: classify_content
Topicality	@framers/agentos-ext-topicality	Embedding-based topic enforcement on user input, with LLM and keyword fallbacks; output passes unevaluated. Tool: check_topic
Code Safety	@framers/agentos-ext-code-safety	OWASP Top 10 code scanning (25 regex rules). Tool: scan_code
Grounding Guard	@framers/agentos-ext-grounding-guard	RAG-grounded hallucination detection via NLI. Tool: check_grounding
Content Policy Rewriter	@framers/agentos-ext-content-policy-rewriter	Opt-in content policy: a keyword pre-filter on streamed text and an LLM judge on the final response that blocks or rewrites it

Quick Start

import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos';
import {
  IGuardrailService,
  GuardrailAction,
  type GuardrailInputPayload,
  type GuardrailOutputPayload,
  type GuardrailEvaluationResult,
} from '@framers/agentos/safety/guardrails';

// Simple content filter
class ContentFilter implements IGuardrailService {
  async evaluateInput({ input }: GuardrailInputPayload): Promise<GuardrailEvaluationResult | null> {
    if (input.textInput?.toLowerCase().includes('prohibited')) {
      return {
        action: GuardrailAction.BLOCK,
        reason: 'Content violates usage policy',
        reasonCode: 'CONTENT_POLICY_001',
      };
    }
    return null; // Allow
  }
}

// Initialize with guardrail
const agent = new AgentOS();
const base = await createTestAgentOSConfig();
await agent.initialize({
  ...base,
  guardrailService: new ContentFilter(), // Optional config-scoped guardrail
});

Guardrail Actions

Action	Effect
ALLOW	Pass content unchanged
FLAG	Pass content, record metadata for audit
SANITIZE	Replace content with modified version
BLOCK	Reject/terminate the interaction

Mid-Stream Decision Override ("Changing Mind")

Guardrails can evaluate streaming chunks in real-time and "change their mind" about allowing content. The examples below import AgentOSResponseChunkType from @framers/agentos (as the Quick Start does): chunk types are lowercase strings such as 'text_delta', so compare against the enum. This enables:

• Stopping generation when cost ceiling is exceeded
• Blocking harmful content as it's being generated
• Redacting sensitive information mid-stream

Example 1: Cost Ceiling Guardrail

Stop generation when the response exceeds a token budget:

class CostCeilingGuardrail implements IGuardrailService {
  // Enable streaming evaluation
  config = { evaluateStreamingChunks: true };

  private readonly maxTokens = 1000;
  // Characters per response stream, and when each stream was last seen
  private readonly streams = new Map<string, { chars: number; seenAt: number }>();

  async evaluateOutput({ chunk }: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null> {
    // The stream's final chunk ends its count
    if (chunk.isFinal) {
      this.streams.delete(chunk.streamId);
      return null;
    }

    // Only evaluate text chunks
    if (chunk.type !== AgentOSResponseChunkType.TEXT_DELTA || !chunk.textDelta) {
      return null;
    }

    const now = Date.now();
    let stream = this.streams.get(chunk.streamId);
    if (!stream) {
      // A stream that ended without a final chunk left its entry: drop the ones idle for ten minutes
      for (const [id, idle] of this.streams) {
        if (now - idle.seenAt > 600_000) this.streams.delete(id);
      }
      stream = { chars: 0, seenAt: now };
      this.streams.set(chunk.streamId, stream);
    }
    stream.chars += chunk.textDelta.length;
    stream.seenAt = now;

    // Estimate tokens (rough: 1 token ≈ 4 chars)
    const tokensUsed = Math.ceil(stream.chars / 4);

    if (tokensUsed > this.maxTokens) {
      this.streams.delete(chunk.streamId);
      // "Change mind" - stop generating mid-stream
      return {
        action: GuardrailAction.BLOCK,
        reason: 'Response exceeded token budget. Please refine your request.',
        reasonCode: 'COST_CEILING_EXCEEDED',
        metadata: { tokensUsed, limit: this.maxTokens },
      };
    }

    return null;
  }
}

The guardrail counts characters per streamId, so each response is measured on its own, and the stream's final chunk, which every output guardrail is called for, ends the count. A stream can end without a final chunk (another guardrail blocked it, or it failed), so an entry left idle for ten minutes is dropped when the next stream starts. It sets no maxStreamingEvaluations: once a stream reaches that limit, the dispatcher stops calling the guardrail on the stream's text deltas (it still calls it on the final chunk), and the count would stop short of the budget.

Example 2: Real-Time PII Redaction

The @framers/agentos-ext-pii-redaction pack detects PII in four tiers (regex patterns from the openredaction library, an NLP prefilter, a BERT NER model, an LLM judge): emails, phone numbers, SSNs, payment cards, IP addresses, IBANs, passports, driver's licenses, other government IDs, dates of birth, API and AWS keys, crypto addresses, medical terms, and person, organization and location names. See the PII Redaction extension docs for full configuration reference.

import { createPiiRedactionGuardrail } from '@framers/agentos-ext-pii-redaction';

const piiPack = createPiiRedactionGuardrail({
  confidenceThreshold: 0.5,
  redactionStyle: 'placeholder',  // also: 'mask', 'hash', 'category-tag'
  enableNerModel: true,            // BERT NER for person/org/location names
  llmJudge: {                      // optional: resolve ambiguous cases
    provider: 'anthropic',
    model: 'claude-haiku-4-5-20251001',
    apiKey: process.env.ANTHROPIC_API_KEY,
  },
});

const agent = new AgentOS();
await agent.initialize({
  ...config,
  extensionManifest: { packs: [{ factory: () => piiPack }] },
});

The extension provides two agent-callable tools (pii_scan and pii_redact) and a streaming guardrail that automatically redacts PII from input and output. It sets canSanitize: true so it runs in Phase 1 (sequential) of the parallel dispatcher.

Custom regex-only PII guardrail

If you only need simple regex patterns (no NER, no LLM), you can write a lightweight custom guardrail instead. This demonstrates the IGuardrailService interface with SANITIZE action:

class SimpleRegexPiiGuardrail implements IGuardrailService {
  config = {
    evaluateStreamingChunks: true,
    canSanitize: true,  // Phase 1: runs before parallel classifiers
  };

  private readonly patterns = [
    { regex: /\b\d{3}-\d{2}-\d{4}\b/g, replacement: '[SSN REDACTED]' },
    { regex: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, replacement: '[EMAIL REDACTED]' },
    { regex: /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, replacement: '[CARD REDACTED]' },
  ];

  async evaluateOutput({ chunk }: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null> {
    if (chunk.type !== AgentOSResponseChunkType.TEXT_DELTA || !chunk.textDelta) return null;

    let text = chunk.textDelta;
    let modified = false;

    for (const { regex, replacement } of this.patterns) {
      const newText = text.replace(regex, replacement);
      if (newText !== text) { text = newText; modified = true; }
    }

    if (modified) {
      return { action: GuardrailAction.SANITIZE, modifiedText: text, reasonCode: 'PII_REDACTED' };
    }
    return null;
  }
}

Note: For production use, the @framers/agentos-ext-pii-redaction extension is strongly recommended over custom regex. It catches names, organizations, locations, and 50+ country-specific ID formats that regex alone misses.

Example 3: Content Policy Mid-Stream

Block harmful content as it's being generated:

class ContentPolicyGuardrail implements IGuardrailService {
  config = { evaluateStreamingChunks: true };

  private readonly prohibitedPatterns = [
    /how to make.*bomb/i,
    /instructions for.*weapon/i,
    // ... more patterns
  ];

  // The text each response stream has produced so far, kept as Example 1 keeps its counts
  private readonly streams = new Map<string, { text: string; seenAt: number }>();

  async evaluateOutput({ chunk }: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null> {
    if (chunk.isFinal) {
      this.streams.delete(chunk.streamId);
      return null;
    }

    if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA && chunk.textDelta) {
      const now = Date.now();
      let stream = this.streams.get(chunk.streamId);
      if (!stream) {
        for (const [id, idle] of this.streams) {
          if (now - idle.seenAt > 600_000) this.streams.delete(id);
        }
        stream = { text: '', seenAt: now };
        this.streams.set(chunk.streamId, stream);
      }
      stream.text += chunk.textDelta;
      stream.seenAt = now;

      for (const pattern of this.prohibitedPatterns) {
        if (pattern.test(stream.text)) {
          this.streams.delete(chunk.streamId);
          return {
            action: GuardrailAction.BLOCK,
            reason: 'Response contains content that violates our usage policy.',
            reasonCode: 'CONTENT_POLICY_VIOLATION',
          };
        }
      }
    }

    return null;
  }
}

Cross-Agent Guardrails

Cross-agent guardrails let one agent (supervisor) monitor and intervene in other agents' outputs. AgentOS attaches them to no stream on its own: neither processRequest() nor agency() reads them. The host wraps an agent's output stream with wrapWithCrossAgentGuardrails(guardrails, { sourceAgentId, observerAgentId, agencyId }, guardrailContext, stream, { streamId }) from @framers/agentos/safety/guardrails. Each guardrail that observes the source agent (observeAgentIds, every agent when empty) evaluates every chunk in turn, a non-final TEXT_DELTA only with evaluateStreamingChunks: true; a BLOCK ends the stream with an error chunk, and a guardrail without canInterruptOthers: true has its BLOCK and SANITIZE downgraded to FLAG. A SANITIZE of a FINAL_RESPONSE rewrites the reply wherever the chunk carries it, as an output guardrail's does: finalResponseText and the reply in updatedConversationContext read modifiedText; finalResponseTextPlain reads it rendered as plain text, as a turn renders its own reply; audioOutput keeps its voice settings, speaks that plain text and drops the url of the audio made from the replaced reply; and a structured value parsed from the replaced reply is withdrawn (structured.value is null, meta.valid is false), since the wrapper holds no schema to check the rewrite against. A modifiedText equal to the reply changes nothing. The wrapper rewrites the stream only: the source agent's runtime saved its conversation before the chunk reached the wrapper, and the stored reply stays as the model wrote it. A rewrite that must reach the stored conversation runs as one of the agent's own output guardrails, whose rewrite the runtime stores (see "The stored reply" below). This is useful for:

• Supervisor patterns in multi-agent systems
• Quality gates across an agency
• Organization-wide policy enforcement

Supervisor Pattern

import {
  ICrossAgentGuardrailService,
  GuardrailAction,
  type CrossAgentOutputPayload,
  type GuardrailEvaluationResult,
} from '@framers/agentos/safety/guardrails';

class SupervisorGuardrail implements ICrossAgentGuardrailService {
  // Observe specific worker agents (empty = all agents)
  observeAgentIds = ['worker-analyst', 'worker-writer'];

  // Allow this guardrail to block/modify other agents' streams
  canInterruptOthers = true;

  // Evaluate streaming chunks in real-time
  config = { evaluateStreamingChunks: true };

  async evaluateCrossAgentOutput({
    sourceAgentId,
    chunk,
    context,
  }: CrossAgentOutputPayload): Promise<GuardrailEvaluationResult | null> {
    // Check for confidential information leakage
    if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA && chunk.textDelta?.includes('CONFIDENTIAL')) {
      return {
        action: GuardrailAction.BLOCK,
        reason: `Agent ${sourceAgentId} attempted to expose confidential information`,
        reasonCode: 'CROSS_AGENT_CONFIDENTIAL_LEAK',
        metadata: {
          blockedAgent: sourceAgentId,
          supervisor: 'supervisor-agent'
        },
      };
    }

    return null;
  }
}

Quality Gate Guardrail

class QualityGateGuardrail implements ICrossAgentGuardrailService {
  observeAgentIds = []; // Observe all agents
  canInterruptOthers = true;

  async evaluateCrossAgentOutput({
    sourceAgentId,
    chunk,
  }: CrossAgentOutputPayload): Promise<GuardrailEvaluationResult | null> {
    // Only evaluate final responses
    if (chunk.type !== AgentOSResponseChunkType.FINAL_RESPONSE) {
      return null;
    }

    const response = chunk.finalResponseText;

    // Check response quality
    if (response && response.length < 50) {
      return {
        action: GuardrailAction.FLAG,
        reason: 'Response may be too brief',
        reasonCode: 'QUALITY_WARNING',
        metadata: {
          responseLength: response.length,
          agent: sourceAgentId
        },
      };
    }

    return null;
  }
}

Configuration Options

GuardrailConfig

Option	Type	Default	Description
evaluateStreamingChunks	boolean	false	Evaluate TEXT_DELTA chunks (real-time) vs only FINAL_RESPONSE
evaluateToolCalls	boolean	false	Evaluate tool calls: a block refuses a call the runtime runs itself before it runs, and keeps an actionable call from the host and ends the stream; a sanitize is applied as a block. See Tool Calls
maxStreamingEvaluations	number	undefined	Rate limit streaming evaluations per guarded stream (the turn's stream and each continuation stream count on their own)
canSanitize	boolean	false	Run this guardrail in Phase 1 so SANITIZE results chain deterministically
timeoutMs	number	undefined	Per-guardrail timeout. On timeout/error the dispatcher fails open for that guardrail, unless failClosed is set or the guard is required
failClosed	boolean	false	A throw or a timeout blocks instead of passing. Forced on for a required guard

Output Payload Extras

GuardrailOutputPayload includes ragSources?: RagRetrievedChunk[] for output-time grounding checks. This field is populated when the response was generated with RAG retrieval, and is what the grounding guard uses to compare claims against retrieved evidence.

Performance Considerations

Mode	Latency	Cost	Use Case
Final-only (default)	+1-500ms once	Low	Policy checks needing full context
Streaming	+1-500ms per chunk	High	Real-time PII redaction, immediate blocking

Required Guards, Hold Mode and Replacement Replies

A product whose safety rules are code names the guards it cannot run without, and the runtime holds them to it:

const agentos = await AgentOS.create({
  extensionManifest: { packs: [{ factory: () => createPhraseListPack(neverDo) }, { factory: () => createPhraseListPack(selfHarm) }] },
  requiredGuardrails: [
    { id: 'never-do', stages: ['output'], timeoutMs: 8_000 },
    { id: 'self-harm', stages: ['input', 'output'], timeoutMs: 8_000 },
  ],
});

• Boot. initialize() throws SYS_CONFIGURATION_ERROR (with missing and missingStage in its details) when a required id has no active guard, the guard is disabled by an override, or it does not implement a required stage.
• Every request. processRequest(), handleToolResults() and resumeExternalToolRequest() check again and answer one error chunk, SYS_GUARDRAIL_REQUIRED_MISSING, while a required guard is missing. Nothing reaches a provider.
• Posture. A required guard runs under its id, fail-closed (a throw, a timeout past timeoutMs, or an answer whose action is not a GuardrailAction blocks, as GUARDRAIL_ERROR or GUARDRAIL_MALFORMED), whatever its own config says.
• Hold mode. A guard required on output, or guardrailOutputMode: 'hold', holds every TEXT_DELTA until the final guards have judged the whole reply. Allowed or flagged, the deltas go out before the final chunk; blocked or sanitized, they are dropped. An actionable tool call closes the window: the text so far is judged as a final reply before the tool call goes out. The same guards run on the continuation after an external tool result.
• Replacement replies. A BLOCK whose evaluation carries a non-empty replacementText reaches the caller as a FINAL_RESPONSE holding that text in finalResponseText, and in finalResponseTextPlain rendered as plain text as a turn renders its own reply, with metadata.guardrail.output[0].action === 'block' and the guard's reasonCode, in place of an error chunk. A block without one (or with an empty string) yields the error chunk.
• The stored reply. With conversational persistence on, a reply a guard blocked with a replacement or sanitized is rewritten in the conversation's history (and one blocked without a replacement is emptied) before the final chunk leaves, so the history holds what the person saw: the stored message is matched by the text the guards judged, never by position, on a turn and on a tool continuation alike. The message's metadata.modificationInfo records the guard's reason code. A block on a streamed delta judges no whole reply and rewrites nothing; hold mode is where nothing streams before the final verdict. With appendOnlyPersistence on the conversation manager the stored row keeps the first text and the rewrite stays in memory; the runtime logs a warning when that happens, and a product that must not keep a replaced reply does not run an append-only store. A sanitized final chunk's updatedConversationContext reads the rewrite too.
• Declared stages. A guard that has both methods but runs on one stage declares it in IGuardrailService.stages (the phrase-list guard does, from its stages option); a required guard is held to the stages it declares, and a stage it does not declare is reported as missing instead of passing on the method's existence.
• The text before a tool call. In hold mode the text held before an actionable tool call is judged as a reply of its own, with the text in finalResponseText and as plain text in finalResponseTextPlain, as a final reply carries them: a block replaces it and the tool call does not go out; a sanitize rewrites what streams. That verdict is not reported through onVerdict, since nothing is stored at that point; the turn's final chunk, which carries the whole text, is judged and recorded on its own.
• Every verdict names its guard. metadata.guardrailId is set on each evaluation when the guard has an id; a block's error chunk carries it in details.metadata.

PhraseListGuardrail

A guard over a reviewed list of phrases. Each entry blocks on a match, or asks a judge whether the match, read in the text, crosses the rule. Text is normalised before matching (NFKC, case, zero-width characters and combining marks removed, Cyrillic and Greek look-alikes mapped to Latin, curly quotes straightened), so a lookalike letter or an invisible joiner does not get past the list.

import { PhraseListGuardrail, StaticPhraseListSource, createLlmPhraseJudge, createPhraseListPack } from '@framers/agentos';

const neverDo = await PhraseListGuardrail.create({
  id: 'never-do',
  stages: ['output'],
  source: new StaticPhraseListSource({
    version: '2026-10-08', reviewedAt: '2026-10-08', reviewedBy: 'counsel',
    entries: [
      { phrase: 'you will pass', match: 'word', onMatch: 'block', ruleId: 'outcome_promise' },
      { phrase: 'index fund', match: 'word', onMatch: 'judge', ruleId: 'money' },
    ],
  }),
  judge: createLlmPhraseJudge({ provider: 'openai', model: 'gpt-6-luna', criteria: 'Investment, tax or legal advice.', apiKey, fallbackProviders: [] }),
  replacementFor: (ruleId) => TEMPLATES[ruleId],
});

PhraseListGuardrail.create refuses an empty list, a list that does not load, and judge entries without a judge; reload() swaps the list in whole and keeps the last good one when the new one fails. The judge fails closed: a throw, an answer without block and confidence, or a confidence under judgeThreshold (default 0.7) blocks. createLlmPhraseJudge takes fallbackProviders: [] by default, so the judge reaches only the provider it was given.

phraseMatches(entries, text) runs the same match with no guard and no judge: the text goes through the same normalisation, the entries compile with the same word boundaries, and the entries that occur come back in list order. A judge entry comes back like any other; what a match means is the caller's to decide. It throws on an entry whose phrase normalises to nothing, as PhraseListGuardrail.create does.

import { phraseMatches, type PhraseEntry } from '@framers/agentos';

const entries: PhraseEntry[] = [
  { phrase: 'coming soon', match: 'word', onMatch: 'block', ruleId: 'timing' },
  { phrase: 'next', match: 'word', onMatch: 'judge', ruleId: 'timing' },
  { phrase: 'in\\s+beta', match: 'regex', onMatch: 'block', ruleId: 'stage' },
];

phraseMatches(entries, 'Exports are COMING SOON.').map((e) => e.phrase); // ['coming soon']
phraseMatches(entries, 'The nextcloud connector'); // []: a word entry keeps its word boundaries
phraseMatches(entries, 'Now in beta').map((e) => e.ruleId); // ['stage']

Hard limits in the persona

IPersonaDefinition.hardLimits (and hardLimits: in a SOUL file's front matter, when the file is loaded as a persona) renders as the last block of the system prompt of every GMI turn for that persona, under the heading "Hard limits", after everything the turn added and after every fragment of the persona's own prompt whatever priority it carries. The guards hold the same rules in code; the block tells the model.

A GMI built by agent({ runtime: 'gmi' }) or gmi() has no such block. Its persona comes from personaFromAgentOptions(), which sets no hardLimits, and the front matter of a soul file is not applied on that path (GMIs from agent()).

Tool Calls

An output guardrail can judge the tool calls a turn requests. With config.evaluateToolCalls: true, evaluateOutput receives each TOOL_CALL_REQUEST chunk of a guarded stream, the turn's and each continuation's, as the stream carries it: chunk.toolCalls lists each call's id, name and parsed arguments.

class ShellCommandGuard implements IGuardrailService {
  readonly id = 'shell-commands';
  readonly config: GuardrailConfig = { evaluateToolCalls: true };

  async evaluateOutput({ chunk }: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null> {
    if (chunk.type !== AgentOSResponseChunkType.TOOL_CALL_REQUEST) return null;
    for (const call of chunk.toolCalls) {
      if (call.name === 'shell_execute' && /rm\s+-rf\s+\//.test(String(call.arguments.command))) {
        return { action: GuardrailAction.BLOCK, reason: 'A destructive shell command', reasonCode: 'SHELL_DESTRUCTIVE' };
      }
    }
    return null;
  }
}

• Block. The stream ends with the guard's verdict in place of the chunk: the error chunk, or the final response holding replacementText. The stream never hands the host an actionable call (executionMode: 'external'), so the host does not execute it. In hold mode the text held before the call is dropped with it, and a block on that text withholds the call too. AgentOS drops the pause the runtime saved for the call when the turn stopped for it, before the verdict reaches the host. getPendingExternalToolRequest() no longer returns the call, the conversation answers it with a refused tool result (code AUTH_PERMISSION_DENIED) so the history stays well formed, and the stream is closed, so handleToolResult() cannot continue it. The stream is closed even when the conversation store refuses that save, and AgentOS logs a warning. The conversation in memory drops the pause all the same, and the conversation's next save (on its next turn, before it is evicted from memory, or at shutdown) stores it without the pause: a restart finds the pause stored only when no such save succeeded before the process stopped. A host that wraps a stream with wrapOutputGuardrails() itself reads the withheld calls in the verdict's toolCallIds (onVerdict). A block on a chunk that holds no actionable call, a streamed delta or the chunk of calls the runtime runs itself, withholds no call: it ends the host's stream while the turn goes on inside the runtime, and when the turn then stops for calls the host runs, AgentOS drops that pause in the same way once the turn ends. The host's stream does not wait for the rest of the turn. From the verdict until the pause is dropped, the pause is never offered: getPendingExternalToolRequest() returns null for it, resumeExternalToolRequest() refuses it with one error chunk (SYS_INVALID_STATE), and handleToolResult() on the blocked stream yields one error chunk (SYS_INVALID_STATE).
• Sanitize. A guard cannot rewrite a tool call. A SANITIZE is applied as a BLOCK that keeps the guard's reason and reason code.
• Flag and allow. The chunk goes on with the evaluation in metadata.guardrail.output.
• Calls the runtime runs itself. AgentOS asks the guard about each call with executionMode: 'internal' before the call runs, as a TOOL_CALL_REQUEST chunk holding that one call, with the sources the turn retrieved in ragSources, on processRequest(), handleToolResult() and resumeExternalToolRequest() alike. A continuation the orchestrator is given no gate for (AgentOSOrchestrator.orchestrateToolResult() takes none) is judged with the gate its turn ran with, which the runtime keeps with the turn's stream. A continuation retrieves nothing of its own, so its calls are judged with the sources the user turn it continues retrieved, which the runtime keeps with the turn's stream: a turn of another conversation on the session's GMI meanwhile, or a GMI built again for the session, does not change them. A turn resumed after a restart is judged without them: the pause stores the calls, never the documents the turn retrieved. A block (or a sanitize) refuses the call: the tool does not run, and the model reads a refused tool result (code AUTH_PERMISSION_DENIED, with the guard's reason), as it reads a permission refusal. When a model step asks for several such calls, every call of the step is judged before any of them runs, and the calls that pass alone are judged once more together, as the stream's chunk for the step holds them, so a guard that rules on calls together (notes read and a shell command run in one step) refuses each of them before any runs. On the stream processRequest() returns, the guard also judges the stream's chunk for the step, so it is asked about such calls twice; a block there ends the host's stream with the verdict while the turn finishes inside the runtime. A continuation's stream carries no chunk for the calls the GMI runs itself. A refused call never becomes an actionable request and no pending request is saved for it; the FINAL_RESPONSE lists it in finalToolCalls beside the calls that ran. To have a person approve a tool before it runs, gate the tool: with a hitlManager and toolOrchestratorConfig.hitl.enabled, the tool orchestrator asks for an approval before a tool with hasSideEffects: true runs (Human-in-the-Loop).
• Several guards. The guards that evaluate tool calls run in parallel on each tool-call chunk; the first block in registration order is the verdict.
• What the guards do not see. The guards judge the calls a turn's model requests. The steps of a composed tool (one forge_tool built from a chain of other tools) and the tool tasks of a workflow are not model requests: they run through the tool orchestrator's processToolCall() with no tool-call chunk, so a guard that sets evaluateToolCalls judges the composed tool's own call, with the arguments the model gave it, and never the steps that call runs, nor a workflow's tool tasks. Those calls meet the tool orchestrator's own checks: the disabled list, the permission check, argument validation and, with hitl.enabled, the approval a tool with hasSideEffects: true waits for.

Why it is opt-in. A guard written for text may read any chunk it is given as text: one that takes every non-final chunk for a TEXT_DELTA reads textDelta as undefined, and a fail-closed guard that throws on it would block every tool call. A guard that sets evaluateToolCalls is written to read tool calls, and a guard without it never receives one. A guard that has a tool-call branch already, as the code-safety pack's guard does, runs it once its config sets the flag.

Using Multiple Guardrails

Multiple guardrails are dispatched in two phases: sanitizers first, then parallel classifiers:

import { createPiiRedactionGuardrail } from '@framers/agentos-ext-pii-redaction';
import { createMLClassifierGuardrail } from '@framers/agentos-ext-ml-classifiers';
import { createTopicalityGuardrail } from '@framers/agentos-ext-topicality';
import { createCodeSafetyGuardrail } from '@framers/agentos-ext-code-safety';
import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';

const piiPack = createPiiRedactionGuardrail({
  redactionStyle: 'placeholder',
  confidenceThreshold: 0.5,
});

const mlPack = createMLClassifierGuardrail({
  categories: ['toxic', 'injection'],
});

const topicalityPack = createTopicalityGuardrail({
  allowedTopics: ['customer support', 'billing', 'product features'],
  blockedTopics: ['politics', 'violence', 'gambling'],
});

const codeSafetyPack = createCodeSafetyGuardrail();

const groundingPack = createGroundingGuardrail({
  contradictionAction: 'flag',
});

await agent.initialize({
  ...config,
  extensionManifest: {
    packs: [
      { factory: () => piiPack },
      { factory: () => mlPack },
      { factory: () => topicalityPack },
      { factory: () => codeSafetyPack },
      { factory: () => groundingPack },
    ],
  },
});

Evaluation Order:

1. Phase 1 sanitizers run first in registration order
2. If any sanitizer returns BLOCK, processing stops immediately
3. Sanitized text from Phase 1 becomes the input to all Phase 2 guardrails
4. Phase 2 guardrails run concurrently, and the worst action wins
5. SANITIZE returned from Phase 2 is downgraded to FLAG to preserve deterministic output

API Reference

IGuardrailService

interface IGuardrailService {
  id?: string;               // Names the guard in verdicts and in requiredGuardrails
  config?: GuardrailConfig;
  evaluateInput?(payload: GuardrailInputPayload): Promise<GuardrailEvaluationResult | null>;
  evaluateOutput?(payload: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null>;
}

ICrossAgentGuardrailService

interface ICrossAgentGuardrailService extends IGuardrailService {
  observeAgentIds?: string[];      // Agents to observe (empty = all)
  canInterruptOthers?: boolean;    // Can BLOCK/SANITIZE other agents
  evaluateCrossAgentOutput?(payload: CrossAgentOutputPayload): Promise<GuardrailEvaluationResult | null>;
}

GuardrailAction

enum GuardrailAction {
  ALLOW = 'allow',      // Pass unchanged
  FLAG = 'flag',        // Pass, record metadata
  SANITIZE = 'sanitize', // Replace content
  BLOCK = 'block',      // Reject/terminate
}

GuardrailEvaluationResult

interface GuardrailEvaluationResult {
  action: GuardrailAction;
  reason?: string;           // User-facing message
  reasonCode?: string;       // Machine-readable code
  metadata?: Record<string, unknown>;
  details?: unknown;         // Debugging detail, not shown to users
  modifiedText?: string | null;  // For SANITIZE action
  replacementText?: string;  // With BLOCK on output, a non-empty string is the reply sent in place of the error chunk
}

Shared Heavyweight Services

Extension packs that need expensive resources (NER models, ONNX classifiers, embedding functions, NLI pipelines) should load them through ExtensionLifecycleContext.services, which is an ISharedServiceRegistry. The extension manager provides a shared registry instance so one agent can reuse the same heavyweight dependency across multiple packs instead of loading it once per guardrail.

Best Practices

1. Start with final-only evaluation - Enable streaming only when real-time filtering is required
2. Use rate limiting - Set maxStreamingEvaluations to control costs
3. Be specific with reason codes - Use consistent, machine-readable codes for analytics
4. Log FLAG actions - Use FLAG for monitoring without blocking user experience
5. Test edge cases - Test with partial PII, edge cases in streaming chunks
6. Consider latency - Each streaming evaluation adds latency to user experience

Folder-Level Permissions

AgentOS has no folder-permission layer. Its guardrails see the user's input and the response chunks, and a guard that sets evaluateToolCalls sees each tool call's arguments, but AgentOS ships no path rules. Wunderland, the agent CLI built on AgentOS, adds one. Its SafeGuardrails class checks a tool call's file paths, and the paths it reads out of rm, cp, mv, cat, touch, mkdir, rmdir, chmod and chown commands, against glob rules with separate read and write flags before the tool runs. It writes each violation to ~/.wunderland/security/violations.log and can notify webhooks or an email address of high and critical ones. An agent's security.folderPermissions in its agent.config.json sets the rules; see Wunderland's guardrails documentation.

Related Documentation

• Architecture Overview
• Human-in-the-Loop
• Agent Communication
• Safety Primitives