Skip to main content

Interface: GenerateTextOptions

Defined in: packages/agentos/src/api/generateText.ts:317

Properties​

__fallbackDepth?​

optional __fallbackDepth: number

Defined in: packages/agentos/src/api/generateText.ts:608

Internal

Internal — DO NOT set from application code. Fallback-hop depth, threaded into the provider-fallback recursion so the leg's usage observer event carries fallbackDepth and hosts can tell leg traffic from primary traffic. Absent (0) on top-level calls.

__rootStartedAt?​

optional __rootStartedAt: number

Defined in: packages/agentos/src/api/generateText.ts:599

Internal

Internal — DO NOT set from application code. The outermost call's Date.now() start, threaded into the provider-fallback recursion so the winning hop's usage-observer durationMs reports true end-to-end wall-clock (spanning failed primary attempts) rather than only its own leg. Absent on a top-level call, where it defaults to that call's start.

_responseFormat?​

optional _responseFormat: Record<string, unknown> | { type: string; }

Defined in: packages/agentos/src/api/generateText.ts:678

Internal

Used by generateObject and AgentSession.send (with responseSchema) to forward a provider-specific response_format payload to the provider. Not part of the public API.

Shape varies by provider: OpenAI accepts json_object or json_schema, Anthropic uses an internal _agentosUseToolForStructuredOutput marker that AnthropicProvider routes to forced tool_use, Gemini uses a _gemini.responseSchema extra. The provider implementations consume whatever shape is here.

_responseFormatBuilder()?​

optional _responseFormatBuilder: (providerId, modelId) => Record<string, unknown> | undefined

Defined in: packages/agentos/src/api/generateText.ts:688

Internal

Rebuilds the provider-native structured-output payload for a FALLBACK leg's provider. When absent, legs receive _responseFormat verbatim (legacy behavior — provider-side guards then drop foreign shapes, so the leg runs with zero provider enforcement). generateObject and AgentSession.send supply this; hand-rolled generateText callers are unaffected. modelId is '' when the fallback entry omits model (the provider's default text model is resolved later).

Parameters​

providerId​

string

modelId​

string

Returns​

Record<string, unknown> | undefined

_transcriptIncludeTrailingCallerMessages?​

optional _transcriptIncludeTrailingCallerMessages: number

Defined in: packages/agentos/src/api/generateText.ts:699

INTERNAL (sessions): how many TRAILING entries of messages belong to THIS call's transcript delta rather than prior history. Sessions that carry a non-string user turn inside messages set 1 so the delta includes it; prompt-based calls leave it unset (the prompt push is captured positionally). Not part of the public API.

apiKey?​

optional apiKey: string

Defined in: packages/agentos/src/api/generateText.ts:498

Override the API key instead of reading from environment variables.

baseUrl?​

optional baseUrl: string

Defined in: packages/agentos/src/api/generateText.ts:500

Override the provider base URL (useful for local proxies or Ollama).

cache?​

optional cache: false | { ttl?: "5m" | "1h"; }

Defined in: packages/agentos/src/api/generateText.ts:453

Per-call prompt-cache control, forwarded to cache-capable providers (Anthropic today; others ignore it).

  • false — this request emits NO cache_control at all: the provider's automatic markers (request-level marker, thinking-mode block markers, the moving message-tail) are suppressed AND any caller-placed system/message markers (e.g. SystemContentBlock.cacheBreakpoint) are stripped before the wire. Set this on TRUE one-shots — a single never-repeated call pays the cache-write premium (1.25x/2x input) on bytes nothing will ever read back.
  • { ttl: '1h' } — the automatic markers (including the moving message-tail pinned for multi-turn history) carry a 1-hour TTL instead of the 5-minute default. Set this on slow loops: codegen orchestrator/tool steps gap 2-14 minutes and human-paced conversation turns regularly exceed 5 minutes, so the default-TTL entry expires between steps and every turn re-writes. Caller-placed markers keep their own TTLs (SystemContentBlock.cacheTtl).
  • omitted / { ttl: '5m' } — default 5-minute auto markers.

cacheDiagnostics?​

optional cacheDiagnostics: boolean | { previousMessageId?: string | null; }

Defined in: packages/agentos/src/api/generateText.ts:480

Enable Anthropic prompt-cache diagnostics (beta cache-diagnosis-2026-04-07) across the agentic loop. The loop auto-threads each step's response id into the next step's diagnostics.previous_message_id, so every step after the first carries a comparison verdict: cacheMissReason: null = prefix stable, a populated reason (system_changed / tools_changed / messages_changed / ...) = where the cached prefix diverged. Per-step verdicts surface on the GenerationHookResult.cacheDiagnostics hook field; the last step's verdict lands on GenerateTextResult.cacheDiagnostics. Anthropic-only (other providers ignore the option) and best-effort — diagnostics never block or fail a request.

The object form seeds the FIRST step's previous_message_id so a caller can thread across REQUESTS, not just across the steps of one call: pass the prior turn's GenerateTextResult.providerMessageId and the first step's verdict names any divergence from that turn's prompt. true keeps the in-call-only behavior (first step compares nothing).

chainOfThought?​

optional chainOfThought: string | boolean

Defined in: packages/agentos/src/api/generateText.ts:512

Chain-of-thought instruction prepended to the system prompt when tools are available. Encourages the model to reason explicitly before choosing an action.

  • false (default): no CoT injection.
  • true: inject the default CoT instruction.
  • string: inject a custom CoT instruction.

customModelParams?​

optional customModelParams: Record<string, unknown>

Defined in: packages/agentos/src/api/generateText.ts:496

Provider-specific TOP-LEVEL request-payload parameters, forwarded verbatim into ModelCompletionOptions.customModelParams. Provider implementations spread these onto the outgoing request body (OpenRouter / OpenAI / Anthropic / Ollama all honor it), so this is the escape hatch for params the typed options don't model — e.g. OpenRouter provider-routing preferences:

customModelParams: { provider: { sort: 'throughput' } }

Keys collide last-wins with the typed fields at the provider layer, so only pass params the target provider understands.

effort?​

optional effort: string

Defined in: packages/agentos/src/api/generateText.ts:413

Reasoning depth / token-spend control forwarded to effort-capable models (Opus 4.5+, Sonnet 4.6, Fable/Mythos 5) as output_config.effort (low|medium|high|xhigh|max). Independent of thinking and tool_choice; the provider drops it on unsupported models or invalid values.

fallbackProviders?​

optional fallbackProviders: FallbackProviderEntry[]

Defined in: packages/agentos/src/api/generateText.ts:572

Ordered list of fallback providers to try when the primary provider fails with a retryable error (HTTP 402/429/5xx, network errors, auth failures).

Default behavior (omit / undefined): auto-build the canonical fallback chain for the primary provider via buildFallbackChain, filtered to providers that have API keys present in the environment. No import needed: fallback is on by default.

Strict mode ([]): explicitly opt out of fallback. The primary provider's error is re-thrown after exhausting any provider-internal retries. Use this when billing isolation, capability auditing, or provider-pinned testing requires a single-provider guarantee.

Custom chain (array of entries): specify exactly which providers (and optional model overrides) to try, in order. Each entry's model defaults to the provider's text-generation default from PROVIDER_DEFAULTS when omitted. Providers are tried left-to-right; the first successful response wins.

Examples​

const result = await generateText({
  provider: 'anthropic',
  prompt: 'Hello',
});
// On retryable Anthropic failure, walks anthropic -> openai -> gemini -> ...
const result = await generateText({
  provider: 'anthropic',
  prompt: 'Hello',
  fallbackProviders: [],
});
const result = await generateText({
  provider: 'anthropic',
  prompt: 'Hello',
  fallbackProviders: [
    { provider: 'openai', model: 'gpt-4o-mini' },
    { provider: 'openrouter' },
  ],
});

frequencyPenalty?​

optional frequencyPenalty: number

Defined in: packages/agentos/src/api/generateText.ts:390

Frequency penalty forwarded to the provider (OpenAI / OpenRouter range -2..2). Reduces verbatim token repetition. Anthropic has no equivalent and the provider drops it, so it's a no-op on Claude models.

hostPolicy?​

optional hostPolicy: HostLLMPolicy

Defined in: packages/agentos/src/api/generateText.ts:625

Host-level routing hints that can be forwarded into the model router without requiring callers to construct raw router params directly.

maxSteps?​

optional maxSteps: number

Defined in: packages/agentos/src/api/generateText.ts:370

Maximum number of agentic steps (LLM calls) to execute before returning. Each tool-call round trip counts as one step. Defaults to 1.

maxTokens?​

optional maxTokens: number

Defined in: packages/agentos/src/api/generateText.ts:398

Hard cap on output tokens. Provider-dependent default applies when omitted.

messages?​

optional messages: Message[]

Defined in: packages/agentos/src/api/generateText.ts:338

Full conversation history. Appended before prompt when both are supplied.

model?​

optional model: string

Defined in: packages/agentos/src/api/generateText.ts:332

Model identifier. Accepted in two formats:

  • Plain model name (e.g. "gpt-4o") when provider is also set. Preferred.
  • "provider:model" combined string (e.g. "openai:gpt-4o").

Either provider or model (or an API key env var for auto-detection) is required.

onAfterGeneration()?​

optional onAfterGeneration: (result) => Promise<void | GenerationHookResult>

Defined in: packages/agentos/src/api/generateText.ts:661

Called after each LLM generation step. Can check output against guardrails, redact PII, or transform the response. Return a modified result to transform output, or void to pass through.

Parameters​

result​

GenerationHookResult

Returns​

Promise<void | GenerationHookResult>

onBeforeGeneration()?​

optional onBeforeGeneration: (context) => Promise<void | GenerationHookContext>

Defined in: packages/agentos/src/api/generateText.ts:655

Called before each LLM generation step. Can inject memory context into messages, sanitize input via guardrails, or modify the prompt. Return a modified context to transform input, or void to pass through.

Parameters​

context​

GenerationHookContext

Returns​

Promise<void | GenerationHookContext>

onBeforeToolExecution()?​

optional onBeforeToolExecution: (info) => Promise<ToolCallHookInfo | null>

Defined in: packages/agentos/src/api/generateText.ts:666

Called before each tool execution. Can modify arguments, apply permission checks, or return null to skip the tool call entirely.

Parameters​

info​

ToolCallHookInfo

Returns​

Promise<ToolCallHookInfo | null>

onFallback()?​

optional onFallback: (error, fallbackProvider) => void

Defined in: packages/agentos/src/api/generateText.ts:580

Callback invoked when a fallback provider is about to be tried after the primary (or a previous fallback) failed. Useful for logging or metrics.

Parameters​

error​

Error

The error that triggered the fallback.

fallbackProvider​

string

The provider identifier being tried next.

Returns​

void

planning?​

optional planning: boolean | PlanningConfig

Defined in: packages/agentos/src/api/generateText.ts:521

Enable plan-then-execute mode. When true (or a PlanningConfig), an upfront LLM call decomposes the task into numbered steps before the tool-calling loop begins. The plan is injected into the system prompt so the model executes with full awareness of the strategy.

Set to false or omit to skip planning entirely (the default).

policyTier?​

optional policyTier: "safe" | "standard" | "mature" | "private-adult"

Defined in: packages/agentos/src/api/generateText.ts:649

Caller's intended content policy tier. When set to 'mature' or 'private-adult' AND no explicit fallbackProviders was supplied, the auto-built fallback chain is constructed via buildPolicyAwareFallbackChain instead of the default availability chain: prepending an uncensored OpenRouter model (Hermes 3 405B) so a content-policy refusal from the primary (gpt-4o, Claude, etc.) re-routes to a model that can complete the request rather than hard-failing.

Combined with the isContentPolicyRefusal branch in isRetryableError, this also makes the existing fallback loop fire on OpenAI's 400 + code: 'content_policy_violation' : which the network-only retryable matrix would otherwise treat as a hard error.

Has no effect for safe/standard tiers (or when omitted): those keep the existing availability-only fallback behavior.

Mirrors the existing policyTier parameter on import('./generateImage.js').GenerateImageOptions and import('./editImage.js').EditImageOptions.

presencePenalty?​

optional presencePenalty: number

Defined in: packages/agentos/src/api/generateText.ts:396

Presence penalty forwarded to the provider (OpenAI / OpenRouter range -2..2). Nudges the model toward new topics. Anthropic has no equivalent and the provider drops it, so it's a no-op on Claude models.

prompt?​

optional prompt: string

Defined in: packages/agentos/src/api/generateText.ts:334

Single user turn to append after any messages. Convenience alternative to building a messages array.

promptCacheKey?​

optional promptCacheKey: string | false

Defined in: packages/agentos/src/api/generateText.ts:420

OpenAI prompt-cache shard key (spec batch-1 C2; other providers ignore it). 'auto' derives a sha256-hashed key from sessionId (omitted when no session id is set; raw ids never leave the process); an explicit string is sent verbatim; false/absent omit the field.

promptCacheRetention?​

optional promptCacheRetention: "in_memory" | "24h" | "30m"

Defined in: packages/agentos/src/api/generateText.ts:426

OpenAI prompt-cache retention request. Emitted only when the fail-closed capability table allows the model/value combination; unsupported combos are omitted with a debug log. See openai-cache-params.ts.

provider?​

optional provider: string

Defined in: packages/agentos/src/api/generateText.ts:324

Provider name. When supplied without model, the default text model for the provider is resolved automatically from the built-in defaults registry.

Example​

`"openai"`, `"anthropic"`, `"ollama"`

requestTimeout?​

optional requestTimeout: number

Defined in: packages/agentos/src/api/generateText.ts:365

Per-call request timeout in milliseconds, forwarded to the provider for this call only. Large-output callers (e.g. structured-output generation that emits long strings) can raise the abort window without slowing the provider's default failover for chat or narration traffic. Native path only; providers without a request timeout ignore it.

router?​

optional router: IModelRouter

Defined in: packages/agentos/src/api/generateText.ts:615

Optional model router for intelligent provider/model selection. When provided, the router's selectModel() is called before provider resolution. The router result overrides model/provider. If the router returns null, falls back to standard resolution.

routerParams?​

optional routerParams: Partial<ModelRouteParams>

Defined in: packages/agentos/src/api/generateText.ts:620

Routing hints passed to the model router. Extracted automatically from system prompt and tool names when not provided.

serviceTier?​

optional serviceTier: "priority" | "auto" | "default" | "flex"

Defined in: packages/agentos/src/api/generateText.ts:432

OpenAI service tier (service_tier, verbatim). No default. 'flex' bills at ~batch rates but can 429 under load; automatic tier fallback is deliberately not implemented.

sessionId?​

optional sessionId: string

Defined in: packages/agentos/src/api/generateText.ts:461

Per-conversation affinity key, forwarded to providers that support request affinity (OpenRouter sends it as session_id for provider sticky routing — upstream prompt caches are host-scoped, so a load-balanced conversation otherwise cold-misses the cache a prior turn wrote on a different host). Pass a stable id per conversation.

source?​

optional source: string

Defined in: packages/agentos/src/api/generateText.ts:589

Optional source label forwarded to the global LLM usage observer registered via setGlobalLlmObserver. Hosts use this to tag the emitted telemetry row with a caller-defined meter key (e.g. 'narrator_turn', 'companion_reply', 'world_compile_job').

Has no effect when no observer is registered.

system?​

optional system: string | SystemContentBlock[]

Defined in: packages/agentos/src/api/generateText.ts:336

System prompt injected as the first message. Accepts a plain string or structured blocks with cache breakpoints.

temperature?​

optional temperature: number

Defined in: packages/agentos/src/api/generateText.ts:382

Sampling temperature forwarded to the provider (0-2 for most providers).

thinking?​

optional thinking: object

Defined in: packages/agentos/src/api/generateText.ts:406

Extended-thinking switch forwarded to thinking-capable models (Opus 4.7/4.8). Any positive budgetTokens enables adaptive thinking — the only form this family accepts; the number itself is not sent and maxTokens passes through unchanged. Omitted = thinking off (provider default behavior). Has no effect on models that do not support thinking.

budgetTokens​

budgetTokens: number

toolChoice?​

optional toolChoice: string | Record<string, unknown>

Defined in: packages/agentos/src/api/generateText.ts:357

Provider tool_choice passthrough. Forwarded verbatim to the provider so callers can force a specific tool, force tool use, or set 'auto'. Provider support varies; Anthropic + OpenAI honor it. Native path only (ignored on the prompt-emulation shim).

toolMode?​

optional toolMode: ToolMode

Defined in: packages/agentos/src/api/generateText.ts:380

Tool-calling strategy. 'auto' (default) uses native provider tool-calling, and on a tool-unsupported provider error falls back to a prompt-based shim (tool schemas rendered into the prompt, <tool_call> blocks parsed from the model's text). 'native' forces native only. 'prompt' forces the shim. The shim makes AgentOS tools work on models without native tool-use (e.g. the uncensored OpenRouter catalog). Shim roundtrips are capped by maxSteps (default 5 when unset on the shim path).

tools?​

optional tools: AdaptableToolInput

Defined in: packages/agentos/src/api/generateText.ts:350

Tools the model may invoke.

Accepted forms:

  • named high-level tool maps
  • external tool registries (Record, Map, or iterable)
  • prompt-only ToolDefinitionForLLM[]

Prompt-only definitions are visible to the model but return an explicit tool error if the model invokes them without an executor.

topP?​

optional topP: number

Defined in: packages/agentos/src/api/generateText.ts:384

Nucleus sampling top-p forwarded to the provider.

usageLedger?​

optional usageLedger: AgentOSUsageLedgerOptions

Defined in: packages/agentos/src/api/generateText.ts:502

Optional durable usage ledger configuration for helper-level accounting.