Skip to main content

Interface: VisionPipelineConfig

Defined in: packages/agentos/src/io/vision/types.ts:345

Configuration for the VisionPipeline.

All fields are optional — the factory function createVisionPipeline auto-detects available providers and fills in sensible defaults.

Example​

const config: VisionPipelineConfig = {
strategy: 'progressive',
ocr: 'paddle',
handwriting: true,
documentAI: true,
embedding: true,
cloudProvider: 'openai',
cloudModel: 'gpt-4o',
confidenceThreshold: 0.8,
preprocessing: { grayscale: true, sharpen: true },
};

Properties​

cloudApiKey?​

optional cloudApiKey: string

Defined in: packages/agentos/src/io/vision/types.ts:403

API key for the cloud vision provider. When unset, the provider's key is read from its environment variable (OPENAI_API_KEY and so on), as generateText does. createVisionPipeline refuses a key, and a base URL, that comes with no cloudProvider and no cloudModel naming a provider (openai:gpt-4o): it would go to whichever provider the environment names.


cloudBaseUrl?​

optional cloudBaseUrl: string

Defined in: packages/agentos/src/io/vision/types.ts:412

Base URL for the cloud vision provider, such as a proxy or a local OpenAI-compatible server. When unset, the provider's default is used. Needs a named provider, as cloudApiKey does. An empty or whitespace-only value is unset: it needs no provider, and the provider's base-URL variable, then its default, apply as with none.


cloudFallbackProviders?​

optional cloudFallbackProviders: FallbackProviderEntry[]

Defined in: packages/agentos/src/io/vision/types.ts:421

The providers the cloud tier may try, in order, after cloudProvider fails: generateText's fallbackProviders. Default [], so the image goes to cloudProvider alone and never to another vendor whose key is in the environment. A listed provider uses its own key from the environment, never cloudApiKey or cloudBaseUrl.


cloudModel?​

optional cloudModel: string

Defined in: packages/agentos/src/io/vision/types.ts:393

Cloud model override. When unset, the provider's default vision model is used.

Example​

'gpt-4o', 'claude-sonnet-4-6', 'gemini-2.5-flash'

cloudProvider?​

optional cloudProvider: string

Defined in: packages/agentos/src/io/vision/types.ts:387

Cloud vision LLM provider name for Tier 3 fallback. Must match a provider known to generateText() (e.g. 'openai', 'anthropic', 'gemini'). When unset, cloud vision is disabled.


confidenceThreshold?​

optional confidenceThreshold: number

Defined in: packages/agentos/src/io/vision/types.ts:429

Minimum confidence to accept an OCR result without escalating to cloud. Only applies to 'progressive' strategy — if OCR confidence is below this threshold, the pipeline escalates to the next tier.

Default​

0.7

documentAI?​

optional documentAI: boolean

Defined in: packages/agentos/src/io/vision/types.ts:373

Enable document understanding via Florence-2 (@huggingface/transformers). Produces a DocumentLayout of the lines of text it reads, each with its box.

Default​

false

embedding?​

optional embedding: boolean

Defined in: packages/agentos/src/io/vision/types.ts:380

Enable CLIP image embeddings (@huggingface/transformers). Runs in parallel with other tiers — does not affect text extraction.

Default​

false

handwriting?​

optional handwriting: boolean

Defined in: packages/agentos/src/io/vision/types.ts:366

Enable handwriting recognition via TrOCR (@huggingface/transformers). Only triggered when OCR confidence is low and content appears handwritten.

Default​

false

ocr?​

optional ocr: "none" | "paddle" | "tesseract"

Defined in: packages/agentos/src/io/vision/types.ts:359

OCR engine for Tier 1 text extraction.

  • 'paddle' — PaddleOCR (via ppu-paddle-ocr). Best accuracy for most scripts.
  • 'tesseract' — Tesseract.js. Wider language support, slightly lower accuracy.
  • 'none' — Skip OCR entirely (useful for photo-only pipelines).

Default​

'paddle' (if installed), else 'tesseract' (if installed), else 'none'

preprocessing?​

optional preprocessing: VisionPreprocessingConfig

Defined in: packages/agentos/src/io/vision/types.ts:436

Image preprocessing options applied before any tier runs. Uses sharp for resizing, grayscale conversion, sharpening, and normalization.


strategy​

strategy: VisionStrategy

Defined in: packages/agentos/src/io/vision/types.ts:350

How to combine tiers.

Default​

'progressive'