Document Ingestion

Memory.ingest() turns files, folders and URLs into memory traces: it picks a loader by file extension, extracts the text, splits it into chunks, and stores each chunk as a trace in the brain with a full-text index entry.

---

Overview

⊕ Click to expand

Quick Start

import { Memory } from '@framers/agentos';

const mem = await Memory.createSqlite('./brain.sqlite', {
  ingestion: { chunkStrategy: 'fixed', chunkSize: 512, chunkOverlap: 64 },
});

// One file
await mem.ingest('./report.pdf');

// A folder, with glob filters relative to it
await mem.ingest('./docs', {
  include: ['**/*.md', '**/*.pdf'],
  exclude: ['**/node_modules/**'],
});

// A URL
await mem.ingest('https://example.com/api-docs');

await mem.close();

ingest() decides what the source is in this order: an existing folder, an existing file, an http:// or https:// URL. Anything else lands in result.failed.

---

Supported File Types

Format	Extensions	Loader	What it does
PDF	.pdf	PdfLoader	Text extraction with fallbacks (see below)
DOCX	.docx	DocxLoader	mammoth.extractRawText(); replaced by the Docling loader when Docling is available
HTML	.html, .htm	HtmlLoader	Removes <script> and <style> blocks and tags and keeps the text, in time linear in the page's length, malformed markup included. Decodes the common named entities and every numeric one, each once (&amp;lt; is the text &lt;), and reads a numeric one that names no character (0, a surrogate, past U+10FFFF) as U+FFFD, as a browser shows it
Markdown	.md, .mdx	MarkdownLoader	Parses and strips YAML front matter (gray-matter), read as data: a block marked as JavaScript (---js) is dropped and never evaluated; the title comes from the front matter or the first heading
Text	.txt, .csv, .tsv, .json, .yaml, .yml	TextLoader	Reads the file as UTF-8 text, unparsed; format names the extension
URL	http://, https://	UrlLoader	Fetches the URL and routes by content type

LoaderRegistry maps extensions to loaders. A folder scan considers only files whose extension a loader claims. registry.register(loader) adds a loader for its supportedExtensions, replacing an earlier one for the same extension.

PowerPoint, Excel, OpenDocument, RTF and EPUB files are read from their bytes by OfficeLoader, which the registry does not register (below).

Reading through the global fetch, UrlLoader throws on a non-2xx response. A text/html response goes to the HTML loader, application/pdf to the PDF loader, and every other content type, Markdown included, is kept as raw text with format: 'text'.

An address a person gives a server

import { guardedFetch, GuardedFetchError, HtmlLoader, PdfLoader, UrlLoader } from '@framers/agentos/cognition/memory';

const loaders = {
  getLoader: (extension: string) =>
    extension === '.html' ? new HtmlLoader() : extension === '.pdf' ? new PdfLoader() : undefined,
};
const loader = new UrlLoader(loaders, {
  fetchDocument: (url) => guardedFetch(url, { maxBytes: 2 * 1024 * 1024, deadlineMs: 10_000 }),
});

try {
  const doc = await loader.load(addressFromTheForm);
} catch (error) {
  if (error instanceof GuardedFetchError) {
    // error.reason: 'address', 'host', 'scheme', 'port', 'redirects', 'status', 'type', 'size', 'deadline' or 'network'
  }
}

A server that fetches an address someone typed can be pointed at itself or at the network it sits on: http://169.254.169.254/ is where cloud metadata services answer, and a name can resolve to 127.0.0.1. guardedFetch reads such an address under these rules, and UrlLoader takes it as fetchDocument:

• The host is resolved first, and the fetch is refused (reason 'address') when any address it resolves to is not public: loopback, private, carrier-grade NAT, link-local, unique-local, multicast, documentation, reserved or unspecified, in IPv4 and IPv6. An IPv4-mapped address (::ffff:0:0/96), one under the well-known NAT64 prefix (64:ff9b::/96) and a 6to4 one (2002::/16) are judged by the IPv4 address each carries; the deprecated IPv4-compatible form (::/96) and the rest of 64:ff9b::/32 are refused whatever they carry. isPublicAddress(address) is that rule. An IPv6 address is also read for an IPv4 one under a NAT64 prefix of this network, which the resolver reveals for ipv4only.arpa (RFC 7050), and refused when that IPv4 address is not public. Such a prefix only adds refusals: an address its own range refuses stays refused whatever it carries. When the lookup of ipv4only.arpa fails, a host with an IPv6 address is refused ('network'). allowAddresses exempts the addresses it lists, for a test's own server.
• The connection goes to an address that was checked, on a connection of its own, so a second lookup cannot send it elsewhere. When the host has several, they are tried in turn, alternating IPv6 and IPv4 (Node's autoSelectFamily), each given two seconds to connect, so a host whose first address cannot be reached from this network is read at the next. An https: connection checks the certificate against the host's name.
• Only http: and https: addresses without a user name or password are read ('scheme'), on ports 80 and 443 unless allowPorts names others ('port').
• allowHost, when given, is a rule over the host, (host: string) => boolean, asked about the host of the address and about the host of every redirect's target, after the scheme check and before the port check and the lookup. It sees the host alone, as the URL parser writes it (URL Standard, host parsing): lower case, without a port, without the brackets around an IPv6 address, without a trailing dot (http://DOCS.GOOGLE.COM.:8080/x reaches it as docs.google.com). An international name reaches it in its ASCII form (https://bücher.example/ as xn--bcher-kva.example), an IPv4 address in dotted decimal however the address wrote it, an IPv4-mapped IPv6 address and one under the well-known NAT64 prefix 64:ff9b::/96 (RFC 6052) included, since a connection to either goes to the IPv4 address it carries, through a NAT64 gateway for the second (http://0x5db8d70e/, http://[::ffff:93.184.215.14]/ and http://[64:ff9b::93.184.215.14]/ as 93.184.215.14), and any other IPv6 address in compressed hex (http://[2606:4700:4700:0:0:0:0:1111]/ as 2606:4700:4700::1111). A subdomain is a host of its own, so a rule for a site and its subdomains compares on a dot boundary, host === 'example.com' || host.endsWith('.example.com'); host.endsWith('example.com') takes notexample.com too. The rule judges the host the address names, before any lookup. A name is not resolved for it, so a rule that refuses an address does not refuse a name that resolves to that address. The rule never reads the NAT64 prefixes the fetch learns from the resolver (RFC 7050), at the first address or at a redirect, so an address under a prefix of the network's own reaches it in hex, and a rule that refuses an IPv4 address does not refuse it under such a prefix either. A 6to4 address (2002::/16, which RFC 7526 keeps for unicast) reaches it in hex too (http://[2002:5db8:d70e::5db8:d70e]/ as 2002:5db8:d70e::5db8:d70e). The IPv4 address it carries is the address of the 6to4 router of a site (RFC 3056, sections 2, 5.1 and 5.3), and the connection ends at a host of that site, so a rule that refuses 93.184.215.14 does not refuse that address, and one that allows 93.184.215.14 does not allow it. Handing the rule the router's address would let an allow-list of it reach every host of the site. The address check judges every address the host resolves to by its own range, and an IPv4-mapped address, one under the well-known NAT64 prefix or a 6to4 one by the IPv4 address it carries. A prefix the resolver reveals only adds refusals, so an address its own range refuses, such as one under the local-use prefix 64:ff9b:1::/48 (RFC 8215), stays refused whatever it carries. The fetch is refused ('host') unless the rule answers true, so a refused host is never resolved or connected to; a rule that throws refuses too, with what it threw as the error's cause and a message that keeps its details out, and a value that is not a function is a TypeError before any lookup. As on a native error, cause is not enumerable, so Object.keys, JSON.stringify and a spread of the error leave it out too. Error serializers and util.inspect read it by name, as they read a native error's. Pino's default err serializer folds any cause with a string message, an error or a plain object such as { message: 'blocklist at 10.0.0.53 timed out' }, into the logged message, and the cause's stack, when it has one, into the logged stack. console.error prints any cause as [cause]: Node's util.inspect adds an own cause to the keys it prints and writes a key that is not enumerable in brackets. So a host that logs these errors decides what its rule throws. The rule is asked under the deadline and the caller's signal like every other step: its time counts against deadlineMs, read from the clock once the rule returns since a rule runs synchronously and holds the deadline's timer, so a rule that outlasts the deadline leaves the read refused 'deadline' with no lookup started and no connection begun; a rule that aborts the signal ends the read with that signal's reason the same way. A host the rule allows is still resolved and checked as every host is: the rule narrows what the fetch reads and never widens it. A server that reads a link a person pasted uses it to keep a provider's own site from being fetched as a page, a redirect there included, when that provider has a connector of its own.
• Up to maxRedirects redirects are followed (3 unless set), each checked as the first address is ('redirects' past them).
• The body is read as it arrives and refused once it passes maxBytes, counted on the wire and again after a gzip, deflate or br coding is decoded ('size'): a small body that inflates past the cap stops there, and so does a large one that decodes to little. A body in another coding is refused ('type').
• Only the media types in accept are read ('type'): text/html, application/xhtml+xml, application/pdf and text/plain unless set. An answer with no Content-Type counts as application/octet-stream (RFC 9110, section 8.3), which that default list does not hold. An answer that is neither a 2xx nor a redirect (301, 302, 303, 307 or 308) is refused ('status').
• One deadline, deadlineMs, covers the host rule, the lookups, the redirects and the body ('deadline'). An abort of signal ends the fetch with the signal's own reason.

Callers that show a refusal reason to a visitor should show one message for both 'address' and 'network' so the visitor cannot distinguish internal names that resolve from names whose lookup fails.

With fetchDocument, the answer's media type picks the loader: HTML and XHTML go to the .html loader, PDF to the .pdf loader, anything else is read as UTF-8 text, and metadata.source is the last address after redirects. UrlLoader takes its loaders from any object with getLoader(extension), so a server builds no LoaderRegistry, whose constructor probes for Docling. When that object has no loader for an HTML, XHTML or PDF answer of fetchDocument, load() throws an error that names the media type and not the address, so such an answer is never returned as raw markup.

Word files and what they inflate to

import { DocxLoader, DocumentTooLargeError } from '@framers/agentos/cognition/memory';

const loader = new DocxLoader({ maxInflatedBytes: 64 * 1024 * 1024 });
try {
  const doc = await loader.load(uploadedBuffer);
} catch (error) {
  if (error instanceof DocumentTooLargeError) {
    // error.limit is the bound; mammoth never read the file
  }
}

A .docx file is a ZIP archive, and mammoth inflates every part it reads, so a file of a few megabytes can inflate to gigabytes. Before mammoth reads a file, DocxLoader inflates each entry of the archive with node:zlib under maxInflatedBytes, one entry's output at a time, and throws DocumentTooLargeError (code 'DOCUMENT_TOO_LARGE', limit the bound) once the entries pass it together. The size an entry states for itself is not trusted. The bound is 134,217,728 bytes (128 MiB) when maxInflatedBytes is left out, which is how LoaderRegistry registers the loader; registry.register(new DocxLoader({ maxInflatedBytes })) replaces it. When the Docling loader replaces DocxLoader for .docx, this bound does not apply.

The bound is on what the archive inflates to, not on the memory a read takes. Every entry counts toward it, images and other media included, though mammoth.extractRawText() reads only the XML parts. mammoth parses each XML part it reads into an element tree, which takes a multiple of the part's size, so for files from people you do not know, set maxInflatedBytes well below the memory the process can spare, or read them with loadIsolated (next section).

A file the loader cannot read as a ZIP archive the way mammoth's reader does is refused with DocxLoader: not a Word archive before mammoth runs: no end of central directory record, a ZIP64 record, a central directory that does not end where the end record starts (bytes between the two, or bytes in front of an archive whose offsets do not count them), an entry outside the file, entries that share their data (their compressed data together larger than the bytes before the central directory, where every entry's data lies apart), a compression method other than stored or deflated, or data that does not inflate.

A file from someone you do not know

import {
  loadIsolated,
  DocumentTooComplexError,
  DocumentReadTimeoutError,
  DocumentTooLargeError,
} from '@framers/agentos/cognition/memory';

try {
  const doc = await loadIsolated('docx', upload, {
    maxHeapMb: 256,
    timeoutMs: 30_000,
    maxExternalMb: 128,
    maxInflatedBytes: 64 * 1024 * 1024,
  });
} catch (error) {
  if (error instanceof DocumentTooComplexError) {
    // error.resource is 'heap' or 'external'; error.limitMb is the bound the read passed
  } else if (error instanceof DocumentReadTimeoutError) {
    // error.timeoutMs is the deadline
  } else if (error instanceof DocumentTooLargeError) {
    // error.limit is the Word loader's bound
  } else {
    // the loader's own error, with its name and message
  }
}

loadIsolated(kind, bytes, options) reads with the loader of its kind in a node:worker_threads worker: 'pdf' with PdfLoader, 'docx' with DocxLoader, 'text' with TextLoader, 'markdown' with MarkdownLoader, each office format ('pptx', 'xlsx', 'odt', 'ods', 'odp', 'rtf', 'epub') with OfficeLoader, and 'scan', a scanned PDF read by OCR, with OfficeLoader too (below). It answers the content, metadata and format that loader answers. It copies the bytes into a buffer of its own before it moves them to the worker, so the caller's buffer stays usable. The answer is copied into the caller's heap, so the caller holds whatever text the worker read within its bounds. It refuses a read in six ways:

Refusal	When	What it carries
DocumentTooComplexError, code 'DOCUMENT_TOO_COMPLEX'	The worker reaches its heap cap (resource 'heap'), or its memory outside the heap passes maxExternalMb (resource 'external')	limitMb, the bound the read passed
DocumentReadTimeoutError, code 'DOCUMENT_READ_TIMEOUT'	The read passes timeoutMs	timeoutMs
DocumentTooLargeError, code 'DOCUMENT_TOO_LARGE'	A Word file's entries inflate past maxInflatedBytes, which goes to DocxLoader (its default when left out), or an office document or a scan reaches one of OfficeLoader's bounds	limit and the loader's message; from OfficeLoader, a cause that names the bound by officeparser's code
ScannedDocumentError, code 'SCANNED_DOCUMENT'	A scan's pages gave no text	The first failure officeparser reported for a picture OCR could not read, as its cause, when it reported one
The signal's reason	The caller aborts signal	Whatever the signal was aborted with
The loader's own error	The loader throws anything else, such as DocxLoader: not a Word archive	An Error with the loader's error's name and message

Any other failure of the worker rejects with the worker's own error (for a bundle that leaves out the worker's entry file, Node's Cannot find module error; for a scan, the error tesseract.js throws outside officeparser's promise), and a worker that stops with neither an answer nor an error rejects with an Error that names its exit code.

On every path, an answer included, the worker is ended before the call settles. A kind outside the twelve, bytes that are not a Uint8Array, or one of OfficeLoader's options given to a kind that does not take it reject with a TypeError. A bound that is not a positive, finite number, a timeoutMs past 2,147,483,647, 'scan' without an ocrPages of 1 or more, or an OfficeLoader option that loader refuses rejects with a RangeError. A signal already aborted rejects with its reason. Each of these is refused before any worker starts.

What each bound holds, and what it leaves:

• maxHeapMb (default 256) caps the worker's old generation through resourceLimits.maxOldGenerationSizeMb. A read that reaches the cap ends the worker, and the process that called loadIsolated reads on. --max-old-space-size overrides the cap, as Node documents for resourceLimits, wherever the process was given it: on its command line or in NODE_OPTIONS.
• timeoutMs (default 30,000, at most 2,147,483,647, the longest delay a Node timer holds) counts from the worker's start, the loading of the loader included.
• maxXmlElements and maxTableCells go to OfficeLoader for the office kinds and 'scan', and ocrPages and pdfProcessMemoryMb for 'scan' alone; each holds a read as it does in the caller's thread (below).
• signal ends the worker when the caller aborts it, and the call rejects with the signal's reason.
• maxExternalMb bounds what the heap cap does not count. Node's resource limits leave out array buffers, and the loaders hold large ones: JSZip, which mammoth reads a Word file with, keeps a part's inflated chunks and then their concatenation, and pdf.js keeps every decoded byte of a stream. With maxExternalMb, the worker's external_memory (its array buffers and external strings) is read with worker.getHeapStatistics() every 20 ms, and the worker is ended once it passes the bound. The check samples and does not cap: it runs on a timer on the caller's event loop, so what the worker allocates between two checks can pass the bound before the worker is ended (a busy caller's thread delays the next check), and a read that ends between two checks is answered whatever it held at its peak. Without maxExternalMb, nothing bounds that memory. worker.getHeapStatistics() comes with Node 22.16 and 24.0; on an earlier Node, a read given maxExternalMb is refused with a TypeError.

A read still shares its process: the caps end the worker, and a fault the worker cannot catch ends both. The worker is a thread of that process, with its access to files, the network and a copy of its environment variables: it bounds a read's memory and time, and it is not a sandbox for code. The worker's entry is isolatedLoadWorker.js, compiled beside loadIsolated.js in the package's dist. The package ships ES modules only, and loadIsolated finds the entry from its own URL (new URL('./isolatedLoadWorker.js', import.meta.url)), so a bundler that moves loadIsolated.js has to carry that file beside it. The worker is started with an empty execArgv in place of the Node options it would inherit from the caller's thread: an --input-type given with node -e would refuse its entry file, and the caller's --require and --import preloads would load inside the capped heap. NODE_OPTIONS from the environment applies to it as to any Node process. V8's options are the whole process's, whatever a worker's execArgv, so a --max-old-space-size the process was started with still overrides maxHeapMb.

PowerPoint, Excel, OpenDocument, RTF and EPUB files

import { OfficeLoader, DocumentTooLargeError } from '@framers/agentos/cognition/memory/office';

const loader = new OfficeLoader({
  maxInflatedBytes: 32 * 1024 * 1024,
  maxXmlElements: 400_000,
  maxTableCells: 200_000,
});
try {
  const doc = await loader.load(uploadedBuffer, { format: 'pptx', signal: AbortSignal.timeout(30_000) });
} catch (error) {
  if (error instanceof DocumentTooLargeError) {
    // the document reached a bound; error.cause names it by officeparser's code
  }
}

OfficeLoader reads .pptx, .xlsx, .odt, .ods, .odp, .rtf and .epub documents in memory through officeparser, an optional peer dependency: npm install officeparser (the peer range is ^8.1.1; officeparser 8.1.1 asks for Node.js 22.13 or later). The package is imported on the first read, and a read without it throws an error that starts OfficeLoader needs the optional peer dependency officeparser. LoaderRegistry does not register the loader, so Memory.ingest() does not read these formats: a host builds the loader and calls load itself.

The loader has an entry of its own, @framers/agentos/cognition/memory/office, which exports OfficeLoader, OFFICE_TYPES, ScannedDocumentError and DocumentTooLargeError, the same class DocxLoader throws. Importing the entry loads no package and no Node built-in: officeparser loads on the first read, loadIsolated on the first scan, and officeparser loads pdf.js and tesseract.js only to read a scanned PDF, so a host that never reads an office document loads none of the three. @framers/agentos/cognition/memory exports the same names. examples/office-documents.mjs reads an OpenDocument text and an EPUB book from that entry, reads the text again in a worker thread with loadIsolated, cancels one read with its own signal, refuses a document past a bound and a PDF while OCR is off, and reads the files named on its command line (node examples/office-documents.mjs deck.pptx).

load(bytes, { format, signal }) takes the file's bytes and its format's name: pptx, xlsx, odt, ods, odp, rtf or epub, with or without a leading dot (OFFICE_TYPES maps each extension to its name). It reads no file from a path and does not guess a format from the bytes, so a path throws, and so does a call that names none of the formats. content is the document's text as officeparser writes it, with comments and pictures left out; metadata carries wordCount, and pageCount when officeparser reports a page count.

officeparser inflates the parts of an archive it reads, builds an element tree from each XML part and a node for each table cell, so a small file can ask for far more memory than its size. The loader passes it three bounds:

Option	What it bounds	When left out
maxInflatedBytes	What the parts officeparser reads from an archive inflate to, together, counted as they inflate	67,108,864 bytes (64 MiB)
maxXmlElements	The XML elements read from a document's parts, an EPUB's chapters included. An .xlsx workbook's sheets are read without building elements, so their cells count toward maxTableCells alone; an .ods file's sheets are XML, and count toward both	officeparser's default (2,000,000 in 8.1.1)
maxTableCells	The cells of an .xlsx workbook's sheets, and of an OpenDocument file's sheets and tables; officeparser counts no other format's cells (a .pptx or .epub table's cells are XML elements, under maxXmlElements). officeparser allows a document one more cell for each byte of its file; the loader takes that allowance out of the limit it passes, so the option is the bound itself	officeparser's default (1,000,000 in 8.1.1, and one more for each byte of the file)

A document that reaches a bound is refused with DocumentTooLargeError, the error DocxLoader throws. So is a document past a bound officeparser only warns of and reads on from (a sheet's cells are one), and a document that reaches another of officeparser's own limits: an archive of more than 10,000 entries (in 8.1.1), the content a document repeats by reference, the grid its tables fill when written as text, and a PDF's content budgets. At the first bound a read reaches, the loader aborts the signal officeparser checks between its steps, so the read stops there and none of the document's text is written. The grid is the text writer's bound: the writer reports it to the parse's warning handler and lists it among its result's messages, and the loader refuses the document on either. The error's limit is the loader's maxInflatedBytes whichever bound was reached, and its cause names the bound by officeparser's code, such as XML_ELEMENT_LIMIT_EXCEEDED or TABLE_CELL_LIMIT_EXCEEDED. Its message names the same code, as in the document reached the bound officeparser reports as XML_ELEMENT_LIMIT_EXCEEDED; past maxInflatedBytes, it gives that bound in bytes, as the Word loader's message does: the document inflates past 33554432 bytes. An .rtf file is not an archive and holds no XML: none of the three bounds its read. A bound that is not a positive, finite number is refused with a RangeError when the loader is built.

signal, given to load, cancels that read and no other: load rejects with the signal's reason, and the loader reads on. officeparser checks the signal between its steps and ends OCR and the pdf.js process on it. A step that reads one part's markup finishes before an abort is seen, so a signal alone does not bound the time a document takes. A deadline is a signal made for the read, AbortSignal.timeout(ms).

For a file from someone you do not know, loadIsolated reads each office format in a worker thread under a heap cap and a deadline (above): loadIsolated('xlsx', bytes, { maxTableCells: 200_000 }) answers what the loader answers in the caller's thread, and passes its bound through as DocumentTooLargeError with the same message and cause.

A scanned PDF

const loader = new OfficeLoader({ ocrPages: 3, pdfProcessMemoryMb: 256 });
const doc = await loader.load(scannedBuffer, { format: 'pdf', signal: AbortSignal.timeout(120_000) });

With ocrPages at 0, the default, load(bytes, { format: 'pdf' }) throws ScannedDocumentError (code 'SCANNED_DOCUMENT'). With a whole number above 0, the loader reads the scan in a worker thread of its own, through loadIsolated's kind 'scan': officeparser reads the first ocrPages pages with pdf.js, and tesseract.js recognizes the text in each picture on them, in English. content holds that text with any text the pages themselves carry, and a scan whose pages give no text throws ScannedDocumentError.

• The worker thread has loadIsolated's heap cap, 256 MB of old generation, and no deadline of its own: the read's signal is its deadline, and an abort ends the worker. A read that fills the worker's heap rejects with DocumentTooComplexError.
• pdf.js runs in a Node process of its own, which officeparser starts with a heap of pdfProcessMemoryMb (256 MB when left out; 64 or more, since officeparser replaces a smaller heap with its own 1,024 MB). officeparser keeps the process for its thread's later reads; a scan's worker thread ends after its read, and the process exits when its channel to that thread closes.
• When pdf.js would read the scan in the reading thread instead, outside that heap (no process could start, officeparser found no pdf.js worker file on disk, or a worker was preloaded as globalThis.pdfjsWorker), the loader refuses the scan before pdf.js opens it, with an Error that names the reason.
• tesseract.js 7 fetches its English data from cdn.jsdelivr.net the first time a worker starts, and writes a copy, eng.traineddata, into the process's working directory, which later workers read. A failed write is not an error: the next worker fetches the data again.
• officeparser ends its OCR workers after ten seconds without a picture to read.
• officeparser gives an OCR worker 60 seconds to start and a recognition 30 seconds. A picture OCR could not read is left out of content, and the read goes on. When the pages give no text at all, the ScannedDocumentError carries the first failure officeparser reported as its cause.
• tesseract.js 7.0.0 throws a failure its worker reports (language data it could not fetch, or could not start from) from the worker's message listener unless it is given an error handler, and officeparser 8.1.1 gives it none. That error is thrown outside officeparser's promise: it ends the scan's worker thread, load rejects with it (initialization failed for language data tesseract.js cannot start from), and the caller's process reads on.
• Given officeparser's parseOffice as its parse option, the loader reads a scan in the thread that calls load, and such a failure is an uncaught exception of that thread: on the main thread, it ends a Node process that handles none.

An ocrPages that is not a whole number of 0 or more, and a pdfProcessMemoryMb under 64, are refused with a RangeError when the loader is built: officeparser reads a page range it cannot parse as every page.

---

PDF Extraction

The registry builds PdfLoader with two optional fallbacks, probed once when the registry is created:

Engine	Used when	Needs
Docling (python3 -m docling --output-format json <file>)	python3 -m docling --version succeeds; then every PDF and DOCX goes to Docling, and unpdf is not used	Python with docling installed
unpdf	No Docling	Nothing: a dependency of @framers/agentos
tesseract.js OCR	No Docling, tesseract.js resolves, and unpdf returns fewer than 50 characters per page on average	tesseract.js installed

No option switches these engines: the registry uses whatever it finds. The ingestion config fields ocrEnabled and doclingEnabled are declared and not read.

---

Chunking Strategies

ChunkingEngine splits the extracted text. ingest() reads chunkStrategy (default 'semantic'), chunkSize (default 512 characters) and chunkOverlap (default 64 characters) from the ingestion config the brain was opened with; the strategy cannot change per call.

Strategy	What it does
fixed	Cuts every chunkSize characters, moved back to the last whitespace so no word is split (a window with no whitespace is cut hard), and starts the next chunk chunkOverlap characters before the cut
semantic	With an embedding function, splits the text into sentences, embeds them, and starts a new chunk where the cosine similarity of two neighbouring sentences is below 0.3; a group longer than twice chunkSize is cut with fixed. Without one, it is fixed
hierarchical	Starts a section at every Markdown heading (# to ######); a section longer than chunkSize is cut with fixed; each chunk carries its heading, level and parent headings
layout	Keeps each fenced code block and each run of lines containing `

ingest() calls the engine without an embedding function, so through Memory the default semantic strategy runs as fixed. The brain stores each chunk's text and position; the heading and block-type metadata the engine attaches are not stored.

---

What a document becomes

For each loaded document, ingest():

1. Computes the SHA-256 of the extracted text. When any document in the brain already has that hash, the document is skipped: it stays in succeeded and adds no chunks.
2. Inserts a documents row (path, format, title, hash, chunk count, the loader's metadata).
3. For each chunk, inserts a memory_traces row (type: 'semantic', scope user, strength 1.0, no embedding), its memory_traces_fts entry, a graph node when the graph is on (the default), and a document_chunks row.

Chunk traces get no embedding, so recall() finds them through full-text search.

The skip compares content, not paths. A file whose text changed is stored again as a new document, and the traces of its earlier version stay in the brain. For a flat vector collection without the brain (doc citations, a help index), see Incremental Vector Ingestion.

---

Folders

FolderScanner walks the folder and loads every candidate file before any chunk is stored:

const result = await mem.ingest('./project/docs', {
  recursive: true,
  include: ['**/*.md', '**/*.pdf', '**/*.txt'],
  exclude: ['**/node_modules/**', '**/.git/**', '**/dist/**'],
  onProgress: (processed, total, current) => {
    console.log(`[${processed}/${total}] ${current}`);
  },
});

console.log(result.succeeded.length, result.failed.length, result.chunksCreated, result.tracesCreated);

• recursive defaults to true in ingest() (the IngestOptions comment says false).
• include and exclude are minimatch patterns (dot files included) matched against the path relative to the folder; a file must match an include pattern when there are any, and must match no exclude pattern.
• A folder path containing a .. segment is rejected, and the error lands in result.failed.
• A file that fails to load goes to failed and the scan goes on. onProgress(processed, total, current) fires after each file is loaded or fails.
• A file that loaded but failed to store appears in both succeeded and failed.
• IngestOptions.format is declared and not read.

interface IngestResult {
  succeeded: string[];                             // files (or the URL) that loaded
  failed: Array<{ path: string; error: string }>;  // load or store errors
  chunksCreated: number;
  tracesCreated: number;                           // one trace per chunk
}

---

Images

No loader extracts images, and ingest() writes nothing to document_images. The ingestion config fields extractImages and visionLlm are declared and not read.

MultimodalAggregator is a separate utility for hosts that extract images themselves: new MultimodalAggregator({ describeImage }) and processImages(images) add a caption to each ExtractedImage that lacks one, in parallel with Promise.allSettled; an image whose captioning fails keeps no caption, and without describeImage the images pass through unchanged. For image retrieval, see Multimodal RAG.

---

Configuration Reference

Read by ingest():

Where	Option	Default
ingestion config	chunkStrategy	'semantic' (runs as fixed through Memory)
ingestion config	chunkSize	512 characters
ingestion config	chunkOverlap	64 characters
ingest() options	recursive	true
ingest() options	include, exclude	none
ingest() options	onProgress	none

Declared and not read: ingestion.extractImages, ingestion.ocrEnabled, ingestion.doclingEnabled, ingestion.visionLlm, and the format option of ingest().

---

Source Files

File	Purpose
io/facade/Memory.ts	ingest() and the per-document storage
io/ingestion/LoaderRegistry.ts	Extension routing and the optional loaders
io/ingestion/PdfLoader.ts	unpdf with the OCR and Docling fallbacks
io/ingestion/OcrPdfLoader.ts	tesseract.js OCR
io/ingestion/DoclingLoader.ts	Python Docling subprocess
io/ingestion/OfficeLoader.ts	OfficeLoader, OFFICE_TYPES and ScannedDocumentError: the office formats through officeparser, and a scanned PDF by OCR; the entry @framers/agentos/cognition/memory/office
io/ingestion/loadOfficeparser.ts	The one module that imports officeparser, on the first read that needs it
io/ingestion/DocumentTooLargeError.ts	DocumentTooLargeError, which the Word and office loaders and loadIsolated throw, in a module that imports nothing
io/ingestion/loadIsolated.ts	loadIsolated, DocumentTooComplexError and DocumentReadTimeoutError
io/ingestion/isolatedLoadWorker.ts	The worker thread's entry: one read with the loader of its kind
io/ingestion/FolderScanner.ts	Folder walking and glob filters
io/ingestion/ChunkingEngine.ts	The four strategies
io/ingestion/UrlLoader.ts	URL fetching
io/ingestion/guardedFetch.ts	guardedFetch and isPublicAddress, for an address a person gives a server
io/ingestion/MultimodalAggregator.ts	Image captions for hosts
io/facade/types.ts	IngestionConfig, IngestOptions, IngestResult

All paths are under src/cognition/memory/.