Memory Operations

Three operational subsystems sit around the AgentOS memory facade: Wunderland's auto-ingest pipeline, which extracts facts from conversation turns, six agent-facing tools that let an agent read and write its own memory at runtime, and an import/export layer (four export formats, six import formats). The tools and the import/export layer work on one Memory instance's SQLite brain, so a trace added by one is found by the other.

Subsystem	Trigger	Direction
Auto-Ingest (Wunderland)	`processConversationTurn()` after a turn	Conversation → vector store or cognitive memory
Agent Tools	LLM tool call (`memory_add`, `memory_search`, …)	Agent ↔ memory
Import/Export	Explicit `memory.export()` / `memory.import()`	Memory ↔ external format

Auto-Ingest Pipeline

MemoryAutoIngestPipeline (Wunderland) takes one user message and one assistant reply, asks an LLM for facts worth keeping, filters them by the agent's HEXACO-derived memory config, and stores the rest. The host supplies the LLM call (llmCaller), so the model is the host's choice.

How It Works

Extract: llmCaller returns candidate facts, each with a category, an importance score (0-1) and entities. When the call fails, the turn stores nothing.
Filter: a fact whose category the personality config does not enable is skipped. The category's boost is added to the importance, and a fact below the threshold is skipped; the threshold is the lower of the personality-derived value and storage.autoIngest.importanceThreshold (default 0.4). At most maxPerTurn facts are kept, the lower of the two limits (default 3).
Store: with a cognitive memory manager attached, each fact goes through CognitiveMemoryManager.encode() as a trace tagged with its category and auto_ingest. Without one, the facts are upserted into the auto_memories collection of the agent's vector store, with category, importance, entities, conversation id and timestamp as metadata, and embedded only when the host passes embedFn.

The pipeline does no deduplication.

Fact Categories

Category	What it captures
`user_preference`	Likes, dislikes, stated preferences
`episodic`	What happened in the conversation
`goal`	What the user wants to achieve
`knowledge`	Technical or domain facts learned
`correction`	Corrections to prior beliefs or statements

HEXACO Personality Modulation

derivePersonalityMemoryConfig() starts from a threshold of 0.4, three facts per turn and the five categories above, then adjusts them for each trait above 0.6:

Trait (> 0.6)	Effect
Openness	Lowers the importance threshold (floor 0.2), allows one more fact per turn (up to 5), enables the `emotional_context` category. Openness below 0.4 raises the threshold (cap 0.7) and allows one fact fewer.
Conscientiousness	Enables the `action_item` category with a 0.15 boost, adds 0.1 to `goal`
Agreeableness	Adds 0.15 to `user_preference`
Emotionality	Enables `emotional_context` with a 0.1 boost, adds 0.1 to `episodic`
Honesty	Adds 0.2 to `correction`

The function also sets a deduplication threshold, a compaction interval, a retrieval top-K and a sentiment flag from the traits; nothing reads those four fields. Wunderland derives the config from the agent's personality block in agent.config.json.

Configuration

Add a storage.autoIngest section to agent.config.json:

{
  "storage": {
    "autoIngest": {
      "enabled": true,
      "importanceThreshold": 0.4,
      "maxPerTurn": 3
    }
  }
}

Key	Default	Description
`enabled`	`true`	Toggle the pipeline on/off
`importanceThreshold`	`0.4`	Minimum boosted importance to store a fact; the pipeline uses the lower of this and the personality-derived threshold
`maxPerTurn`	`3`	Maximum facts stored per turn; the pipeline uses the lower of this and the personality-derived limit

Integration Points

wunderland chat builds the pipeline from the agent's storage config and traits and calls processConversationTurn() after each reply without awaiting it. It builds the pipeline with a placeholder llmCaller that returns no facts, so a chat session stores nothing through it. Wunderland's tool-failure learner also passes tool failures through the pipeline.

Relationship to Observer / Reflector

The auto-ingest pipeline and the Observer/Reflector system are complementary but operate differently:

	Auto-Ingest	Observer / Reflector
Trigger	Every turn	Token threshold (30K / 40K)
Granularity	Per-turn fact extraction	Batch observation + consolidation
LLM cost	Cheap model, small prompt	Larger prompt with conversation history
Output	Individual facts in vector store	Observation notes + long-term memory traces
Purpose	Capture details before they scroll out of context	Compress and consolidate accumulated knowledge

With a cognitive memory manager attached, auto-ingested facts become traces and are retrieved with the same composite score as the observer's notes (vector similarity, Ebbinghaus strength, emotional congruence, recency, graph activation, importance). Without one, they stay in the auto_memories collection.

Key Files

The auto-ingest pipeline ships in Wunderland, not in the @framers/agentos package.

File (in `jddunn/wunderland`)	Purpose
`src/memory/auto-ingest/MemoryAutoIngestPipeline.ts`	Pipeline orchestrator
`src/memory/storage/PersonalityMemoryConfig.ts`	HEXACO-to-config mapping
`src/memory/storage/AgentStorageManager.ts`	Holds the per-agent vector store and the pipeline (`setAutoIngestPipeline()`)
`src/cli/commands/chat.ts`	Builds the pipeline and calls it after each reply

Agent Memory Tools

Six ITool implementations let agents read, write, search, and consolidate their own memory traces at runtime. Memory.createTools() returns them; register them one by one, as a pack, or through AgentOS.create({ memoryTools }).

Overview

Tool	Name	Description	Side Effects
`MemoryAddTool`	`memory_add`	Store a new memory trace	Write
`MemoryUpdateTool`	`memory_update`	Update content or tags of an existing trace	Write
`MemoryDeleteTool`	`memory_delete`	Soft-delete a trace by ID	Write
`MemoryMergeTool`	`memory_merge`	Merge two or more traces into the most-retrieved one	Write
`MemorySearchTool`	`memory_search`	FTS5 full-text search over traces	Read-only
`MemoryReflectTool`	`memory_reflect`	Trigger on-demand consolidation	Write

createTools() leaves out memory_reflect when the Memory was created with selfImprove: false (there is no consolidation loop) or when called with { includeReflect: false }. All tools implement the ITool interface and belong to the memory category.

Registration

Register the tools directly

import { AgentOS, Memory } from '@framers/agentos';

const memory = await Memory.createSqlite({ path: './brain.sqlite', selfImprove: true });
const agentos = await AgentOS.create();

for (const tool of memory.createTools()) {
  await agentos.getToolOrchestrator().registerTool(tool);
}

Through the AgentOS configuration

import { AgentOS, Memory } from '@framers/agentos';

const memory = await Memory.createSqlite({ path: './brain.sqlite', selfImprove: true });

const agentos = await AgentOS.create({
  memoryTools: {
    memory,
    includeReflect: true,     // Include memory_reflect tool
    identifier: 'primary-memory-tools',
    manageLifecycle: true,    // AgentOS closes Memory on shutdown
  },
});

As an extension pack

import { createMemoryToolsPack } from '@framers/agentos';

await agentos.getExtensionManager().loadPackFromFactory(
  createMemoryToolsPack(memory),
  'memory-tools',
);

Tool Reference

`memory_add`

Store a new memory trace in the agent's brain database.

Input Parameters:

Parameter	Type	Required	Default	Description
`content`	`string`	Yes	---	The text content to remember
`type`	`string`	No	`'episodic'`	Memory type: `episodic`, `semantic`, `procedural`, `prospective`, `relational`
`scope`	`string`	No	`'user'`	Visibility scope: `thread`, `user`, `persona`, `organization`
`tags`	`string[]`	No	`[]`	Free-form tags for filtering

Output:

{ "traceId": "mt_3f2b8c1e-5d4a-4e6f-9b7c-2a1d0e8f6c5b" }

Behaviour:

Creates a trace with strength 1.0 (full encoding strength at creation time) and no embedding.
Refuses a type or scope outside the lists above with an error result and stores nothing, also when the caller runs the tool without checking its arguments against the schema.
Stores the SHA-256 hash of the content in the trace's metadata. It does not check for an existing trace with the same content: calling it twice stores two traces.
ID format: mt_<uuid>, a v4 UUID, as the Memory facade names its traces.
Files the trace under the scope id resolved from the execution context (for example the user id for user scope), in the trace's scope_id column and in its metadata.
Writes the trace's full-text entry in the transaction that writes the trace: in the memory_traces_fts table on SQLite (FTS5 on better-sqlite3; FTS4 on sql.js and the IndexedDB adapter), in the trace's _tsv column on Postgres. memory_search finds the trace as soon as the call returns.

Example:

{
  "name": "memory_add",
  "arguments": {
    "content": "User prefers dark mode and TypeScript.",
    "type": "semantic",
    "tags": ["preference", "ui", "language"]
  }
}

`memory_update`

Update the content or tags of an existing memory trace.

Input Parameters:

Parameter	Type	Required	Description
`traceId`	`string`	Yes	ID of the trace to update
`content`	`string`	No	New text content (replaces old)
`tags`	`string[]`	No	New tags array (replaces old)

Output:

{ "updated": true }

Behaviour:

Locates the active (not deleted) trace by ID in memory_traces.
Updates only the specified fields (content, tags, or both).
When the content changes, recomputes the content hash and clears the stored embedding.
Rebuilds the brain's full-text index: the FTS5 table on SQLite; on Postgres, the _tsv column of the brain's own rows.
Returns { "updated": false } if the trace was not found, was deleted, or neither field was given.

Example:

{
  "name": "memory_update",
  "arguments": {
    "traceId": "mt_3f2b8c1e-5d4a-4e6f-9b7c-2a1d0e8f6c5b",
    "content": "User prefers dark mode, TypeScript, and VS Code.",
    "tags": ["preference", "ui", "language", "editor"]
  }
}

`memory_delete`

Soft-delete a memory trace by ID.

Input Parameters:

Parameter	Type	Required	Description
`traceId`	`string`	Yes	ID of the trace to delete

Output:

{ "deleted": true }

Behaviour:

Sets deleted = 1 on the trace (soft-delete, not physical removal).
Soft-deleted traces are excluded from search results and retrieval.
The trace remains in the database for audit/provenance purposes.
Deletes the trace's knowledge-graph copy, the episodic record a memory manager on the same Brain writes of each trace it stores, when the tool was made by Memory.createTools(); Memory.forget() does the same.
Returns { "deleted": false } if the trace was not found.

Example:

{
  "name": "memory_delete",
  "arguments": {
    "traceId": "mt_3f2b8c1e-5d4a-4e6f-9b7c-2a1d0e8f6c5b"
  }
}

`memory_merge`

Merge two or more memory traces into one surviving trace.

Input Parameters:

Parameter	Type	Required	Description
`traceIds`	`string[]`	Yes	IDs of traces to merge (minimum 2)
`mergedContent`	`string`	No	Content for the merged trace (if omitted, the contents are joined with `\|`)

Output:

{
  "survivorId": "mt_aaa",
  "deletedIds": ["mt_bbb", "mt_ccc"]
}

Behaviour:

Fails when fewer than two of the IDs name active traces.
The trace with the highest retrieval count survives and takes the merged content; its embedding is cleared. On a tie the first of traceIds survives, and joined contents follow the order of traceIds.
Tags from all source traces are unioned onto the survivor.
Retrieval counts are summed, and the survivor keeps the latest access time and the highest decay stability among the sources.
The other traces are soft-deleted, and the brain's full-text index is rebuilt (on Postgres, the brain's own rows). Their knowledge-graph copies go, as memory_delete's do.

Example:

{
  "name": "memory_merge",
  "arguments": {
    "traceIds": ["mt_aaa", "mt_bbb", "mt_ccc"],
    "mergedContent": "User's deployment preferences: Docker Compose, blue-green strategy, Friday deploys."
  }
}

`memory_search`

Full-text search over memory traces using the FTS5 index with BM25 ranking.

Input Parameters:

Parameter	Type	Required	Default	Description
`query`	`string`	Yes	---	Full-text search query
`type`	`string`	No	---	Filter by memory type
`scope`	`string`	No	---	Filter by visibility scope
`limit`	`number`	No	`10`	Maximum results to return

Output:

{
  "results": [
    {
      "id": "mt_abc123",
      "content": "User prefers dark mode and TypeScript.",
      "type": "semantic",
      "scope": "user",
      "strength": 0.87,
      "tags": ["preference", "ui"]
    }
  ]
}

Behaviour:

Queries the brain's full-text index: the memory_traces_fts virtual table on SQLite (FTS5 on better-sqlite3; FTS4 on sql.js and the IndexedDB adapter, whose SQLite builds have no FTS5), the _tsv column of the brain's own rows on Postgres.
The best match comes first on every backend: by BM25 on FTS5, by the number of matches (length(offsets())) on FTS4, by ts_rank on Postgres.
Only active traces are returned (soft-deleted traces are excluded).
Finds a trace as soon as the write that made it returns, on both backends: memory_add, Memory.remember(), MemoryStore.store() with a Brain attached (as CognitiveMemoryManager attaches the brain its config names) and Brain.importFromSqlite each bring the index with the write.
A trace stored again under its id with new words (a MemoryStore revival, or a trace Brain.importFromSqlite rewrites) is found by its new words only.
Optional type and scope filters are applied via SQL WHERE clauses on the join; a scope filter also matches the scope id resolved from the execution context.
limit takes 1 to 100.
The query runs as FTS5 syntax first; when SQLite rejects it as a syntax error, the tool converts it to a natural-language FTS5 query and runs it again.

FTS5 Query Syntax

The tool accepts FTS5 operators directly, and falls back to a converted query for natural-language input that FTS5 rejects:

Syntax	Meaning	Example
`word`	Match any form (Porter stemming)	`deploy` matches "deployment", "deployed"
`"phrase query"`	Exact phrase match	`"dark mode"`
`word1 AND word2`	Both terms required	`docker AND compose`
`word1 OR word2`	Either term	`typescript OR javascript`
`word1 NOT word2`	Exclude term	`deploy NOT staging`
`prefix*`	Prefix match	`type*` matches "typescript", "types"

Example:

{
  "name": "memory_search",
  "arguments": {
    "query": "deployment preferences",
    "type": "semantic",
    "limit": 5
  }
}

`memory_reflect`

Trigger on-demand memory consolidation --- the analogue of slow-wave sleep. Runs the full 6-step ConsolidationLoop.

Input Parameters:

Parameter	Type	Required	Description
`topic`	`string`	No	Reserved for future topic-scoped consolidation (currently ignored)

Output:

{
  "pruned": 3,
  "merged": 1,
  "derived": 0,
  "compacted": 2,
  "durationMs": 42,
  "personalityDecayed": 0
}

Behaviour:

Runs the 6 consolidation steps in order:

Prune --- soft-delete traces below strength threshold (default 0.05).
Merge --- deduplicate near-identical traces (similarity > 0.95).
Strengthen --- record Hebbian co-activation edges from retrieval feedback.
Derive --- synthesise insight traces from memory clusters (LLM-backed, skipped if no LLM).
Compact --- promote old high-retrieval episodic traces to semantic type.
Re-index --- rebuild the brain's full-text index (on Postgres, its own rows) and log run to consolidation_log.

If a consolidation cycle is already in progress (mutex), returns immediately with zero counts.

Example:

{
  "name": "memory_reflect",
  "arguments": {}
}

With a learning layer

Where a learning layer runs (GMI, Learning), the GMI puts the turn's scope ids and the learner session's recorder on every tool call (sessionData.learning), and the memory tools file, read and write by them (scopeContext.ts, learningWrite.ts):

Filing. memory_add files a user or persona trace under the turn's own scope id (the learner's, <namespace>/<userId> with a namespace, or the session's form on a turn with no learner) and a thread trace under the call's conversation, else its session, in the session's form (<namespace>//<conversation>, or /<conversation> with no namespace). Where the turn's memory names a session of its own, a gmi() session id's generation, the call carries that session's ids (sessionData.learning.threadIds) and the thread trace goes under the first of them in the same form, in place of the call's conversation and session, which name the id the caller opened: an id's next generation finds no thread note an earlier one filed, and forgetUser reaches each generation's through the session row. It refuses an organization trace (organization_scope), and a thread trace on a call that names neither a conversation nor a session (no_thread). On a second user's turn with no learner each trace names that user (turnUserId in its metadata), so forgetUser for that user finds it.
Recording. Each write runs through the recorder: the freeze, the user's tombstone, a pending ledger row (memory_write, actor tool:<name>), the write, the tombstone again, the verdict. A kept write succeeds with details: { status: 'kept', ledgerId }; in mode off and under a suspension it succeeds with no row (status: 'written'). A write that is not kept changes nothing and fails with details: { status: 'rejected', reason }, the candidate's ledgerId beside it where one was written: write_failed (the error ends the message), forgotten (the user was forgotten; what the write made is deleted for good), store_unavailable (the learning store could not be read or the verdict written; the write is taken back) or conflict. A turn whose session gives scope ids and no recorder refuses every write (no_recorder). A write is taken back through the facade that made the tools, so the trace's memory-graph node, index entry and knowledge-graph copy go with its row.
By id. memory_update, memory_delete and memory_merge reach a live trace whose scope and scope id are one of the turn's own (user, persona, and the conversation's and the session's thread, or the turn's threadIds where the call carries them) and that the turn's own user filed. A trace outside those scopes reads as not found (updated: false, deleted: false, a merge's Found only N), so a call learns nothing about another scope; another user's trace inside them (a shared thread, a session two users spoke on) is refused (other_user), and a merge that names one is refused whole. A trace filed under no scope id and an organization trace are changed by no turn. A thread note memory_add files names whose turn filed it (metadata.threadFiler: the learner's user scope id, or the session's form on a turn with no learner), and only a turn of the same filer reaches it by id: an anonymous turn on a learner's session, which names no second user, reaches none of the learner's thread notes, and the learner none of its.
Compare-and-set. The three read each trace they change first and write only while it is still filed where they read it (its scope and scope id) and holds the content and tags they read: a trace another writer changed in between fails the write (conflict, its row rejected) and nothing changes. A trace moved to another scope id in between (a host's MemoryStore.refileScope, a Brain's re-file) fails it the same way. A recall's access counts are left out of the comparison.
Search. memory_search reads the scope_id column: the turn's own scopes, its organization's id (the turn's organizationId; the user context's only when it is the namespace of the turn's scope ids, since the user context keeps an organization an earlier turn's override set) and the traces filed under no scope id (an ingested document's). A scope filter narrows that to the scope's own ids and its traces filed under no scope id. A row that holds no scope id while its metadata names one is not among those: a release older than the column writes such rows to a Brain another process holds open, and no turn's search reads one until the Brain's next open files it under the id its metadata names. A learner's trace is found by the learner's later sessions, never by a user of the same id in another namespace or by a session with no learner.
Freeze. Only a freeze stops a write: AGENTOS_LEARNING_FREEZE=1, a turn in mode frozen, or a run-time freeze of the turn's scope, read at the call (frozen). Mode off, a suspension and a turn with no learner write.
memory_reflect refuses every call that carries sessionData.learning (withdrawn), after the freeze: consolidation merges and derives across every scope of the brain. A host that registers the tools itself where a layer runs leaves it out (createTools({ includeReflect: false })) and passes the facade the tools come from as learning.memoryTools, so forgetUser and revert reach its Brain through the facade's deletes and restores (Deleting and restoring traces); the full runtime's pack (memoryTools, or standaloneMemory.tools) leaves it out wherever the runtime has a layer, and says so once when includeReflect: true asked for it.
Approval gate. With the tool orchestrator's approval gate on (toolOrchestratorConfig.hitl.enabled), a write refused by the freeze, organization_scope, no_thread, no_recorder or withdrawn is answered before the gate (ITool.refusalBeforeApproval): no one is asked to approve a write that cannot act, and with no hitlManager the call names its refusal rather than the missing manager. A refusal that reads the trace (other_user, conflict) and a trace not found are answered when the approved write runs.

With no sessionData.learning, each tool files, reads and reaches by id as the sections above say, and only AGENTOS_LEARNING_FREEZE=1 stops a write, before the approval gate as above. A learner session that gives no scope ids (a host's own, written against an earlier release) leaves the tools filing, reading and reaching by id the same way; its freeze still stops a write, and memory_reflect still refuses.

When Agents Should Use Each Tool

Situation	Recommended Tool
Agent learns a new fact about the user	`memory_add` with `type: 'semantic'`
Agent wants to record a conversation event	`memory_add` with `type: 'episodic'`
Agent discovers a previous memory is outdated	`memory_update` to correct the content
Agent finds a memory is wrong or harmful	`memory_delete` to soft-remove it
Agent notices several memories say the same thing	`memory_merge` to consolidate them
Agent needs to look up what it knows about a topic	`memory_search` with relevant keywords
Agent has been running for a while and wants to clean up	`memory_reflect` to trigger consolidation
Agent needs to remember a future task	`memory_add` with `type: 'prospective'`
Agent sees a gisted memory and needs the full original text	`rehydrate_memory` with the trace ID

rehydrate_memory (opt-in)

Retrieves the original verbatim content of a memory trace whose content has been compressed by temporal gist. Neither Memory.createTools() nor createMemoryToolsPack() adds it (MemoryToolsExtensionOptions.includeRehydrate is declared but not read): build it with new RehydrateMemoryTool(archive), passing an IMemoryArchive, and register it yourself. The package entry points do not export the class; import it from @framers/agentos/memory/io/tools/RehydrateMemoryTool.

Input: { traceId: string } — the ID of the gisted/archived trace.

Output: { verbatimContent: string | null, archivedAt: number | null } — the original content before gisting, or null if the trace is not archived or integrity verification fails.

Side effects: writes a row to archive_access_log so the retention sweep knows which traces are still in use.

Source Files

All under src/cognition/memory/io/tools/:

File	Purpose
`MemoryAddTool.ts`	`memory_add` implementation
`MemoryUpdateTool.ts`	`memory_update` implementation
`MemoryDeleteTool.ts`	`memory_delete` implementation
`MemoryMergeTool.ts`	`memory_merge` implementation
`MemorySearchTool.ts`	`memory_search` implementation
`MemoryReflectTool.ts`	`memory_reflect` implementation
`RehydrateMemoryTool.ts`	`rehydrate_memory` implementation (opt-in)
`index.ts`	Barrel exports for all tools
`scopeContext.ts`	Scope ID resolution from execution context, and the turn's own scopes where a learning layer put them on the call
`learningWrite.ts`	The four writers' path through a learner session's recorder: the by-id reach, the compare-and-set, the take-back

Import and Export

The Memory I/O subsystem exports to four formats and imports from six. The SQLite and JSON exports carry the whole brain: JSON holds traces, graph rows, documents, chunks, images, conversations, and messages. Markdown and Obsidian exports carry the active traces only. Trace deduplication is on by default on import.

Overview

Direction	Formats	Use Case
Export	SQLite, JSON, Markdown, Obsidian	Backup, sharing, human review, Obsidian knowledge management
Import	SQLite, JSON, Markdown, Obsidian, ChatGPT, CSV	Restore, migrate, ingest existing knowledge, import chat history

All operations are available via the Memory facade API.

Export Formats

SQLite (One Brain, Every Table)

The highest-fidelity export. Writes the brain's own rows of every portable table into a new SQLite file: traces with their embeddings, the knowledge graph, documents and chunks, conversations and messages, the consolidation log and the rest. It runs from a SQLite or a Postgres brain alike. A database can hold several brains, and the file holds this brain's rows only.

await mem.export('./backup.sqlite', { format: 'sqlite' });

Nothing may exist at the path yet.
Includes raw embedding BLOBs (no re-embedding needed on import).
The rows keep the brain's id; importing the file into a brain with another id forks it.
Suitable for disaster recovery, agent cloning, or migration.

JSON (Programmatic)

A single structured JSON document containing the full portable brain payload. Designed for programmatic consumption, browser-safe transfer, and cross-runtime restore.

await mem.export('./memories.json', { format: 'json' });

// Optional: include raw embedding vectors (increases file size significantly)
await mem.export('./memories-with-embeddings.json', {
  format: 'json',
  includeEmbeddings: true,
});

Each trace entry is the raw memory_traces row, with snake_case columns and JSON-encoded tags, emotions and metadata strings:

{
  "brain_id": "brain",
  "id": "mt_3f2b8c1e-5d4a-4e6f-9b7c-2a1d0e8f6c5b",
  "type": "semantic",
  "scope": "user",
  "content": "User prefers TypeScript over Python",
  "embedding": null,
  "strength": 0.87,
  "created_at": 1711234567890,
  "last_accessed": 1711234600000,
  "retrieval_count": 3,
  "tags": "[\"preference\",\"language\"]",
  "emotions": "{}",
  "metadata": "{\"source\":\"user_statement\"}",
  "deleted": 0,
  "scope_id": ""
}

scope_id is the trace's scope id, which its metadata's scopeId also names; '' for a trace filed under none, as here.

includeConversations: false leaves out the conversation and message rows.

Markdown (Human-Readable)

Exports each of the brain's active traces as a standalone .md file with YAML front-matter, in a {scope}/{type}/ folder; another brain's traces in the same database are not read:

await mem.export('./memory-export', { format: 'markdown' });

Produces files like:

memory-export/
  user/
    episodic/
      mt_3f2b8c1e-5d4a-4e6f-9b7c-2a1d0e8f6c5b.md
    semantic/
      mt_8a41d6f2-0c7e-4b39-a5d8-6e2f91c4b073.md
  thread/
    procedural/
      mt_c95e0b7a-1f2d-4c68-8e3b-7d4a2f6e9b10.md

Each file:

---
id: mt_8a41d6f2-0c7e-4b39-a5d8-6e2f91c4b073
type: semantic
scope: user
strength: 0.87
tags:
  - preference
  - language
createdAt: 1711234567890
---
User prefers TypeScript over Python

A trace filed under a scope id also writes it, as scopeId after scope, so an import files the note under it. The scope id is the trace's metadata scopeId, else its scope_id column, as a JSON import reads the trace's row. A trace filed under '', like the one above, writes no scopeId. A Brain a learning layer serves holds its plain user and persona scope ids in the encoded form (A Brain's scope-id form), and each note it exports says so after scopeId (scopeIdForm: encoded), so an import takes the note's scope id as it is written. A trace a second user filed on another user's session (its metadata's turnUserId) writes that user's id next, as turnUserId, so an import files the note as that user's trace, which a delete of the user's traces (deleteScope with turnUserId) finds.

A trace's scope and type name its folders and its id names its file, so each must be a safe file or folder name: not empty; no slash, backslash or colon, none of ", *, <, >, ? and |, and no control character; not starting with a dot; not starting or ending with a space, or ending with a dot; at most 200 bytes in UTF-8. The ids, scopes and types AgentOS makes all are. A trace imported or added with any other value is left out of the export, and export() lists it in refused with the field and the reason. Below the export folder, folders are made one name at a time and no folder or file is written through a link: a trace whose folder or file there is a symbolic link, or whose file there has a second name (a hard link), is left out too (problem: 'link'). The export folder itself may be a link; to export into a linked folder, pass it as the export folder. A trace whose file's place holds anything but a regular file, such as a folder, a named pipe, a socket or a device, is left out as well (problem: 'special'): the export opens a file without waiting, so a named pipe that no process reads never holds it up. Of two traces whose files would be one file, the older is written and the newer left out (problem: 'collision'). On every platform, that includes two names that differ only in case or Unicode form, which macOS and Windows treat as one name (mt_case and MT_CASE, or the long s ſ and s); names they keep apart, such as ß and ss, are both written. It also includes two names the file system resolves to one file in another way, such as a Windows short name; the export finds those by comparing the files it opens. Every other trace is written, and no file is written outside the export folder. An error opening or writing a file for any other reason, such as a full disk, rejects export() once every other trace has been tried. The checks run as each folder is made and each file opened, so they do not stop another process that swaps a folder below the export folder for a link while the export runs:

const { refused } = await mem.export('./memory-export', { format: 'markdown' });
for (const trace of refused) {
  console.warn(`not exported: ${trace.id} (${trace.field}: ${trace.problem})`);
}

A JSON export writes the brain's traces, and a SQLite export the brain's rows of every portable table, into one file; their refused is empty.

Obsidian (Wikilinks + Tags)

Extends the Markdown exporter with Obsidian-specific features for integration with Obsidian knowledge vaults:

await mem.export('./vault', { format: 'obsidian' });

Produces:

#tags in the body for every tag on the trace.
A Related: list of [[wikilinks]] to the labels of the knowledge-graph nodes the trace's node links to in the brain's own graph (empty when the graph has no edges for it). Another brain in the same database never adds a link, though node ids repeat across brains.
The same YAML front-matter and {scope}/{type}/ folders as the Markdown export, with the same rule for the names a trace's folders and file take, and the same refused list.

Example output:

---
id: mt_8a41d6f2-0c7e-4b39-a5d8-6e2f91c4b073
type: semantic
scope: user
strength: 0.87
tags:
  - preference
  - language
createdAt: 1711234567890
---
User prefers TypeScript over Python

#preference #language

**Related:**
- [[TypeScript]]
- [[tooling]]

Import Formats

SQLite

Merges one brain's SQLite file into the current brain. The importer reads the rows of the one brain the file holds: a file whose rows carry several brain ids imports nothing, and errors names the ids. A SQLite export holds one brain, so export each brain and import its file. A file older than the brain_id column holds one brain, and every row of it is read. A source trace is not inserted when a trace of the same scope and scope id in the brain has the same content or carries the content's SHA-256 as its import_hash or content_hash: its tags are unioned into that trace, which keeps the newer created_at, and the row counts as skipped. Once the import has committed, the importer rebuilds the brain's full-text index, once, when it inserted a trace or a merge changed a trace's tags (on Postgres, the _tsv column of the brain's own rows), so memory_search finds the imported traces, and a trace by a merged tag. Knowledge nodes are matched by label and type, edges by source, target and type. A trace's episodic copy comes in with its trace alone: the copy of a source trace merged into a trace the brain holds, or written under another id because its id names another trace, is not imported, nor are the edges that name it. A trace's memory-graph node (type memory_graph, whose id is its trace's) comes in with its trace alone and under that id: the node of a source trace merged into a trace of another id, written under a new id or not written, or of a trace the brain does not hold, is skipped, and so are the edges that name it; a node the brain holds under that id is its duplicate. So no node the import writes stands for a trace the brain does not hold, and an export imported into the brain it came from adds no memory-graph node.

const result = await mem.importFrom('./backup.sqlite', { format: 'sqlite' });
console.log(`Imported: ${result.imported}, Skipped: ${result.skipped}`);

JSON

Parses a JsonExporter-format JSON file and restores traces plus any included graph/document/conversation rows. A trace's episodic copy comes in with its trace alone, as in the SQLite import: the copy of a trace merged into one the brain holds, or written under another id, is not imported, nor are the edges that name it. A trace record whose metadata is not a JSON object (text that is not JSON, an array, null) is stored with a metadata object of its own, which names the record's scope id and import hash; a SQLite import treats a source row's metadata the same way. A knowledge node's properties and an edge's metadata that are not a JSON object are stored the same way, with the import hash a second import finds them by. A trace's memory-graph node comes in with its trace alone and under its trace's id, as in the SQLite import above: a node left out takes the edges that name it with it.

const result = await mem.importFrom('./memories.json', { format: 'json' });

Markdown

Walks a directory of Markdown files with YAML front-matter. Each file whose body is not blank becomes one memory trace. The id, type, scope, scopeId, turnUserId, tags, strength and createdAt fields are read from front-matter (defaults: a new id, episodic, user, '', the session owner's, no tags, 1.0, now); the body, without its leading and trailing whitespace, becomes the trace content. A scopeId that is a string files the trace under that scope id, in its scope_id column and its metadata; any other value is ignored. A turnUserId that is a non-empty string is written into the trace's metadata, as a second user's trace on another user's session; any other value is ignored. A note whose id names a trace of its scope and scope id, whichever user filed it, and whose body is that trace's text without its leading and trailing whitespace, is that trace's own note and is skipped, so a Markdown export imported into the brain it came from adds nothing, whatever whitespace surrounds a trace's text. That comparison reads a CRLF or CR line break as LF, so a note an editor saved again with other line endings is still its trace's own note; a note that is imported keeps the line endings its file holds. Front-matter is read as data: a block marked as JavaScript (---js) gives no fields and is never evaluated.

const result = await mem.importFrom('./notes/', { format: 'markdown' });

Obsidian Vault

Extends the Markdown importer with Obsidian-specific parsing:

const result = await mem.importFrom('./my-vault/', { format: 'obsidian' });

Each [[wikilink]] (or [[Target|Alias]]) becomes a related_to knowledge-graph edge from the note's node to a node for the target label, created when missing.
YAML front-matter fields are read as in the Markdown importer.
#tags in the body are extracted and added to the trace's tag array.
Embedded images (![[image.png]]) are not imported.
The folder a note sits in does not set its type: a note without type in its front-matter is imported as episodic.
A note whose id names a trace of its scope and scope id is that trace's own note, and is skipped, when its body is the trace's text without its leading and trailing whitespace, followed by nothing but what the Obsidian export writes after it: the tag line, the Related: list, or both, a blank line apart. So a vault exported from a brain and imported into it adds nothing. The trace's text is read whole, a line of hashtags it ends in included. Each link of the list is a line - [[label]], or runs over the line breaks of a label the brain's graph holds, as the export writes a label that holds a line break; the import reads the graph for such labels once, the first time a list does not read as one line a link. A note whose link runs over the line breaks of a label the graph no longer holds is not its trace's own note, and its import adds the trace again. A note with anything else after the text, such as a paragraph added in Obsidian before or after its links, is not the trace's own note. Line breaks compare as in the Markdown import.
A trace's own note counts as skipped, and the import adds to the trace the tags of its tag line the trace does not hold, a tag compared in the form the tag line writes it (whitespace as -), and a related_to edge from the trace's node to each label of its Related: list the node does not link to, a link read as any note's link is (its target, without an alias). So a tag or a link added to the note in Obsidian reaches the trace, and a note left as the export wrote it adds nothing to a trace that still holds the tags and links the export wrote. The import cannot tell a tag or a link added in Obsidian from one the trace lost after the export: a tag or a link removed from a trace after its export comes back when that export is imported, as a SQLite import brings back the tags of a record it matches. The note's front matter is read only to find its trace, as in the Markdown import: a tag added to its tags is not added to the trace.
An import that inserted a note, or added a tag to a trace the brain holds, rebuilds the brain's full-text index once, after its last note, as the SQLite import does, so memory_search finds a trace by a tag its note added.

ChatGPT Export

Imports from ChatGPT's conversations.json export file. Each conversation becomes a row in the conversations table, and each user/assistant message pair becomes an episodic memory trace:

const result = await mem.importFrom('./conversations.json', { format: 'chatgpt' });

Each message pair becomes one episodic trace in user scope whose content is [user]: ... followed by [assistant]: ..., with no tags; its metadata names the conversation row.
A conversation keeps one row, named after it: cv_chatgpt_ followed by the SHA-256 of the export's id for it (conversation_id, else id), or, when the export names it by no id, of its title and creation time. A later import of the conversation, from the same export or a later one, finds the row by that name, adds no row, and files the pairs it adds under that row; a conversation the export names by an id keeps its row through a rename and when the export gives it no create_time. The import writes the row with an insert that does nothing when the brain holds a row of that name, so two imports that write one conversation's row at the same time, in one process or in two, write one row and file under it. A row of any other name, such as cv_ and a random UUID, is never taken, since nothing in it says which conversation it is: the next import of its conversation writes the row it names beside it, so the conversation has two rows from then on, and the pairs filed under the first keep naming it. The row's metadata records the export's id as chatgpt_id.
When the export's update time is later than the row's, the row takes that update time and the conversation's title. An export that gives no update time has the conversation's creation time stand in for it, and the time of the import stands in for a creation time the export does not give, so a conversation the export gives neither time counts as updated at each import.
A conversation the export names by no id is known by its title and creation time alone: two such conversations of one title and creation time share a row, and one renamed between two exports, or given no create_time, gets a row of its own, since the file then holds nothing that names it.
With deduplication on (the default), a pair's trace is named after the brain and its text, mt_chatgpt_ and their SHA-256, and written with an insert that does nothing when the brain holds a trace of that name, so two imports that store one pair at the same time, each after finding no duplicate of it, store one trace; the import whose insert wrote nothing counts the pair in skipped. With dedup: false, each import stores each pair again, under a random id.
System and tool messages are skipped.
The user message's timestamp from the export becomes the trace's created_at.
Long conversations produce many traces, which can then be consolidated via mem.consolidate().

CSV

Imports flat CSV files with a header row. A content column is required. The optional columns id, type, scope, strength, created_at (or createdAt), last_accessed, retrieval_count, deleted, tags (a JSON array, or comma- or pipe-separated), metadata (a JSON object) and scope_id map onto the trace's fields:

const result = await mem.importFrom('./knowledge-base.csv', { format: 'csv' });

Expected CSV structure:

content,type,tags
"User prefers dark mode",semantic,"preference,ui"
"Deploy with Docker Compose",procedural,"deployment,docker"

A row's scope id is its metadata cell's scopeId, else its scope_id cell, which is then written into the metadata too. The metadata cell is read before the duplicate check: a cell that is not valid JSON, or whose JSON is not an object (an array or null, say), makes the row an error, listed in errors and not imported, even when a trace of the same scope already holds the row's text.

SHA-256 Deduplication

All import paths deduplicate traces by their text and its SHA-256:

Before inserting a trace, the importer computes SHA-256(content) and stores it in the trace's metadata as import_hash.
When a trace of the same scope, scope id and user holds the record's text, or its metadata names that hash as import_hash or as content_hash (the hash remember(), ingest() and memory_add store), the row is skipped (counted in skipped), whatever its type and whichever writer stored the trace. A trace the cognitive memory store (MemoryStore) wrote carries no hash, and its text finds it. A trace the brain forgot (through forget(), memory_delete, a merge or the consolidation prune) counts as any other, so an import does not bring a forgotten trace back, also from an export taken before it was forgotten; remember() stores its text again as a new trace, as does an import with dedup: false. A JSON, SQLite or CSV file can hold a forgotten and a live record of one text, as an export of a brain that forgot a text and then stored it again, or merged two traces of it, does; the import reads a file's live records before its forgotten ones, so the live record is imported and the forgotten one is its duplicate, whatever order the file holds them in, and the memory graph keeps the live trace's node and edges and none of the forgotten record's. A trace of another scope id is not a duplicate: a record whose text another tenant's, user's or session's trace holds is imported under its own scope id. Inside one scope id, a trace another user filed is not a duplicate either: a record's user is its metadata's turnUserId (a second user's trace on another user's session), a Markdown or Obsidian note's its front-matter turnUserId, and otherwise the session owner's, ChatGPT records included, so a second user's words and the same words of the session's owner stay two traces, as remember() keeps them, and a delete of either user's traces leaves the other's. A record's scope id is its metadata scopeId, else its scope_id (the field a JSON or SQLite export of a schema-3 brain carries, or a CSV row's cell), else ''; a Markdown or Obsidian note's is its front-matter scopeId, else ''; ChatGPT records file under ''. A trace a release older than the scope_id column wrote holds '' there whatever scope id its metadata names; before its first duplicate check the importer files such traces of the brain under that scope id, so a record of that scope id finds it and a record filed under '' does not. The SQLite importer merges the skipped row's tags, and the later of the two created_at times, into the trace it matched; when that changes the trace's tags, it rebuilds the full-text index, so the trace is found by the merged tags. An Obsidian note that is its trace's own note adds to that trace the tags of its tag line and the links of its Related: list the trace lacks, and rebuilds the full-text index when it adds a tag (see Obsidian Vault above).
Deduplication is enabled by default (dedup: true) and can be disabled per-import.

// Disable dedup (allows duplicate content)
const result = await mem.importFrom('./data.json', {
  format: 'json',
  dedup: false,
});

An import files each record under the scope id its file names, in every format that carries one, so a file can add traces to any tenant's, user's or session's scope id in the brain, and tags and graph links to the traces there: a SQLite file merges tags into the traces its records match, an Obsidian note adds tags and links to the trace it names, and a JSON or SQLite file adds graph rows. Import a file into a brain that several tenants share only when it comes from a source you trust, or set its scope ids yourself first.

A Brain's scope-id form

A Brain a learning layer serves holds its plain user and persona scope ids in the encoded form: a plain id that holds a /, a % or a : is percent-encoded (team/ana is filed team%2Fana), and the Brain records it in brain_meta (scope_id_form = encoded). The written form of a plain id gives the rule and the re-file that writes the record. Every import reads its source's form from the source's own record and keeps the Brain's:

Source	Its form
JSON export	encoded when its `meta` holds the record
SQLite file	encoded when the brain it holds has the record in `brain_meta`
Markdown or Obsidian note	encoded when its front matter says `scopeIdForm: encoded`
CSV file, ChatGPT export	plain

A source with the record whose user and persona scope ids, in its traces and its knowledge-graph copies, hold none of /, % and : reads the same in both forms, and an import takes it as a plain one: a Brain without the record stays without it, and the learning layer that opens that Brain later re-files the plain ids a host with no layer filed there meanwhile.

A plain source goes into a Brain that holds the record with each user and persona scope id encoded as it is written, in the trace's scope_id column and its metadata, and in each trace's knowledge-graph copy.
A source with the record goes into a Brain that holds the record as it is written.
A source with the record goes into a Brain without it when that Brain holds no plain id the encoded form changes, in a trace or in a knowledge-graph memory node, and the Brain then gets the record. A persona id is read part by part, so the :: between its user id and its persona id counts for neither. Otherwise nothing of the source is imported and errors names the record (scope_id_form); a Markdown or Obsidian folder is weighed note by note, and Brain.importFromSqlite throws.
An import reads the Brain's form in the transaction it writes in (each Markdown or Obsidian note in a transaction of its own), so a re-file of the Brain, or another import into it, that commits first is followed. On Postgres the import first takes the brain's advisory lock, as a re-file does.
An export reads its Brain's record and its rows in one form: when a re-file commits while the export reads, it reads again. The SQLite importer reads its source file the same way, and Brain.importFromSqlite copies again when its source's record changed while it copied. A read of the record that fails for another reason than a missing brain_meta table or brain_id column (a locked file, a dropped connection) fails the export or the import, rather than read the source as one without the record.

A source and a Brain that both lack the record take each other as they are, so where no learning layer has opened any Brain involved, every import files its ids as they are written.

The RAG MigrationEngine runs the same handshake before it copies any table, and refuses where an import would encode, since it copies rows as they are. A Qdrant source or target takes part through its SQLite sidecar's brain_meta: its traces are in Qdrant, where the handshake reads no scope id, so a brain the sidecar holds counts as holding ids the two forms read apart. A source that names no brain (a Pinecone index, a Qdrant source with no sidecar) is refused into a target that holds the encoded form and traces, and a Qdrant target with no sidecar takes no source brain that holds the record. A read of a form (a brain's record, whether it holds traces or knowledge-graph copies, or the list of the brains a store holds) or a clear of a target's record that fails for another reason than a missing table or column (a store without brain_meta or knowledge_nodes, or one from before brain ids) fails the migration, on a SQLite, PostgreSQL or Qdrant side alike (a locked file, a dropped connection): nothing is copied when a read fails, the error is in errors, and verified is false.

A Brain that takes the record from an import holds the encoded form from then on, whoever serves it: a host with no learning layer files its plain ids as they are, and the record reads one that holds a /, a % or a : as encoded or namespaced (team/ana then names user ana of namespace team), and no re-file moves it. Serve such a Brain with a learning layer.

ImportResult

Every import operation returns a summary:

interface ImportResult {
  imported: number;  // Rows written
  skipped: number;   // Records not written
  errors: string[];  // Human-readable messages for failures
}

imported counts the traces the import wrote, and, from a JSON file, its knowledge nodes and edges, documents, chunks, images, conversations and messages too; from a SQLite file, its knowledge nodes and edges too. An import also writes rows it does not count: a JSON file's brain metadata, the knowledge nodes and edge an Obsidian note's wikilink adds, and the conversations row of each conversation in a ChatGPT export. skipped counts the records it did not write: a duplicate trace (see SHA-256 Deduplication), a graph, document, chunk, image, conversation or message row the brain holds already, a Markdown or Obsidian note whose body is blank, a CSV row whose content is blank, a memory-graph node whose trace the import did not write under its id and an edge that names one, and a row that lacks a field it needs, such as an edge with no source or target. errors holds a message for each record that failed to import, for an Obsidian note's link or tag the import could not write, for a source that could not be read or parsed, and for a source, or a Markdown or Obsidian note, that the Brain's scope-id form refuses (A Brain's scope-id form).

After an import whose imported count is above zero, importFrom and importFromString rebuild the brain's HNSW index, when the brain has one, from the brain's embedded traces, and its full-text index; a SQLite or Obsidian import rebuilds the full-text index itself, also after an import that inserted nothing but gave a trace the brain holds new tags, so importFrom does not rebuild it again.

ExportOptions Reference

interface ExportOptions {
  /** Serialisation format. When omitted, the output path names it (see below). */
  format?: 'sqlite' | 'json' | 'markdown' | 'obsidian';

  /** Include raw embedding vectors in a JSON export. Default: false. */
  includeEmbeddings?: boolean;

  /** Include the conversation and message rows in a JSON export. Default: true. */
  includeConversations?: boolean;
}

With no format, a .json output path writes JSON, a .sqlite or .db path SQLite, and any other path a Markdown folder. A SQLite export always holds the embeddings and the conversation rows, as it holds every portable table; a Markdown or Obsidian export holds neither.

ImportOptions Reference

interface ImportOptions {
  /** Source format. Default: 'auto', read from the source (see below). */
  format?: 'auto' | 'sqlite' | 'json' | 'markdown' | 'obsidian' | 'chatgpt' | 'csv';

  /** Skip a record a trace of its scope, scope id and user duplicates (see SHA-256 Deduplication). Default: true. */
  dedup?: boolean;
}

With format: 'auto', a .json file is read as a ChatGPT export when its text holds "mapping" and "conversation_id", and as JSON otherwise; a .sqlite or .db file as SQLite, a .csv file as CSV, a folder as Markdown, and any other path as JSON. An Obsidian vault is a folder too: name format: 'obsidian' to read its tags and wikilinks. importFromString takes the format, 'json' or 'csv', as its own argument and dedup alone in its options.

Source Files

All under src/cognition/memory/io/:

File	Purpose
`JsonExporter.ts`	JSON export (one document)
`JsonImporter.ts`	JSON import with dedup
`MarkdownExporter.ts`	Markdown directory export with YAML front-matter
`MarkdownImporter.ts`	Markdown directory import
`ObsidianExporter.ts`	Obsidian vault export (wikilinks + tags)
`ObsidianImporter.ts`	Obsidian vault import (wikilinks -> graph edges)
`SqliteExporter.ts`	The brain's own rows of every portable table, in a new SQLite file
`SqliteImporter.ts`	Merge of one brain's SQLite file, with dedup + tag union
`ChatGptImporter.ts`	ChatGPT `conversations.json` parser
`CsvImporter.ts`	CSV import with required `content` column

Auto-Ingest Pipeline​

How It Works​

Fact Categories​

HEXACO Personality Modulation​

Configuration​

Integration Points​

Relationship to Observer / Reflector​

Key Files​

Agent Memory Tools​

Overview​

Registration​

Register the tools directly​

Through the AgentOS configuration​

As an extension pack​

Tool Reference​

memory_add​

memory_update​

memory_delete​

memory_merge​

memory_search​

FTS5 Query Syntax​

memory_reflect​

With a learning layer​

When Agents Should Use Each Tool​

rehydrate_memory (opt-in)​

Source Files​

Import and Export​

Overview​

Export Formats​

SQLite (One Brain, Every Table)​

JSON (Programmatic)​

Markdown (Human-Readable)​

Obsidian (Wikilinks + Tags)​

Import Formats​

SQLite​

JSON​

Markdown​

Obsidian Vault​

ChatGPT Export​

CSV​

SHA-256 Deduplication​

A Brain's scope-id form​

ImportResult​

ExportOptions Reference​

ImportOptions Reference​

Source Files​

Auto-Ingest Pipeline

How It Works

Fact Categories

HEXACO Personality Modulation

Configuration

Integration Points

Relationship to Observer / Reflector

Key Files

Agent Memory Tools

Overview

Registration

Register the tools directly

Through the AgentOS configuration

As an extension pack

Tool Reference

`memory_add`

`memory_update`

`memory_delete`

`memory_merge`

`memory_search`

FTS5 Query Syntax

`memory_reflect`

With a learning layer

When Agents Should Use Each Tool

rehydrate_memory (opt-in)

Source Files

Import and Export

Overview

Export Formats

SQLite (One Brain, Every Table)

JSON (Programmatic)

Markdown (Human-Readable)

Obsidian (Wikilinks + Tags)

Import Formats

SQLite

JSON

Markdown

Obsidian Vault

ChatGPT Export

CSV

SHA-256 Deduplication

A Brain's scope-id form

ImportResult

ExportOptions Reference

ImportOptions Reference

Source Files