Skip to main content

Read Router (Stage 3: Reader Strategy + Model Dispatch)

Stage 3 of the Cognitive Pipeline. Two sibling primitives, both at the read stage, both classifier-driven, both orthogonal:

  • Read Router picks the reader strategy (single-call, two-call-extract-answer, commit-vs-abstain, verbatim-citation, scratchpad-then-answer) based on a classified read intent. It controls how the reader generates the answer.
  • Reader Router picks the reader model (gpt-4o, gpt-5 or gpt-5-mini) based on the query category (single-session-user, multi-session, temporal-reasoning, etc.). It controls which model generates the answer.

The two primitives compose. A typical Phase B configuration runs both: Reader Router decides the model, Read Router decides the strategy that model follows.

Sibling stage primitives: Ingest Router (input), Memory Router (recall).

Read Router — read-strategy selection​

Classifies a query+evidence pair and picks a reader strategy. Different intents need different strategies:

  • A precise-fact lookup with clear evidence works fine with one reader call.
  • A multi-source synthesis question benefits from two-call extract-then-answer (the Emergence Simple pattern reduces distractor influence).
  • A time-interval question needs an explicit scratchpad to reason about dates before committing.
  • An adversarial question (likely unanswerable from evidence) needs commit-vs-abstain to avoid wrong commits on missing evidence.

Picking the wrong strategy costs accuracy or money — usually both. Read Router classifies the query+evidence and picks per-message.

Five read intents​

type ReadIntent =
  | 'precise-fact'
  | 'multi-source-synthesis'
  | 'time-interval'
  | 'preference-recommendation'
  | 'abstention-candidate';
IntentExamples
precise-fact"What is X's email?", "When was the last release?"
multi-source-synthesis"Summarize all topics", "Aggregate counts across sessions"
time-interval"How many days since X?", "In what order did Y, Z, W happen?"
preference-recommendation"Any tips for X?", "Can you suggest Y?"
abstention-candidatelikely unanswerable from evidence

Five reader strategies​

type ReadStrategyId =
  | 'single-call'
  | 'two-call-extract-answer'
  | 'commit-vs-abstain'
  | 'verbatim-citation'
  | 'scratchpad-then-answer';
StrategyCallsCost (illustrative)Description
single-call1$0.0150one reader.invoke call
two-call-extract-answer2$0.0280claim extraction + answer call
commit-vs-abstain2$0.0220binary commit/abstain + answer-or-refuse
verbatim-citation1$0.0170single call with verbatim-quote rule
scratchpad-then-answer1$0.0190single call with scratchpad scaffold

Three shipping presets​

PresetStrategy mixWhen to use
precise-fact (default)single-call for facts, two-call for synthesis, scratchpad for timebalanced workloads with mixed intents
synthesistwo-call for synthesis + preferences, verbatim for factssynthesis-heavy workloads (multi-doc Q&A, research)
temporalscratchpad for facts/synthesis/timetime-heavy workloads (timelines, scheduling)

Quickstart​

import {
  LLMReadIntentClassifier,
  ReadRouter,
  FunctionReadDispatcher,
} from '@framers/agentos/read-router';

type Answer = { text: string; citations: string[] };

// Stand-ins. Replace `openaiAdapter` with whatever LLM adapter your runtime
// exposes; replace each `myXxxReader` with the real reader strategy
// implementation, and `query` / `evidenceChunks` with the per-request inputs.
declare const openaiAdapter: any;
declare const query: string;
declare const evidenceChunks: any[];
async function mySingleCallReader(_q: string, _e: any): Promise<Answer>      { return { text: '', citations: [] }; }
async function myTwoCallReader(_q: string, _e: any): Promise<Answer>         { return { text: '', citations: [] }; }
async function myCommitOrAbstainReader(_q: string, _e: any): Promise<Answer> { return { text: '', citations: [] }; }
async function myVerbatimReader(_q: string, _e: any): Promise<Answer>        { return { text: '', citations: [] }; }
async function myScratchpadReader(_q: string, _e: any): Promise<Answer>      { return { text: '', citations: [] }; }

const router = new ReadRouter({
  classifier: new LLMReadIntentClassifier({ llm: openaiAdapter }),
  preset: 'precise-fact',
  budget: { perReadUsd: 0.03, mode: 'cheapest-fallback' },
  dispatcher: new FunctionReadDispatcher<Answer>({
    'single-call': async (q, evidence) => mySingleCallReader(q, evidence),
    'two-call-extract-answer': async (q, evidence) => myTwoCallReader(q, evidence),
    'commit-vs-abstain': async (q, evidence) => myCommitOrAbstainReader(q, evidence),
    'verbatim-citation': async (q, evidence) => myVerbatimReader(q, evidence),
    'scratchpad-then-answer': async (q, evidence) => myScratchpadReader(q, evidence),
  }),
});

const { decision, outcome } = await router.decideAndDispatch(query, evidenceChunks);
console.log(decision.classifier.intent);          // 'multi-source-synthesis'
console.log(decision.routing.chosenStrategy);     // 'two-call-extract-answer'
console.log(decision.routing.estimatedCostUsd);   // 0.0280
console.log(outcome.text);                         // final answer

Decision-only flow​

const { classifier, routing } = await router.decide(query, evidence);

if (routing.chosenStrategy === 'commit-vs-abstain') {
  // your custom abstain-aware reader
}

Manual intent override​

const decision = await router.decide(query, evidence, {
  manualIntent: 'time-interval',  // skip classifier
});

Few-shot classifier prompt​

const router = new ReadRouter({
  classifier,
  preset: 'precise-fact',
  useFewShotPrompt: true,
});

API surface​

Reader Router — reader-model selection​

Picks the best answer reader per query, dispatched per category, on top of whatever retrieval architecture MemoryRouter chose.

Validated on LongMemEval-S Phase B N=500 alongside MemoryRouter: 85.6% [82.4%, 88.6%] at $0.0090/correct, 4 second avg latency. Beats Mastra OM gpt-4o (84.2% published) on accuracy. Beats EmergenceMem Simple Fast (80.6% measured apples-to-apples in our harness — public reference repo ships with no LICENSE; not legally redistributable) by +5.0 pp at 6.5× lower cost-per-correct. Statistically tied with EmergenceMem Internal (their 86.0% point estimate sits inside our [82.4%, 88.6%] CI), but Emergence's 86.0% comes from closed-source SaaS at emergence.ai/web-automation-api — not a library you can install. AgentOS ships under Apache-2.0.

What it actually does​

When a query arrives, the MemoryRouter's classifier at Stage 2 has already produced a category prediction (one of six: single-session-user, single-session-assistant, single-session-preference, knowledge-update, multi-session, temporal-reasoning). At Stage 3, selectReader(category, preset) returns the reader tier for that category, and the caller dispatches the answer call to it. With the min-cost-best-cat-2026-04-28 preset:

predicted_category ──► reader_tier
  temporal-reasoning ──► gpt-4o     (long-context arithmetic + ordering)
  single-session-user ──► gpt-4o    (exact recall of user statements)
  single-session-assistant ──► gpt-5-mini  (shorter answers from assistant outputs)
  single-session-preference ──► gpt-5-mini (preference questions structure well in scratchpad)
  knowledge-update ──► gpt-5-mini   (current-state lookup)
  multi-session ──► gpt-5-mini      (cross-session synthesis from chunks)

Each reader gets the retrieved context from Stage 2 plus the question. selectReader() is a table lookup and makes no LLM call. The min-cost-best-cat-gpt5-tr-2026-04-29 preset returns gpt-5 instead of gpt-4o for temporal-reasoning and single-session-user.

Why route at all​

Two readers behave very differently on the same retrieved evidence. Per-category Phase B at full N=500 on the same retrieval stack (hybrid retrieval and Cohere rerank behind the minimize-cost policy router):

Categorygpt-4o readergpt-5-mini readerBest pick
temporal-reasoning (n=133)84.7%72.9%gpt-4o (+11.8 pp)
single-session-user (n=70)94.3%90.0%gpt-4o (+4.3 pp)
single-session-preference (n=30)63.3%86.7%gpt-5-mini (+23.4 pp)
single-session-assistant (n=56)98.2%100.0%gpt-5-mini (cheaper, ties or wins)
knowledge-update (n=78)85.7%87.2%gpt-5-mini (cheaper, ties or wins)
multi-session (n=133)76.2%79.7%gpt-5-mini (+3.5 pp)
Run score83.2% (416/500)83.2% (416/500)tied
Category rows weighted by n84.2%83.2%gpt-4o (+1.0 pp)

Both runs scored 416 of 500. The gpt-4o run recorded six cases without a category, all scored wrong; its category rates cover the other 494 cases (temporal-reasoning 131, multi-session 130, knowledge-update 77), so its rows weight to 84.2%. The per-category profiles are mirror images. Picking the better reader per category reaches 87.0% (435/500): +2.8 pp over gpt-4o's weighted 84.2% and +3.8 pp over gpt-5-mini. A run that routed on the gpt-5-mini classifier's real predictions, with canonical-hybrid retrieval for every case, scored 85.6% (428/500). The largest per-category gap is single-session-preference (63.3% gpt-4o → 86.7% gpt-5-mini at the same retrieval), and about half of the cases (53% in the measured run) route to the cheaper gpt-5-mini reader.

Calibration table​

The shipped table is MIN_COST_BEST_CAT_2026_04_28, derived from the Phase B per-category data above. For each category, the table picks the reader that produces higher accuracy. When accuracies are within statistical noise (single-session-assistant, knowledge-update), the table picks the cheaper reader (gpt-5-mini at ~12× lower per-token cost than gpt-4o).

import { selectReader } from '@framers/agentos/memory-router';

// `selectReader(category, preset)` looks up the right reader tier in the
// shipped calibration table and returns the model identifier ('gpt-4o' or
// 'gpt-5-mini'). The caller dispatches to the configured reader for that
// tier — typically the same reader instance the rest of the pipeline uses.
const tier = selectReader(predictedCategory, 'min-cost-best-cat-2026-04-28');
const reader = tier === 'gpt-4o' ? gpt4oReader : gpt5miniReader;

When the predicted category is missing or the preset name is unknown, the function throws ReaderRouterUnknownCategoryError / ReaderRouterUnknownPresetError (both exported from the same module). Use them to surface clear diagnostics in the calling pipeline.

Standalone-classifier mode​

selectReader() never calls a model. Without a MemoryRouter in the pipeline, the host gets the category from LLMMemoryClassifier.classify(query) (one gpt-5-mini-class call per query, ~$0.0001) and passes it in.

Alongside MemoryRouter (the typical config), the host passes the category from the MemoryRouter's decision, and the reader pick adds zero LLM calls.

Why the default classifier is gpt-5-mini and not gpt-4o​

Two independent Phase B measurements at full N=500 on LongMemEval-S confirm that upgrading the classifier from gpt-5-mini to gpt-4o does NOT lift aggregate accuracy on this benchmark, while costing ~12× more per query:

Configurationgpt-5-mini classifiergpt-4o classifierΔ
Tier 3 + reader router84.8% [81.6%, 87.8%]84.4% [81.2%, 87.6%]tied, gpt-4o classifier 87% more expensive
Canonical-hybrid + reader router (the 85.6% headline base)85.6% [82.4%, 88.6%]84.0% [80.6%, 87.0%]gpt-4o classifier −1.6 pp at +44% cost-per-correct

In both runs, the gpt-4o classifier reclassifies edge cases more aggressively, gaining marginally on SSU/SSA/SSP (always within CI) but losing meaningfully on KU (-3.9 to -5.1 pp) and MS (-5.2 pp on the second measurement) as edge cases get routed away from their gpt-5-mini-best dispatch tier.

The agentos-bench --om-classifier-model gpt-4o flag remains wired in for per-workload empirical testing — workloads with very different category distributions from LongMemEval-S may see a meaningful lift — but on this benchmark's category mix the recommended default is unambiguously gpt-5-mini.

Cost per case​

1. Classifier:     ~660 input + 10 output tokens   ~$0.000138/case
2. Dispatched reader (per case):
     ~47% gpt-4o   ~5K-8K in + 20 out                ~$0.0125
     ~53% gpt-5-mini ~5K-8K in + 20 out              ~$0.0010
   Average reader cost: 0.47 × $0.0125 + 0.53 × $0.0010   ~$0.0064/case
3. (Judge call out-of-band)

Per-case AgentOS LLM cost: ~$0.00768/case (measured: $3.84 / 500 = $0.00768)

vs the prior 84.8% Tier 3 + ReaderRouter headline at $0.0410/correct, dropping the policy router's MS/SSP → OM-v11 routing (which imposed 60-120 sec observer pipelines per OM-routed case) is 4.6× cheaper per correct, 5.3× faster avg latency, 15× faster on the p95 tail.

When to use ReaderRouter alone vs with MemoryRouter​

ScenarioUse ReaderRouter aloneUse ReaderRouter + MemoryRouter
Single retrieval architecture (canonical-hybrid only)yes
Need to dispatch between memory backends (canonical-hybrid vs observational-memory)yes
Question category breakdown matters but architecture doesn'tyes
Long-haystack scenarios where OM compression helpsyes
Sem-embed era LongMemEval-S (this benchmark's headline)yes (canonical-hybrid for all categories)(Tier 3 minimize-cost preset's MS+SSP → OM-v11 routing was calibrated on CharHash retrieval and is now stale; see Memory Router)