GMI versus agent() Study
This study compares a GMI on the full AgentOS runtime with the agent()-based reply path of a reference product, the companion chat of wilds.ai, on the same model. It reads both against agent() on its own and against a bare loop that calls the model with no framework. Generalized Mind Instances (GMIs) describes what a GMI adds over a plain agent, and Two API Paths describes the runtime and the lightweight helper the arms are built on. This page sets out the study's design and lists each report set once it is published. It states no measured number: measurements are published only in the report sets.
Status: design published; protocol 0a registered 2026-10-29; record-mode reports published.
The question
Does a GMI on the full runtime improve companion-style, long-horizon conversation over the agent() loop the reference product runs, on memory, responses, personality and text, and at what cost in dollars and latency per turn?
The outcomes the protocol defines are product decisions for the reference product:
- GO: the candidate wins on at least one primary endpoint, loses on none, passes every validity gate and meets the cost and latency ceilings, and the win is attributed to the GMI. GO is the decision to build a GMI factory into the reference product and to move its companions behind a flag and a canary.
- GO-COST: every condition of GO holds except the cost or latency ceilings. The fix comes first; then one more set of tranches per task measures the winning endpoint and the cost again, and the outcome is GO if they pass and KILL if they do not.
- PORT: a part of the GMI reaches the gain when it runs around
agent(). That part goes into AgentOS as an opt-in option, and no factory is built. - KILL: no gain large enough to pay for a migration in the tested configuration. A primary endpoint that is worse and stays worse after one rerun is a KILL as well.
- NOT SHOWN: no KILL condition holds, and GO fails only because some endpoints are inconclusive. One extension runs, sized for those endpoints, and the outcome is then GO or KILL.
- NO VERDICT: a validity gate cannot pass within the spending ceiling.
An axis whose judges fail calibration reads "no verdict" and leaves the decision. A task left with one judge supports only KILL or NOT SHOWN on its judged endpoints.
The arms
Three arms decide the question, and two controls are what the comparisons are read against:
| Arm | Role | Construction |
|---|---|---|
| B | the reference | The reference product's companion reply path, built on agent() with the product's own memory layer, driven in process with its production options by a private runner. |
| C0 | the GMI | A GMI on the full runtime, called through AgentOS.processRequest(), with B's model, B's memory configuration, B's system text for the turn and B's tool definitions. |
| C1 | the candidate | C0 with the five shipped sentiment presets on and two mood-keyed contextual prompt elements that carry the presets' adjustments into the prompt. |
| A | control | agent() without runtime: 'gmi', given the reference product's prompt and tools, with memory removed. |
| Z | control | The bare loop: one streamed chat-completions call per turn over fetch, with A's model, A's static system blocks and the turn's addon, A's window and limits, and no chain-of-thought block, tools, memory or framework. |
The other controls and diagnostics are A given the gold evidence (T1); R, the retrieval anchor, which runs the bench's retrieval over the ingested store and a reader on the subject model (T1); B with one frozen sentence that asks it to notice frustration or confusion, acknowledge it and change approach (T3); and two port arms, which run a part of the GMI around agent() (the GMI's rendered prompt and memory block, or a sentiment tracker and metaprompt executor in the host) on an endpoint the GMI wins.
What each comparison measures:
- C1 − B and C0 − B decide the question.
- C − B is what the GMI runtime adds over the reference product's
agent()-based path, which builds the same memory manager. - B − A is what that memory layer and the product's assembly add to
agent(). - C − A is the sum of the two and is reported as a description; on T1 it is the memory wiring check, not a runtime effect.
- A − Z is what
agent()with the reference product's tools, some of them inert, and its chain-of-thought block adds over one plain call. The study does not separateagent()'s loop from those. - Z is the floor every arm is read against. C against Z is read from absolute scores only: it is never judged in pairs, it is not measured on the pairwise axes, and it is never computed as C − A plus A − Z.
- Z and R vote in no decision rule.
What every arm shares
Every arm calls one subject model, which every report names. The shared system text is the reference's static system blocks and the turn's addon, the reference's system text for that turn without its chain-of-thought block and its retrieved-memory section. What each arm adds to it or lacks:
- B sends its own system text, its chain-of-thought block and its retrieved-memory section included.
- C0 and C1 carry the chain-of-thought block of the reference's static base and their own memory, which the runtime places in the user turn; C1 adds its two mood-keyed prompt elements.
- A carries the chain-of-thought block
agent()puts first when tools are mounted. - Z receives the static blocks and the addon as one system message, with no chain-of-thought block.
Every arm runs under a 60-second wall deadline per turn, and a reply cut at the deadline is judged as it stands. Every arm has a 6,000-token output cap. One function sets the sampling and reasoning fields of every arm's main calls, at the stub or the metering proxy, to the values in the reference's captured request. The same output guard checks every arm's reply. Every arm but Z mounts the reference's tool definitions, and every arm with memory has a store of its own; Z has neither.
Two asymmetries are recorded and reported rather than removed:
- History: C, A and Z keep a 60-message window; B, driven as production runs it, reloads its last 50 stored messages on every turn.
- Tool steps: a GMI turn stops its tool loop at 5 model steps, where B allows 12.