What agent-memory benchmarks actually establish

A memory score can describe an improvement and still leave the adoption question unanswered. We need to know what changed, which evidence reached which model, how outcomes were scored and what work was charged.

This companion preserves the experiments and source checks behind the main decision article. Use the market guide to identify product interfaces. Historical papers, current services and source-code snapshots are separate evidence surfaces.

Evidence cutoff: 29 October 2026. Mutable sources retain capture/version qualifications. The work here comprises source reading, static artifact/count arithmetic and two isolated source-function probes. No complete product/model efficacy benchmark, deletion race or cross-tool migration trial was executed. Published performance belongs to the cited authors. Atlas and its proposed acceptance tests are invented fixtures.

Atlas is a hypothetical project with an adopted Python 3.11 target, a proposed future move to 3.13, a corrected migration date and a retracted approval. Its rare compatibility requirement and the separate project Boreal test retention and scope. The full chronology appears in the correction section [4].

Which experiment are we discussing?

Write down the comparison before reading its headline. “The same benchmark” leaves many consequential choices open.

The sections below use these distinctions to read the cited comparisons. Their consequence is a testable adoption question, rather than a universal architecture winner [2, 5, 76, 90, 103].

Define what is persisting

Several different things are called memory. Model parameters contain learned information. The current request contains working context. A database holds application state. A conversation store preserves messages. A checkpoint records a conversation or execution position. An external memory system selects retained information for later inference or action [6, 8].

One position paper draws a further boundary: external facts, rules and code can improve an agent while the generator’s model weights remain fixed. It calls that contextual state a memo rather than adaptive memory. Its proposed generalization gap depends on an explicit remaining-error assumption; it does not establish that retrieval cannot generalize or that useful memory always requires weight updates [122].

These lifetimes matter. Pydantic AI, for example, separates a serialized conversation, settled execution-step information, searchable history and a cross-conversation notebook. Persisting a support ticket and retaining a standing customer preference require different keys and policies [8].

Prompt caching is another mechanism. It reuses computation for eligible repeated input. The application still has to supply the right material. A cache cannot decide that Atlas's cancelled migration should stop appearing as current guidance. Conversely, an API being stateless does not establish that the provider retains no data or that every supplied token is recomputed at the ordinary price [6, 7].

For external memory, follow this sequence:

Observe → admit → retain → revise → select → assemble context → answer or act.

A system can combine stages or leave them to application code. A file-based profile may use explicit human edits for admission and revision. A managed service may run extraction and consolidation in the background. Both eventually have to place useful information in front of the answering model [2, 48].

The failure location determines the repair. If Atlas's correction never enters storage, improve capture or admission. If it is stored but never returned, inspect retrieval. If retrieval returns it but prompt assembly drops it, inspect truncation and formatting. If the final context contains the correction and the agent still chooses 3.13, investigate the reader or execution policy. Adding another index will not necessarily repair that last failure [2].

Representation changes the easy operation

A memory representation is a decision about which later operations should be easy.

These designs can coexist. A relational store can hold vectors. A filesystem interface can sit over semantic retrieval. A graph can retain original passages alongside derived facts. SQL gives explicit filters and joins, but neither its syntax nor a flexible agent tool establishes authorization. The existing SQL investigation remains practitioner testimony, rather than a controlled SQL-versus-vector result [6, 68].

For Atlas, a small current-state record might be enough for deployment. The transcript is useful when someone disputes the date. A lesson about diagnosing the dependency may help on the next migration. Those assets have different readers and retention obligations.

Preserving an original for verification is also a separate decision from showing it to the executor. Where retention is permitted, keeping the source makes later correction and reinterpretation possible. A short lesson may nevertheless be the better context for completing the next task. The experiments below support testing both choices [3, 66].

Benchmarks ask different questions

A score belongs to a dataset revision, reader, retrieval schedule, prompt, budget, scorer and failure convention. A shared benchmark name leaves all of those free to differ.

ATANT proposes continuity checks beyond transcript questions, including persistence across sessions, updates, temporal order, disambiguation, reconstruction and operational usefulness. Those are useful requirements for Atlas. Its companion evaluation scores only its own reference system and uses a partial LoCoMo run, so its cross-benchmark contrast cannot rank competing products or replace a matched comparison [123].

The inspected LoCoMo copy contains 282 multi-hop, 321 temporal, 96 open-domain and 841 single-hop questions, plus the 446 adversarial ones. Removing that last category removes an abstention challenge. Generated retrieval summaries are different from annotated event-summary targets; captions and image URLs are supplied without the underlying images [116, 71].

LongMemEval retrieval evaluation omits its 30 abstention cases because they lack answer-bearing locations. The original temporal rubric permits off-by-one duration errors, and update judging can accept an old fact alongside the required new one. Those are benchmark definitions, rather than exact interval or stale-fact-erasure guarantees. V2's captured query interface excludes benchmark metadata; it does not establish whether historical scores exploited such metadata [2, 72, 73].

History size is another condition worth preserving: the original LongMemEval S problems contain approximately 115,000 tokens, while M uses 500 sessions and around 1.5 million tokens. Whether a full-history baseline fits depends on the variant and model; neither the benchmark name nor a larger advertised window settles usable performance [2].

Agent-Native Memory's eleven-system comparison adds direct update and temporal slices. Its configured Zep leads knowledge update at 44.4 substring exact match and 36.8 ROUGE-L F1. These are author-reported overlap scores for Zep Local, not exact current-value accuracy or today's hosted service. Complete per-system execution settings remain unresolved here; MemTensor-affiliated coauthors also limit its independence as MemOS corroboration [105].

An agent must eventually do something

MemoryArena's inspected 6cd9de14 adapters make comparison boundaries concrete. RAG chunks are nonoverlapping, at most 2,048 tokens, with top-three retrieval. The long-context adapter retains the newest 120,000 tokens. The hosted Mem0 adapter calls V2 without pinned internal embeddings or service snapshot. In the shopping loop, retrieval happens once per episode/subtask; step writes do not trigger another search [77, 78, 79, 80].

These details suggest an action-conditioned retrieval experiment. They do not establish that cadence caused a published score, or that the inspected preview code exactly matches every historical run.

SWE-ContextBench makes the gap between available and selected guidance visible. In its 99-task Lite comparison, Oracle Summary resolves 34.34%, no context 26.26%, and freely selected summaries 22.22%. The oracle knows task relationships, so it diagnoses useful available guidance rather than demonstrating a deployed retriever. The experience pool is not restricted to successful solutions, and patch-application exclusions remain part of the source's setup [75].

History growth needs its own test. Keep relevant evidence fixed while adding unrelated accessible sessions. When Stored Evidence Stops Being Usable separates wrong answers within budget from excessive visible memory calls, applying budgets after completed rollouts. Backend traversal/indexing are excluded, and its 2,000-query LongMemEval population is unreconciled with the canonical 500. Use the intervention as a testing idea without importing its ranking [106].

Compact memory and stronger baselines

The Mem0 headline describes a tradeoff

In the historical Mem0 paper, full context is more accurate on the overall judge measure, while compact memory uses less context and responds faster [1].

P95 is the 95th percentile of response time. These are the paper's measurements, not a current product comparison. Its roughly 26% relative improvement names a separately constructed OpenAI-memory comparator scoring 52.90, rather than the full-context arm or current ChatGPT. Mem0 also exceeds its tested best RAG result, 60.97, but that baseline explores only one or two chunks rather than a strong hybrid/reranked retrieval setup. The graph improves some categories and reduces others, including single- and multi-hop scores [1].

The result supports a useful quality/context/latency tradeoff. It cannot establish that every application needs extracted memory or that smaller answer context lowers the complete bill.

Stronger simple baselines matter, and so do their limits

MemDelta changes embeddings within fixed RAG and reports accuracy rising from 47.2% to 53.4% over 500 questions, with a paired McNemar test p=0.004. That is direct evidence that a component choice can move a score otherwise attributed to architecture [76].

Its Mem0 comparison is narrower: 88 valid cases from 100 attempts, only user-fact and multi-session types, and approximately matched embeddings. Mem0 scores 72.7 versus cloud RAG 73.9. The reported 90% difference interval spans −10.8 to +8.4 percentage points. Failure to detect a difference is not proof of equivalence. Successful-run accuracy also omits the failed attempts; the five-instance cost pilot is a separate population [76].

MemoryTax supplies another useful control. It tests one LoCoMo conversation with 152 non-adversarial questions and one accepted run per method/model cell. Mem0 OSS 2.1.0 and Graphiti OSS 0.30.2 are historical configurations. With a fixed GPT-4o-mini-written Graphiti store, even the Opus 5.5 reader scores 45%; reading the Opus-written store, it scores 87%. Writer-dependent retained/retrieved evidence is a bottleneck in this setup, rather than a demonstrated universal extraction ceiling [117].

Agents uniquely receive photo captions. With Opus 5.5, Pi scores 99% against full context's 95%; on 88 questions without photo evidence, they score 100% and 99%. That subset is not a separately costed, fully matched-source experiment. Reported whole-rollout costs of $6.95 and $17.84 include construction/read model and embedding charges, exclude judging and depend on cache use. Failed or answerless agent trials are retried, with only accepted trials counted. The all-attempt bill remains unknown [117].

A benchmark critique also needs an audit

The LoCoMo audit identifies real reference errors. A dialogue places a race on Saturday while its answer key says Sunday. Another's dates imply a nine-day trip where the reference gives 19. Released full-context verdicts reject some source-correct answers. That matters when comparing small leaderboard margins [5, 81].

The audit's aggregate headlines need qualification. Its 99 non-citation flags include ambiguity and disputed wording. Its crafted-answer acceptance statistic, 2,902 accepted judgments out of 4,620 or 62.81%, includes correct paraphrases alongside less-specific answers. It is not a wrong-answer false-positive rate. Its asserted 93.57% ceiling also fails under the actual evaluator, which sometimes accepts source-correct answers despite defective references [5, 82, 81].

The auditor reports full-context GPT-4.1-mini scores of 81.95% under a short-answer instruction bundle and 92.62% under seven-step chain-of-thought instructions without a word limit, a derived 10.67-point difference. That is prompt sensitivity in its setup, not a length-only intervention. Three judge repeats are not three independent answer-generation runs [114, 83].

Historical upstream prompt/data identity was recovered at commit 1f2f083d9fd07fd6580064bbdfc7b97da39c47bb; the actual adapter answer constant matches after trimming. Category exclusion, model aliases and mean aggregation align. Retry policy, request caps, formatting and unavailable provider snapshots still prevent operational equivalence. The audit's 92.62 versus historical repository 92.32 is therefore useful configured-system evidence; the paper-v2 93.05 remains a separate result [83, 84, 85, 5, 86].

EverMemOS's fixed-reader/prompt architecture ablations remain positive conditional evidence. Its human panel reports kappa 0.891 and accuracy 0.984 on 25 questions with five raters each, judging reference-based agreement. Agreement with a reference does not independently establish that the reference matches the source conversation [86].

Keep both findings: memory architectures can help under controlled conditions, and benchmark configuration or faulty references can change an apparent winner.

Where added mechanisms help

Retrieval-augmented generation, or RAG, selects stored information and supplies it to a model. Lexical matching is useful for exact versions, identifiers and error strings. Dense embeddings find related meaning and paraphrases. Structured filters constrain fields. Graph traversal follows represented links. File tools let an agent expand the context around a hit. Hybrid systems combine these signals before ranking, deduplication and truncation [6, 68].

For Atlas, the nearest sentence about Python 3.13 may be the wrong one. The useful context may need the proposal, approval, correction and retraction together, plus the mobile-client requirement. Returning one attractive match is different from returning enough evidence to answer.

LongMemEval provides a constructive middle ground between raw history and extracted replacements. Its identified M/round-value comparison expands retrieval keys with facts while retaining original values: Recall@10 rises from 0.692 to 0.784 and GPT-4o top-ten question-answering from 0.670 to 0.720. A better key can help find original context without discarding it [2].

The reader changes the useful budget. The same study reports an 8B reader degrading beyond roughly 3,000 retrieved tokens while GPT-4o continues improving beyond 20,000. Even known supporting evidence supplied directly to the reader produces different results under different reading strategies. An advertised context window is a capacity limit, not a measured guarantee that every included condition will be used [2].

Elastic's reference design preserves verbatim user turns and pre-recalls on the original text before query paraphrase. It combines lexical and dense candidates, fusion and reranking. Its reported Recall@10 of 0.89 on 168 generated corpus-specific questions and zero observed tenant leaks concern that retrieval configuration; they are not a downstream action score or a universal isolation guarantee. Some described guards and ranking features remain unwired in the demonstration [54].

Graph structure can add value

HippoRAG 2 links passage and phrase nodes, filters retrieved triples and uses Personalized PageRank, a graph walk biased toward chosen seeds. The reader receives the top five original passages. Its seven-dataset Llama-3.3-70B comparison reports average question-answer F1 of 59.8 against 57.0 for NV-Embed-v2. The corpus-expansion experiment supports that configured retrieval approach, while associative performance still declines as distractors grow. It does not establish conversational correction or deletion behavior [34].

GAAMA v2 retains original episodes alongside facts, concepts and reflections. Over ten LoCoMo conversations and 1,540 non-adversarial questions, it reports 79.1 against 74.9 for tuned raw-turn RAG, with matching embeddings, GPT-4o-mini generation, prompt and 1,000-word context budget. The measure is fractional reference-information coverage, not binary correctness; extra information is unpenalized. A same-store ranking ablation gives 77.8 for semantic retrieval and 79.0 with weak PageRank, separating a graph-ranking contribution from memory construction [118].

GAAMA's GRAFT repair triggers on 47 of 1,540 questions. Reward rises 51.8→54.6 on that subset, but 79.1→79.2 overall, with temporal regressions. Ordinary retrieval uses no LLM call; repair does. Its MemoryArena adaptation ingests gold answers after scored steps, which narrows claims about autonomous action transfer. These are unreplicated author results without a complete lifecycle cost comparison [118].

A lesson can be better than an unedited trajectory

An execution trace includes observations, actions and often a great deal of irrelevant detail. A lesson distills a reusable strategy. A task-conditioned curator reads prior experience and prepares guidance for the current problem. Preserve permitted originals for verification; choose the executor's input according to task utility.

Decocted Experience fixes its main Seed-OSS-36B-Instruct executor and problem-statement encoder, then compares top-K raw experience with offline lessons. K is eight for math and four for WebShop/software engineering. Lessons outperform raw context on the action workloads; raw is slightly stronger on math. The main effects are graphical, so no precise effect size is inferred here [3].

Its lessons see four attempts, including failures, while each raw entry retains one successful trajectory. Equal K is therefore neither equal source information nor equal prompt tokens. Offline collection and lesson construction are also outside a complete equal-dollar comparison. Those limits prevent attributing the gain entirely to compression; they do not erase the positive lesson-pipeline result. Moderate consolidation can help, while aggressive reduction hurts [3].

SkillOS trains a curator to insert, update and delete Markdown skills while freezing the executor. With its Qwen3-8B ALFWorld executor, reported average success is 61.2 versus 53.1 for the untrained same-interface variant and 55.7 for ReasoningBank. Removing content-quality reward gives 58.6; removing task grouping gives 57.3. That supports learning which skills will help future tasks in this setup [69].

The content-quality reward comes from Qwen3-32B, not independently verified truth. Retrieval is static top-five BM25 over simplified skills, with no matched raw-trajectory arm. Training uses 16 H100 GPUs for approximately three days on ALFWorld and five on WebShop; that hardware duration is not a customer bill. SkillOS and Decocted Experience share authors, so their agreement is not independent replication [69, 3].

JITMEM makes a different choice: retain judged-successful raw trajectories, retrieve task descriptions, then synthesize an ephemeral payload for a frozen executor. Its GPT-5.4 WebShop untrained configuration falls from 39.3±0.7 to 34.8±0.8 without task conditioning and to 31.1±1.0 with the tested distilled-store replacement. This supports source detail and conditioning within that design. A matched direct-raw executor comparison is still missing, so the result cannot establish that every curator call is necessary [66].

For a repeated engineering task, compare raw traces, offline lessons and task-conditioned guidance under equal source access. Add a curator without retrieval and a generic-summary control to distinguish experience transfer from simply giving the model another planning call. Then score the completed task, including the cost of preparing the advice [103].

Hindsight’s technical report is another example of separating an architecture from the comparison it establishes. It distinguishes world facts, the agent’s experiences, synthesized observations and opinions, with four-way retrieval and reflection. The factual-QA experiments use neutral behavioral profiles and leave retrieval-token budgets as placeholders. They do not establish stable personalized beliefs or a complete equal-budget market comparison [125].

Corrections require the consumer query

The proposed Atlas history

Here is Atlas’s proposed test history. On 1 September, its maintainer records Python 3.11 as the deployment target. An older mobile client must remain compatible. A separate project, Boreal, targets Python 3.10.

On 10 September, a colleague proposes trying Python 3.13 if a dependency check succeeds. On 15 September, an authorized maintainer schedules the migration for 1 November. On 20 September, the maintainer corrects the date to 15 November. On 25 September, the migration approval is retracted. In this example, that leaves the established 3.11 target in place.

Those inputs imply different questions:

The dates are invented. The distinctions are the proposed requirements. Someone choosing a system can decide that only today's target matters, or that earlier beliefs and future schedules matter too. Those are different promises [4].

Begin with four controls. With no persistence, the next session lacks the earlier evidence. With complete history, the model can inspect the sequence if it all fits, but must interpret it correctly. A maintained profile can state the current target directly; arbitrary historical questions need more than its latest value. Retrieval preserves a larger archive while introducing another failure: finding the proposal without the correction [2, 4].

For a coding assistant, an authoritative runtime setting may already belong in a versioned project file. Memory can point to that file. A rule that must always apply also needs the appropriate technical enforcement; remembering it is weaker than enforcing it [60, 62].

This gives the first selection rule: choose the simplest persistent control that can satisfy the actual questions. Add machinery when you can identify the operation it needs to perform.

Effective dates and recorded beliefs

There are two clocks in Atlas's migration. One says when the change applies in the world. The other says when the system learned or recorded a particular version.

On 17 September, the team believes migration is planned for 1 November. On 20 September, it learns that the effective date should be 15 November. Asked on 22 September about 5 November, the assistant should use the corrected schedule. Asked what the team believed on 17 September, it should preserve the earlier plan as a belief at that time. The wrong date remains an auditable assertion, rather than becoming a true historical migration [4].

Valid time describes the world interval. System or transaction time describes the recorded version's interval. A proposed two-clock query for world date v, as known at recording date t, selects:

valid_from <= v < valid_to
recorded_from <= t < recorded_to

Open ends can be unbounded. Scope and authority still have to apply. A correction closes the prior recorded version and adds the corrected one. This is an application contract for the example, not a claim that every memory API implements it [4].

Cardinality matters too. Atlas can have several supported test runtimes while having one deployment target in a given environment and interval. Matching endpoints and a relation name cannot decide whether a new value adds to a set or replaces a current value. The inspected Graphiti PR1729 proposal narrows contradiction candidates by structure but records no cardinality. Its USES/RENDERS and USES/MIGRATED_OFF examples expose the remaining intent question; deployment of that proposal was not established [112].

Revision history provides a different facility. Google's documented immutable revisions and rollback offer concrete audit and recovery controls, with a default revision lifetime of 365 days and access to child revisions for up to 48 hours after deleting a parent. Those snapshots do not by themselves establish correct world-time answers [45].

Come back later and ask a different question

Atlas's maintainer corrects the migration date. The assistant acknowledges the correction. That exchange looks reassuring, but it has not tested the next session, the background summary or the retrieval query used by a different task.

Carlos Tabosa's discussion comment points to that missing regression: change a fact, then test later in a different context to see whether the stale version returns. It is a proposed test, not a measured vendor failure. The useful distinction is the property being checked. An action can be authorized while still depending on obsolete information; conversely, recalling the right date does not establish permission to deploy [120].

Here is how to turn the comment into an Atlas fixture:

1. Record the authorized 1 November plan and demonstrate that the assistant can recall it.
2. Correct the date to 15 November, then wait for the system's documented completion condition.
3. Open a fresh session. Ask for the release checklist for 5 November, without repeating the correction in the question.
4. Ask a different task, such as preparing a deployment command. Inspect retrieved evidence, final context and the proposed action.
5. After retracting the migration, repeat the task and check that Python 3.11 remains the deployment target. Ask separately what the team believed on 17 September; that historical question should still recover the earlier plan as an earlier belief.

The test should reject a stale assertion used as current guidance. It should not demand that all historical evidence disappear. Run authorization checks alongside it, including an unauthorized attempt to change the target. Correction, history and permission are three observable promises, even when one implementation helps enforce all three [4, 16].

The read path changed the interpretation of a code finding

At Graphiti commit e9e4a483d0c7995ed865bfd98143d1a71d5f3886, an isolated probe extracted two temporal functions from the Python source. With fixed time and simple fixture objects, four predeclared checks passed: preserve a future end, expire a past-ended assertion, avoid extending an existing finite end, and clip an overlapping future end. A fifth observational case showed an older candidate receiving a future valid-time end while its transaction expiration became now [4, 15].

It would be tempting to conclude that the old fact had disappeared too early. The consumer path did not support that conclusion. Default search supplies empty temporal filters. An explicit valid-time query can keep the old fact eligible until its future end; a query requesting expired_at IS NULL selects a different view. Default relevance search can also return retired assertions [16, 67].

A second constructor-only probe produced four output rows and passed three explicit assertions about filter generation. Neither probe ran a database, complete contradiction resolver, ranking pipeline or answering model. Open PRs 1867 and 2052 were outside the latest listed v0.31.2 release; inspected search files were unchanged between the release and the later pin. None of this establishes managed Zep behavior [4, 16].

The result is narrower and more useful: a mutation finding needs the consumer query to explain its effect. Temporal fields make views possible; the application still has to request the intended one.

Admission, control and deletion

Writing means interpreting an event

Atlas's messages contain at least five operations: add a supported test runtime, propose a possible change, approve a future replacement, correct a mistaken date, and retract an approval. Treating every new sentence as another current fact makes the application reconstruct those operations at read time. Overwriting with the newest sentence can lose their meaning instead.

The historical Mem0 paper describes extraction from recent messages and a conversation summary, retrieval of related memories, and ADD/UPDATE/DELETE/NOOP decisions. Its October hosted V3 reference describes asynchronous inferred-add responses/events as ADD-only, with separate explicit update, delete and history endpoints. The latter does not mean the whole service cannot mutate memory, or prove that its background internals never reconcile [1, 10].

Identifier semantics can break an otherwise sensible query. In the captured Mem0 reference, inferred facts are attributed to a user or agent according to speaker. Requiring both identifiers with an AND filter can return nothing; direct imports differ. That is a scope-query problem, not necessarily failed extraction [10].

Background work adds another distinction: accepted versus completed. A successful request may acknowledge queued processing. Microsoft's memory integration explicitly says zero batching delay does not guarantee immediate recall. Check the actual completion signal, stored result and subsequent context before calling an early miss a retention failure [10, 11].

A malformed response is not a lost fact

Write failures include omitted facts, unsupported inferences, mistaken merges, duplicates, malformed output, persistence errors and delayed processing. Their denominators differ.

Yes, GPT-4o-mini already sounds like AI archaeology. These results are still useful for distinguishing a recoverable formatting error from a lost fact.

Anatomy of Agentic Memory reports recoverable format deviations requiring fallback parsing. With GPT-4o-mini and Qwen-2.5-3B respectively, Nemori reports 17.91% and 30.38%; SimpleMem reports 1.20% and 4.82%. Those rates belong to the tested systems and models. They don’t tell us how often current products silently lose information [104].

The investigation inspected the paper's official repository at commit 6431dea and found a README-only snapshot. That bounds the available reproduction route; it does not establish that no other artifacts exist. The details and original limitations remain in the linked research notes [103, 115].

A successful write can also preserve the wrong kind of evidence. A tool returning without error does not establish that its larger task succeeded. Before converting an earlier trajectory into a procedure, record the outcome and the conditions under which it was verified [9].

Admission can be worthwhile even when summaries are risky

JITMEM retains trajectories judged successful before producing task-conditioned guidance. Its reported GPT-5.4 WebShop untrained-curator success is 39.3±0.7; retaining all outcome-labeled trajectories instead of filtering successes yields 35.9±0.5. That supports admission in that pipeline. It does not establish that failed experience is useless: Decocted Experience uses failed attempts when deriving lessons [66, 3].

The attached Truth-Maintained Memory Agent manuscript supplies another configured case. It combines planning, selection, verification, curation and response generation, routing disputed records into a flagged tier normally excluded from retrieval. The authors sample 100 conversations per model from each of MultiWOZ 2.4, SGD and Taskmaster, injecting one false memory per conversation from a 400-fact bank [119].

Reported false-memory rates are 6.8%, 10.5% and 7.6%, versus Embedded RAG's 52.3%, 55.7% and 46.2%. FMR concerns false information in responses, not a direct false-record admission count. The backbone, scorer/denominators and numeric budgets are unspecified; there are no component ablations or uncertainty estimates. Separation between the verifier's known-false dictionary and the evaluation bank is undisclosed. The positive integrated result merits testing, while the artifact cannot isolate a write-time effect or establish lower lifecycle cost [119].

Its quarantine retains content for audit. It supplies no demonstrated guarantee of historical truth, recursive correction of derived records or erasure [119].

Who may change what the agent believes?

Authorization determines who can read or write a record. Query scope determines which records a request considers. Provenance identifies source and transformation. Authority determines what weight that source receives when statements conflict.

A lifecycle security survey proposes authorized writes, visible source lineage, retrieval restricted to the current principal, recoverable state and explicitly bounded forgetting. This is a useful vocabulary for the contract. Provenance does not establish truth; a finite forgetting test supports only its declared storage, access and exposure boundary [121].

An episodic-memory position paper also makes the tension concrete: past-action records can help monitoring and explanation while enabling unwanted retention, deception and unpredictable later use. It advocates interpretable, user-controlled and detachable records, including restricting agents from editing their own memories. That last restriction is a governance proposal to weigh against self-editing designs, not a measured universal safety optimum [124].

Honcho explicitly says workspace credentials can read sessions outside a named recall scope. Neo4j similarly warns that a conversation identifier is not authorization. Elastic adds cluster-level document security alongside application filtering. Enforce authenticated identity below queries the agent can construct [22, 55, 54].

For Atlas, a colleague's proposal and a maintainer's approval should retain their source roles. A cited sentence can be faithfully quoted yet still be mistaken, obsolete or unauthorized for the recipient. Cognee's architecture distinguishes writer stamps, ownership, ledger, access projections and chunk evidence, with uneven coverage across enrichment, session bridging and code paths. Inspect the transformation you rely on [26].

Persistent poisoning can enter through different routes. AgentPoison assumes partial store-write access, white-box embedding access and a later query trigger. MINJA instead uses ordinary query interaction to induce reusable agent-generated records, under shared-memory or identity-compromise assumptions. Hidden in Memory conditions its attack-utility measure on the poisoned memory reaching context; its semantic-retrieval end-to-end rate is estimated. These are controlled threat models, not observed prevalence in commercial deployments [107, 108, 91].

A proposed admission test should ask whether low-trust conversation can become a standing instruction or a cross-user demonstration. Generated records should retain source trust under the tested policy. The policy and test remain proposals, not proven defenses.

Deleting a record is the start of a contract

Suppose Atlas's conversation must be erased. The target may still exist in a profile, summary, lesson, vector entry, prior answer, audit history or an unfinished consolidation job. First decide which of those are in scope and what completion means.

Removing an addressed object, making it inaccessible, deleting its derivatives and preventing an old writer from recreating them are distinct achievements. AgentCore documents that expired records may remain readable during background deletion, typically for days. Its source-event and long-term record APIs are separate; a source delete is not established here as a cascade through every derivative [92, 43].

AfterDeleteBench deliberately retains a deterministic summary containing source text after deleting the source record. Its primary Qwen experiment reports exposure-attributable recovery of 0.680 on 200 IID profiles and 0.640 on 200 held-out-generator profiles, with zero recovery in clean controls. This establishes the modeled channel in a synthetic harness, not commercial-product incident rates. Closing recorded lineage removes that channel; parser mismatch, component reconstruction and saturated utility checks prevent upgrading the result into universal nondisclosure or realistic utility preservation [93].

Cognee v1.6.6 makes the promises concrete. Memory-only forgetting intentionally keeps raw sources so the system can rebuild. Explicit session deletion retains already-distilled lessons. Some forget paths continue after best-effort cache cleanup; explicit session deletion propagates relevant errors for retry. Regeneration can be correct for a rebuild contract while violating a stronger source-erasure promise [94, 95].

There is positive coordination evidence. Session/turn and dataset locks cover participating improve, pipeline/background and concurrent-answer paths, but only within one process. The inspected sequential completion function delegates without the whole-turn lock, and SessionManager generates before adding the answer record. Invalidation follows recorded QA → feedback → lessons → dependent QA to a fixpoint, with exclusions for traces lacking element IDs, append-only tapes and some legacy attribution discovery. No resurrection race, restart or authorization bypass was executed in this investigation [96, 97, 98, 113, 99].

That boundary suggests a finite test: pause a writer after reading an old source, delete the source, then let it attempt to commit. Restart, retry and replay as required by the promised deployment. Durable deletion checks at commit or coordinated quiescence are possible designs; neither is mandatory for every application [99].

The person remembers across tools, too

Now move from Atlas's agent to its maintainer. They settled the runtime in one assistant, investigated the dependency in another and wrote the release checklist in a third. Each tool can remember its own sessions while the person still has to reconstruct the decision between them.

Carsten's comment describes that fragmentation and proposes a user-side archive through his Lingera project. It is a builder's observation and product direction, not evidence that every competing product fails or that his proposed solution is complete. The question it raises is valuable: does the person want a place to search their history, or should each supported tool receive maintained guidance automatically [120]?

Those are different product promises:

The cutoff documentation already describes parts of this space. Honcho advertises cross-tool integrations; Supermemory's service notice says API/MCP/plugins continue; OpenMemory ports supported coding sessions; Claude offers experimental memory import/export. These establish integration or transfer routes, not universal synchronization across every consumer assistant. Claude explicitly warns that its import extracts key information and may omit or fail to incorporate material [23, 19, 40, 109].

For Atlas, a useful proposed demonstration is to record a decision in tool A, retrieve it through tool B, correct it through tool C and return to A with a new task. Then revoke B's access. Inspect what each connector read or wrote, which source remains authoritative and what was already copied. A transcript export can preserve the words while losing the current-value relation or access policy. A live revocation also cannot erase an uncontrolled earlier export [99].

A shared memory service may reduce duplicated state where tools support it. It also becomes another dependency whose outage, access policy and exit route need a test. Keep the authoritative project record identifiable and make every claimed synchronization boundary explicit. “One memory” is a useful goal only after the supported tools and consistency promise are named.

Moving memory means moving its meaning

An export should identify object types and counts, ownership, source versions, corrections, historical views and skipped records. Text alone can preserve every word while losing which assertion is current or who may read it.

EverOS's recoverable Markdown source provides a reconstruction design, not evidence of equivalent answers after moving to another product. Claude's documented experimental import extracts key information and can omit non-work personal details or fail to incorporate material. Hindsight issue #5419 reports recallable legacy observations omitted from export; their creation was not reproduced on 0.10.3, and the closing collaborator cited a v1.1.0 repair. Keep the historical/repair qualifiers rather than projecting a universal current defect [32, 109, 110].

Compare supported, authorized answers before and after import. Pre-delete external archives have their own owner and retention policy. A live-service delete cannot erase an uncontrolled old export. Post-delete migration still needs tests for hidden derivatives and stale jobs [99].

Full accounting

Memory can move work out of the answer request into extraction, embeddings, consolidation and refresh. It can add retrieval, reranking and curation. The reader still has input/output costs. Storage, retries, background jobs and operations remain part of the workload. Avoid charging internal model work twice when it is already bundled in a service fee [90].

Memori’s paper illustrates the same accounting boundary. It builds semantic triples linked with summaries, then supplies compact retrieved context. Its comparator accuracy/token rows come from earlier papers, and its cost table prices added context rather than the full construction-and-serving lifecycle. That supports examining context efficiency, without establishing a matched end-to-end cost win over every alternative [126].

EverMemOS's GPT-4.1-mini LoCoMo accounting reports 9.42 million construction tokens, 4.45 million search tokens and 5.82 million answer tokens. Construction plus search is a derived 70.44% of the 19.69 million deployment-stage token total. That is a token share, not a dollar or compute share; input/output/cache prices differ, and evaluation work has its own boundary [86].

Total Recall usefully measures ingestion, retrieval and answering, but cost comes from synthetic conversations while accuracy comes from 665 non-adversarial questions over four LoCoMo dialogues. Dividing those populations does not produce measured dollars per correct task. Hindsight ingestion did not follow the intended model grid, and one full-history cell stopped at turn 374. A bounded paper and GitHub repository-metadata search did not locate the relevant harness/logs; code and author sites were not exhaustively searched [87, 90].

A cached baseline can reverse the calculation

Measure actual cache classes for both systems. Under the captured OpenAI contract, ordinary input, cache reads and cache writes are mutually exclusive pricing classes; a write is not an additive surcharge. Anthropic's input_tokens excludes cache-read/write classes while OpenAI's captured total includes them. Adding identically named counters across providers can double-count or omit cost [7, 89].

Let O be incremental construction, writes, corrections, refresh, storage and operating overhead relative to the baseline. Let d be net saving per useful read after retrieval and actual caching. At the same required quality, monetary savings need:

useful reads × net saving per read > incremental overhead
N × d > O

If d is nonpositive and overhead is nonnegative, more reads never repay the path. With frequent changes, count useful reads before the affected representation changes. Total traffic across continually rebuilt versions can grow without improving the economics [90].

Here is an illustration with invented inputs, not a vendor prediction: 20,000 source tokens; ordinary input at $1/million; one cache write at 1.25 times ordinary input and later reads at 0.1 times; $0.05 to construct memory; $0.0005 retrieval per read; and an uncached memory payload. Common question/output costs cancel. Other costs and quality differences are zero only inside this model [90].

The hot baseline costs 0.023 + 0.002N; the 500-token memory costs 0.05 + 0.001N. Their strict crossing is N > 27, or N > 77 after the extra refresh. The cold 2,000-token case first wins at three reads. Cacheable compact memory, misses, partial updates and output differences all change those thresholds [90].

Repeating a ten-read version repeats the deficit: $0.060 versus $0.043 in this illustration. Real corrections may update a small scope rather than rebuild everything. Conversely, a more expensive system may be worthwhile because it alone meets the required historical, latency or control behavior. Cost comparison starts after that qualification [90].

The market guide’s tariff reference preserves captured customer charges and their units. It is separate from the historical experiment accounting above.

The proposed test specification

Begin with the job, a serious baseline and a failure that would reject the added mechanism.

For Atlas, retain one stable rare constraint, the conditional proposal, authorized schedule, correction, retraction, Boreal and an unanswerable control. Run the questions before processing, after durable completion and in a fresh session. Include both ingestion orders where that tests historical handling. Keep the exact selected evidence and final context, rather than only the delivered answer [4, 2].

Freeze the source versions, chronology, reader/provider, prompts, budgets, readiness and scorer before comparing. Cross writer and reader models; equalize captions and other source fields. Distinguish gold-history ingestion from autonomous rollouts. For procedural reuse, compare raw traces, lessons and task-conditioned guidance, plus no-retrieval-curator and generic-summary controls. Measure verified action completion and charge the extra calls [117, 118, 3, 66, 103].

For admission, disclose separate verifier dictionaries and evaluation facts. Ablate screening and flagged-read exclusion independently with the same executor, scorer and budget. Test false rejection of legitimate changes alongside acceptance of false ones. These are proposed controls for the attribution gaps in TMMA [119].

Add a distractor-growth ladder while holding relevant evidence, reader and returned-context budget fixed. Record final correctness and the tail of memory-call/latency burden. Visible tool calls do not count all backend work [106].

For deletion, demonstrate exposure before testing removal. Use invented canaries, a never-exposed control and unrelated retained utility. Pause and resume an old-source writer; restart, retry, export and import as the contract requires. Distinguish stale resurrection from an authorized new upload, and physical absence from semantic nondisclosure [93, 99].

The useful receipt connects source arrival, accepted/completed processing, stored versions, retrieved candidates, final context, answer/action, latency, usage and failures. Report all attempted operations alongside conditional quality on successful runs. Predeclare a worthwhile gain and acceptable failure behavior for the workload; there is no universal pass percentage or safe waiting time in these sources [76, 103, 90].

These fixtures remain unexecuted. They describe how to turn a shortlist into a decision.

Choose with a receipt

After testing, keep a short decision record a teammate can inspect:

Choose the simplest option that meets the declared behavior at an acceptable complete cost. If the baseline already does, keep it. If the candidate fixes the important failure, keep the evidence showing where it helped. If neither does, revise the design rather than promoting the higher leaderboard score into a guarantee [76, 90, 103].

Method, corrected interpretations and reading scope

The existing Knowledge investigations gave this guide its starting questions: which dependencies survive compression, whether a file interface hides retrieval, how context differs from cached computation, and whether a procedure records a verified outcome. Their catalogue entries include authored syntheses, partial reads, queued stubs and pending sources. Reading them did not mean rereading every original study, and agreement among derivatives did not create independent corroboration [9, 6, 68].

The biological-memory investigations contributed a useful diagnostic discipline: storage, ordinary access and expression can require different tests. In software, deliberately supplying correct evidence can help diagnose a selection bottleneck, but that intervention also changes the prompt. It validates no biological analogy and gives no reason to prefer a graph or call a background consolidation job biologically proven. Pending biological sources and later unreviewed appendices retain their limits [102, 2].

The working-method study drew on David Senra's documented source reading and reorganization of notes around a subject, alongside engineering accounts of questioning requirements and preserving failure evidence. Those practices informed the investigation and editing; their effectiveness as a combined writing method was not measured. The practical rule was to keep the reader's decision stable while allowing the explanation to change.

Several explanations did change. Positive lesson and curator experiments ruled out a general preference for raw executor context or read-time processing. The Graphiti consumer-path check withdrew an inference of default fact loss. Historical prompt recovery strengthened the benchmark audit's comparison, while source-level checks narrowed the audit's own headline claims. These corrections are preserved in the research record [3, 69, 66, 16, 83, 84, 85, 5].

The full research report, vendor-reference audit, exact-source follow-up and additional original-paper digests preserve the reading trail. Newly observed post-cutoff pages remain dated follow-ups in those records. They do not silently redefine cutoff contracts. Local audit links preserve working evidence; publication still needs accessible source routes.

The source list below contains the references this companion actually uses. The three-piece package retains all 19 papers from the original bibliography and preserves the substantive product and experimental qualifications from v3.

Takeaways

A positive result should survive careful description of its controls. Read failures and mismatches alongside the successful runs. Keep original-source errors distinct from defects in a critique, and source-level observations distinct from deployed behavior. Translate the finding into a declared workload test before using it to choose a system.

Sources

Source numbers retain the same identities across the three v4 pieces; gaps in numbering reflect references used in a companion.

[1] Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory.

[2] LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory.

[3] Decocted Experience Improves Test-Time Inference in LLM Agents.

[4] Interim report — changing-fact-interval-and-authority-contract

Read retained audit.

[5] Interim report — benchmark-judge-validity-and-protocol-lineage

Read retained audit.

[6] Prior investigation: context-retrieval-findings

Read retained audit.

[7] Lifecycle official OpenAI prompt cache prefix retention and usage contract captured 2026-10-29.

[8] Persistence | Pydantic Docs.

[9] Prior investigation: storage-findings

Read retained audit.

[10] Mem0 hosted API reference.

[11] What’s new in Microsoft Agent Framework: Interactive experiences, memory, and resilient execution | Microsoft Agent Framework.

[15] Graphiti e9e4a48 pinned complete edge resolver.

[16] Graphiti e9e4a48 pinned temporal search filters and backend generation.

[19] An update to supermemory: discontinuing company brain and Nova — supermemory.

[22] Honcho V3 design patterns.

[23] One memory for every AI tool · Honcho.

[26] Cognee maintainer architecture document.

[32] EverOS overview.

[34] From RAG to Memory: Non-Parametric Continual Learning for Large Language Models, v2.

[40] OpenMemory current README.

[43] AgentCore DeleteEvent source-event deletion contract October 2026.

[45] Google Memory Bank revision history documented contract.

[48] Memory - Docs by LangChain.

[54] Agent memory on Elasticsearch: hybrid retrieval and DLS | Elasticsearch Labs.

[55] Neo4j Agent Memory - Neo4j Agent Memory.

[60] How Claude remembers your project - Claude Code Docs.

[62] About GitHub Copilot Memory - GitHub Docs.

[66] Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents.

[67] Graphiti e9e4a48 pinned public search and lifecycle API complete.

[68] Prior investigation: sql-findings

Read retained audit.

[69] SkillOS: Learning Skill Curation for Self-Evolving Agents.

[70] Evaluating Very Long-Term Conversational Memory of LLM Agents.

[71] snap-research/locomo.

[72] xiaowu0162/LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025).

[73] xiaowu0162/LongMemEval-V2: Official repository for LongMemEval-V2.

[74] Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks.

[75] SWE-ContextBench: A Benchmark for Context Learning in Coding.

[76] MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation.

[77] MemoryArena pinned RAG adapter 6cd9de14.

[78] MemoryArena pinned bounded recent context adapter 6cd9de14.

[79] MemoryArena pinned Mem0 hosted v2 adapter 6cd9de14.

[80] MemoryArena pinned shopping retrieval schedule and trajectory writes 6cd9de14.

[81] full context CoT scored results.

[82] adversarial v2 scored results.

[83] historical EverMemOS February 2026 evaluation prompts.

[84] historical EverMemOS February 2026 judge loop.

[85] audit full context evaluation runner pinned April 2026.

[86] EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoning.

[87] Total Recall at What Cost? Benchmarking the Serving Cost of Agent Memory.

[89] Lifecycle official Anthropic cache writes reads TTL invalidation captured 2026-10-29.

[90] Interim report — lifecycle-cost-break-even-under-churn

Read retained audit.

[91] Hidden in Memory: Sleeper Memory Poisoning.

[92] Set a time-to-live for long-term memory records - Amazon Bedrock AgentCore.

[93] AfterDeleteBench: Auditing Post-Deletion Information Persistence.

[94] Cognee v1.6.6 pinned forget orchestration and history scope.

[95] Cognee v1.6.6 pinned session deletion and locking.

[96] Cognee v166 session locks process boundary.

[97] Cognee v166 dataset lock backend boundary.

[98] Cognee v166 memify runner lock participation.

[99] Interim report — deletion-closure-and-reingestion

Read retained audit.

[102] Prior investigation: biological-findings

Read retained audit.

[103] Interim report — baseline-insufficiency-and-transfer

Read retained audit.

[104] Anatomy of Agentic Memory, v2.

[105] Are We Ready For An Agent-Native Memory System?, v1.

[106] When Stored Evidence Stops Being Usable.

[107] AgentPoison: Red-teaming LLM Agents.

[108] Memory Injection Attacks on LLM Agents via

Read original source.

[109] Import and export your memory from Claude.

[110] Hindsight legacy observations/export issue #5419.

[112] Graphiti PR1729: structural guard and cardinality limits.

[113] Corpus gap cc3 cc4 pinned coordination and protocol analysis

Read retained audit.

[114] LoCoMo audit full-context baseline README.

[115] Anatomy official repository artifact audit.

[116] Benchmark locus LoCoMo source dataset

Read original source.

[117] MemoryTax: The Bitter Lesson of Agent Memory, v1

Read original source.

[118] GAAMA: Graph Augmented Associative Memory for Agents, v2

Read original source.

[119] Truth-Maintained Memory Agent: Proactive Quality Control for Reliable Long-Context Dialogue

OpenReview source URL. Read from the user-attached manuscript; attachment identity and reading scope. Publication status and public availability before the evidence cutoff are unverified.

[120] Reader-supplied discussion screenshots: correction across later contexts and continuity across assistants.

Discussion notes and retained screenshots. Editorial prompts, not a vendor benchmark or exhaustive market claim.

[121] A Survey on Long-Term Memory Security in LLM Agents, retained v4. Lifecycle taxonomy and proposed Verifiable Memory Governance criteria.

[122] Contextual Agentic Memory is a Memo, Not True Memory, retained v2. Position/theory with conditional separation assumptions.

[123] ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks. Continuity proposal and bounded reference-system evaluation.

[124] Episodic memory in AI agents poses risks that should be studied and mitigated, retained v2. Risk and governance position paper.

[125] Hindsight Technical Report. Original-author architecture and configured evaluation; distinct from the captured service documentation.

[126] Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents. Original-author structured-retrieval and added-context accounting; distinct from current commercial claims.