The first AI ecosystem trial showed a cached answer surviving a changed policy. This follow-on asks a harder question: after a cache is bound to the right policy version, can similar wording still make it answer the wrong question?
Version binding prevented the changed-policy cases. It did not prevent wrong answer polarity. Through MongoDB’s actual semantic-cache adapter, the fixed 0.8 score cutoff reused an incompatible “YES” in all eight same-version questions that asked whether an allowed action was forbidden. Raising the cutoff to 0.9 or 0.95 removed every hit, including the valid ones, in this configuration.
Keep the cases and thresholds fixed
Eight fictional policy domains covered restarts, contact exports, log retention, refunds, backup regions, guest access, releases and audit logging. Each had five authored cases: an identical question, a valid paraphrase, an opposite-polarity question, an explicit new-policy premise and the same question under a new policy version. The cache stored the authored answer YES; no generative model produced these answers.
We froze those forty cases, labels and cutoffs locally before running the same pinned local Voyage 4 nano Q8_0 model used in the earlier note. It produced sixty-four real embedding inputs. We retained all 2,048-dimensional responses and used normalized 512-dimensional prefixes. This was not publicly preregistered or independently human-labelled.
A version key fixes one class of stale answer
One arm keyed each family by model alone. The other included trusted tenant, model, policy and corpus versions in the adapter’s context string. Each family had one stored answer per arm, so unrelated-family competition was excluded. Both arms preserved the same valid hits; version binding excluded the sixteen changed-version cases, while eight same-version polarity errors remained.
| MongoDB score cutoff | Cache binding | Incompatible reuse | Valid hits |
|---|---|---|---|
| 0.8 | Model only | 24 / 24 | 11 / 16 |
| 0.8 | Trusted tenant, model and versions | 8 / 24 | 11 / 16 |
| 0.9 | Model only | 0 / 24 | 0 / 16 |
| 0.9 | Trusted tenant, model and versions | 0 / 24 | 0 / 16 |
| 0.95 | Model only | 0 / 24 | 0 / 16 |
| 0.95 | Trusted tenant, model and versions | 0 / 24 | 0 / 16 |
There were twenty-four incompatible-answer opportunities and sixteen valid-answer opportunities per arm. A cache miss is an abstention, not a correct generated answer. These intentionally paired cases do not estimate a real traffic error rate or select a universally safe cutoff.
The embedding roles matter
Stored questions used the model’s document instruction; lookup questions used its query instruction. Identical wording therefore had different model inputs. The higher-cutoff misses belong to that asymmetric configuration. We did not test symmetric cache embeddings, another embedding model or an entailment validator. High similarity in these cases coexisted with an answer that had the wrong polarity.
Check the corrected memory’s search score
The memory trial used the official MongoDBStore adapter, eight items per tenant and actual recorded model vectors. In each repetition we replaced eight tenant-A policy texts. Each persisted vector matched its replacement text, and all 128 returned semantic scores across both tenants matched the current-vector oracle within 1e-5. The frozen queries distinguished old vectors from replacements, so changed text alone could not pass this check.
After deleting tenant A’s items, its direct lookups and semantic searches were empty; all eight tenant-B documents were unchanged under the live equality assertion and full-document hashes. Complete raw neighbor documents are not reconstructed by the arithmetic audit. PostgreSQL supplied forty-eight exhaustive, tenant-filtered vector observations per repetition across the before, corrected and deleted states. It was an exact-vector control, not a competing semantic-cache implementation.
The search-score check was implemented after inspecting the embeddings and before any database execution. The cases and thresholds stayed fixed. Across the counted pair: 480 actual cache lookups, 96 PostgreSQL observations, sixteen persisted-vector replacements and 256 checked semantic scores. These are repeatability and consistency checks on one authored fixture, not independent corpora or production tenant authorization.
A healthy database can still have an unavailable search index
Two earlier hosted attempts stopped before any cache lookup because the index was not ready within 180 seconds. The first logged thread-creation failures under a 256-task cap. A 1,024-task cap allowed a measured peak above 256, but the next attempt still reported paused initial sync. Both failures and their zero semantic-observation counts remain in the attempt history.
The completed setup first measured the default search filesystem, then gave search data an owned 512MiB RAM filesystem with verified ownership, capacity and free headroom. MongoDB documents disk-pressure pauses. The completed job’s default overlay was about 86.1% used, while its fresh search filesystem remained below 0.104% used. That is consistent with the storage hypothesis; the earlier jobs did not measure disk capacity, so it does not establish their sole cause. No server protection threshold was disabled.
The counted job completed both database repetitions and removed its four owned containers, including the stat-only baseline helper, and internal network. The index filesystem was deliberately disposable. This does not establish index durability, availability, comparative speed, cost or a platform winner.
Inspect and reproduce
Counts, commands and UTC windows · Frozen local protocol · Every authored case · Repetition251 · Repetition252 · Separate arithmetic audit · Evidence hashes.
The executed payload, passive hosted-workflow example and audit source use pinned images and fifty-four hash-checked Python package versions. The complete decoded model record includes all coordinates. Administrative infrastructure logs are retained privately; exported numerical records and transformation hashes are public. A source-formatting verification repeated the same unchanged cases after the first completed trial; table counts use the final reviewed payload and do not expand the case population.
The next useful comparison is an explicitly separate cache-embedding configuration and a grounded answer-validity check. Managed memory extraction, generative answer quality and production authorization remain untested here.