Ferryte · Agent Memory Audit

Acme Support Agent (sample)

ACTION REQUIRED

Prepared by: Pranav Chahal · Ferryte

Date: 2026-07-10 17:24 UTC

Environment: staging

Engagement: AUD-2026-001

Executive summary

This audit instrumented the agent's memory layer (mem0), ran 5 Ferryte structural verification scenarios, and attributed 1 reported wrong answer to the memories that caused them. Result: 3 scenarios passed, 1 warned, 1 failed, with 1 observed blind-spot and 1 structural capability gap.

Prioritized fix list

[source-revocation] Revoked marker from 'zendesk-ticket-8821' re-entered retrieval via derived summary
Stale belief (mem_a41f9c2e): Delete or supersede the stale artifact; add record_supersession() at the update site.
[stale-fact] Superseded fact still ranked above its replacement

Scenario verification

Each scenario seeds canary data, actuates the backend's own delete/scoping APIs, and verifies both retrieval results and raw store contents — reducing the risk of a "canary passed but tainted context still stored" false positive.

Scenario	Verdict	Seeded	Deleted	Findings	Duration
source-revocation	FAIL	6	6	1	412 ms
cross-tenant-isolation	PASS	8	0	0	230 ms
stale-fact	WARN	4	0	1	198 ms
memory-poisoning	PASS	5	0	0	305 ms
mosaic	PASS	6	6	0	244 ms

FAIL revoked-reentry — Revoked marker from 'zendesk-ticket-8821' re-entered retrieval via derived summary zendesk-ticket-8821 → sum_77ac → retrieval
WARN stale-belief-retrieved — Superseded fact still ranked above its replacement

Wrong-answer attribution

Answer 1: “You're on the Legacy Free plan.”

question: “what plan is this customer on?” · tenant: acme

#1 · mem_a41f9c2eStale belief · confidence 1.00 · mem0

belief

Customer acme is on the Legacy Free plan.

tenant

acme

source

zendesk-ticket-8821

retrieval

retrieved 3× into context — this memory actually reached the prompt

shared span

“legacy free plan”

anchor

recorded in context for this exact answer (ans_18ef)

superseded by

mem_77b0d1

diagnosis

Stale belief — A newer fact on the same subject exists; the agent answered from the outdated one.

What this audit could not verify

Ferryte reports its own limits. Anything listed here was not proven safe — it was unverifiable on this backend, on this run.

Backend	Kind	Detail
mem0	derived-summary	Summaries created before instrumentation have no recorded source edge.

Backend	Missing capability	Why it matters
mem0	DERIVED_ENUMERATION	Cannot enumerate derived artifacts; blast radius is incomplete.

Auditor notes

Sample report generated from the multi-tenant leak demo. Names anonymized.

Methodology

Ferryte instruments every memory write, retrieval, and delete via ferryte.instrument(), building a lineage graph from each source to every derived artifact. Attribution ranks memories against a wrong answer using recorded answer→memory edges (when available), retrieval traces, IDF-weighted content overlap with shared-span evidence, and semantic residue. Counterfactual replay ablates a suspect from live retrieval to show what the agent's context would have been without it. All checks run inside the client's environment. Ferryte sends no memory content to a hosted Ferryte service; this local report may contain excerpts and should be reviewed before it is shared.