Arize's Alyx Agent Swaps Knowledge Graph for a Single 8,000-Character Memory File

What Arize shipped

According to the Arize blog post, each user gets one memory file for each Arize space they work in. Alyx reads the whole file on every request, which removes memory retrieval from the read path. Background jobs summarize completed turns and propose targeted edits, and compaction keeps the file within its size limit. Arize notes a trade-off: a rejected update can leave recent information unsaved.

The author says the design resembles the approach OpenAI and Anthropic have taken with ChatGPT and Claude. Per the post, both put a compact memory profile straight into the model's context and keep search for the long tail. Arize argues that for a scoped agent, one always-loaded, well-structured file can be enough, and calls it the right default for most such agents.

Why Alyx needs memory beyond a session

Arize describes long-term memory as context an agent keeps across sessions, such as durable user preferences, goals, conventions and decisions. Sessions end, so a new chat starts from zero. They also degrade as they grow. The post cites Chroma's testing of 18 models, which found that "performance grows increasingly unreliable as input length grows," and the "Lost in the Middle" research from 2023 on models struggling with information buried mid-input. Long sessions are eventually compacted, and a summary written for one task can drop details needed for the next.

Arize says this is especially acute for Alyx because users move between trace views, eval pages, the prompt playground and dashboards, and each can start its own thread. If Alyx learns that ground truth lives in metadata.label, that should still be available when the user opens the eval builder in a new session. The same applies to preferences such as "datasets should have exactly 70 rows."

Where the retrieval and graph approach fell short

The first prototype treated memory as retrieval, modeling users, projects and assets as graph entities. The author spent weeks building on Graphiti, Zep's temporal knowledge graph, and found it capable, particularly for relationships and facts that change over time. Three problems emerged for Alyx.

• Broad preferences were easy to miss. A query built from the latest message, such as "Make a dataset of agent-prod traces," has little semantic overlap with a convention like exactly 70 rows. Loading conventions on every request avoids that gap.
• Graph memory is expensive to maintain. Reading Graphiti's add_episode path, the author counted one LLM call to extract entities, one to resolve ambiguous entities and one to extract facts, plus further per-fact and per-entity calls. For a turn with 5 entities and 6 facts, that is roughly 10 LLM calls without custom types and more than 20 with them. The post also points to rate limits and latency.
• It becomes a data science problem. Debugging shifts to recall at k, embedding models and similarity thresholds. The post says memory benchmarks are shaky, noting that vendors disagree even on LoCoMo: Zep's corrected score for itself is 75.14%, Mem0's paper reported 65.99% for Zep, and a Mem0 re-run posted on GitHub put it at 58.44%.

Eval results and what it means

Arize reports that in its final evals Alyx answered 16 of 18 memory-dependent questions correctly, both across three separate scenarios and in a mixed run that interleaves them. The team also tested from the opposite direction, with questions where memory could make the agent worse, and says Alyx passed every question in that group. The post also covers failures the team found, including preferences that disappeared and mistakes the agent wrote back into its memory.

For teams building scoped agents, the message from Arize is that a simple, always-loaded file can replace a costly retrieval pipeline, though the hard part remains deciding what to preserve and when to apply it. These results are Arize's own, from its internal evaluations.