How Much Past Experience Can an AI Agent Actually Use?

The Dose Makes the Difference

Giving an agent every lesson it has ever learned does not always help. Across eight models — from a 30 billion parameter dense system to frontier proprietary models — the researchers saw three distinct patterns. Strong models with headroom absorb the full guideline set and improve. Weaker models drown in that same volume and do better with a tight core plus a few task‑specific guidelines retrieved per task. Saturated models show no measurable gain at all. The pattern is not determined by parameter count alone. Benchmark headroom, context‑window size, architecture, guideline quality, and task distribution all appear to shape where a model lands.

What the Memory Actually Is

Memory here does not mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, edge cases — distilled from the agent's own prior trajectories. The loop is straightforward. The agent attempts tasks and produces trajectories. ALTK‑Evolve extracts behavioral guidelines from both successful and unsuccessful runs. It consolidates those guidelines into a reusable set. At inference time the agent receives either the full set or a task‑relevant selection. No model weights are updated. The learning loop changes the guidance available to the agent, not the underlying model, which is why the method is cheap to adopt and portable across the eight models tested.

The Benchmark and the Numbers

Evaluation ran on AppWorld — 585 multi‑step tasks across nine simulated apps including calendars, messaging, and payments. Tasks are scored two ways. TGC (Task Goal Completion) measures the share of individual tasks completed fully. SGC (Scenario Goal Completion) is stricter: a scenario bundles several variants of the same request, and it counts as passed only if the agent succeeds on every variant. The three configurations compared were baseline with no memory, full guideline set injected on every ReAct step, and curated retrieval delivering a fixed high‑confidence core plus a few task‑relevant guidelines per task. Both memory configurations draw from the same guideline set mined once from AppWorld's training split only. No test‑split data ever enters the guideline construction.

Representative results on the test_normal split tell the story. gpt‑oss‑120b (117B MoE) gained +16.1 percentage points TGC and +16.1pp SGC with curated retrieval. DeepSeek‑V3.2 (671B MoE) gained +9.5pp TGC and +16.1pp SGC with the full guideline set. Claude Opus 4.6 and GPT‑5.5, both near the ceiling on TGC, still added +7.1pp and +7.2pp SGC respectively with the full set. GLM‑5 (745B MoE) showed zero gain in either metric. The stricter SGC metric often moves more than TGC because good guidelines especially help an agent clear every variant of a scenario, not just the average case.

Cost Is Not What You Might Expect

Injecting a full guideline set inflates every ReAct step's input because the guidelines are re‑sent each turn. DeepSeek‑V3.2 saw token use per task rise from 148K to 263K, a 78 percent overhead. gpt‑oss‑120b with the full set rose from 110K to 166K, a 51 percent overhead. The same model with curated retrieval added only 5 percent — 110K to 116K — while delivering the larger accuracy gain. Better performance here does not require more inference cost. The agent runs about the same number of ReAct steps with memory as without, roughly 18 to 19 on average, so the added cost is input‑token inflation, not longer trajectories. Prompt caching changes the economics substantially. The static portion of the guideline set is identical across steps and can be cached, cutting effective cost. Cache‑aware prompt design — keeping the shared guideline‑set prefix stable so it stays cacheable — is worth engineering for. The researchers hypothesize that context‑window size plays a role: models with larger windows may absorb the full set more effectively, while smaller‑context models benefit more from retrieval that keeps injected content compact. Controlled experiments isolating this factor have not yet been run.

What Comes Next

This is a starting point. A learned selector trained on outcome signal would replace the current cosine‑similarity retrieval, which does not perfectly predict which guidelines help a given task. Below a minimum capability baseline, self‑distillation lacks signal; teacher‑distilled memory for very weak models is a separate problem under exploration. Results are validated on AppWorld — a rigorous multi‑step benchmark, but a single one. Broader agent benchmarks and real‑world deployments are in progress. Isolating context‑window size from raw capability remains an open experiment. The ALTK‑Evolve library, which includes the extraction, consolidation, and retrieval pipeline used here, is available for testing. The full technical report contains the complete method and ablations.