Lean Memory Beats Big Memory: Four Traps for Marathon Helpers

Windows that once held a few thousand tokens now stretch past a million, and my first reaction was delight: a wider desk surely means wider work. A measurement across 18 frontier models says otherwise. As the window filled, each model used every entry less reliably. Researchers call the slide rot, and the raw numbers sit openly on GitHub. My reading of the result is simple: a window can be huge while the spotlight moving across it stays narrow, and each added token eats from that spotlight.

A concrete job makes the picture tangible: an agent shifting an aging data driver to a new one. At the front of its window sits the standing brief, followed by a two-line job note. Project guidance comes next, then definitions for every helper routine the agent may call. From there the window turns into a running log: each move the agent makes, plus the outputs those moves return. After an hour the pile is enormous: file contents, test outputs, error logs, schema dumps. Tucked inside sits a small memo the helper wrote to itself: the new driver restores broken links on its own, so the old retry section may go.

The first breakdown is poisoning : one false entry gets handled as fact. In our story the helper's own memo is wrong; it skimmed the driver manual, misread a passage, and concluded the retry section was spare. To a model, its own memo looks like ground truth, so the agent walks file after file stripping out sound retry logic. The kitchen version: one wrong measure copied into the recipe book spoils every dish. The repair is Validate . Had a small live link check run before that memo entered the window, the error would have died at birth.

The second breakdown is distraction : the log grows so large that the model watches its past more than its task. A language model is at heart a pattern extender, and an hour of its own moves sitting in the window becomes the loudest pattern around, louder than the judgment built during training. The outcome is familiar: the agent paces old ground, retracing steps already taken. It feels like shuffling yesterday's notebooks instead of doing today's chore. The repair is Compact : as the log lengthens, the helper folds it into a short recap and marches on, while detail rests in durable files and enters the window only on demand.

The third breakdown is confusion , and the culprit is tool sprawl: our agent links to version control, a data console, ticket tracking, and plenty more. Each definition occupies the window whether used or not. With more options, picking correctly gets harder; in one measurement the same assignment failed with 46 tools on offer and passed with 19. The overstuffed toolbox says it best: the fuller the case, the longer the hunt for the right wrench. The repair is Curate . A driver migration wants only a handful of tools, so the helper starts with those and fetches another only when truly needed.

The fourth breakdown is clash : two entries that disagree sit side by side. Sooner or later the agent notices that no retries occur, realizes the first memo was wrong, and logs that insight into its history. Now the window holds both versions, wrong and right, with equal weight. Like two shopping lists on the fridge with no mark of which one counts. The repair is Prune . Once fresh information lands, the stale entry goes; better still, the helper may open a crisp window restating the corrected state and leave the tainted material behind.

What I love is the shared thread across all four repairs: each one removes something, and the helper improves by holding less. That insight now belongs to a whole field, not one talk: Anthropic argues for a tight attention budget with few high-signal entries. LangChain proposes four paths — write, select, compress, and isolate. IBM describes assembly and compression pipelines. Anyone may inspect the 18-system measurement through GitHub, while a broad survey on arXiv gathers more than a thousand studies into one frame. The rule in my notebook reads: every entry added to the window is a load on the helper, so keep what the task needs and leave the rest outside.