Local Memory for Forgetful AI: Hindsight and Its 94.6% Long-Range Recall Claim

Does a promise made in the middle of a long conversation still hold three hundred messages later? For standard language models the answer is usually no; as the window grows, details buried in the middle fade out. The presenter brings a fix called local learning memory : Hindsight, an open-source system that lets a coding assistant learn from past interactions and behave more accurately with every use.

Why the classic approaches fall short

The naive approach of stuffing the full context into the window inflates cost as chats grow and loses the critical detail in the crowd. Classic retrieval-augmented generation stores text chunks as-is and cannot track relationships, assumption changes or staleness. The philosophy in the GitHub repository diverges at exactly this point: the goal is not to remember chat history but to make the assistant learn over time. That distinction also answers the old debate between knowledge-graph enthusiasts and pure vector-search advocates.

The numbers look striking at first glance. On the S split of LongMemEval, a 500-question run of Hindsight on a 20-billion-parameter open model reports 83.6% overall accuracy, while full-context GPT-4o stalls at 60.2% on the same split. With a Gemini-3 backbone the score climbs to 91.4% overall, with 94.9% on knowledge updates and 91.0% on temporal reasoning. The Vectorize benchmark board headlines 94.6% long-range recall, and the GitHub notes state the main table was independently reproduced in collaboration with the Virginia Tech Sanghani Center.

Inside the engine: a four-layer memory

Instead of cramming a scattered chat pile into one text file, the system splits knowledge across four specialised stores. The first layer keeps durable facts about the user, dates and the workspace; the second acts as an activity log recording completed tasks with the tools used and their outcomes. The third layer tracks working assumptions that update as requirements change, and the fourth keeps tidy compiled memory pages written quietly in the background. The DeepWiki architecture notes show how these layers sit on Postgres with pgvector indexes, while the Agent Memory Atlas review explains the split between raw documents and world facts, experience facts, observations and user reflections.

Finding a memory runs through the TEMPR layout with four arms racing in parallel. Semantic search matches different words with the same meaning, an exact keyword sweep catches precise strings such as function names and error codes, a relation graph follows person-tool-topic links to gather context plain text search misses, and a temporal filter drops stale entries to focus on recent learnings. The Vectorize developer docs on recall explain that nothing is decided up front about which arm fits a query; every applicable arm runs and a cross-encoder re-ranks the results.

The cost story is the project's boldest claim. Because memory lookups happen on the device, each query incurs no cloud bill, and the server allows switching across roughly 25 model providers without rebuilding the memory database. The Python library is sized at a few hundred megabytes of system RAM so it can sit next to an editor on an ordinary laptop. The Docker and client guides on GitHub sketch a setup that boots in a single container with embedded Postgres and can attach to fully local models through Ollama.

Setup and the first bank

The setup story is deliberately plain: open a virtual environment, install the client with pip, optionally drop a Gemini key into the environment file or pick the local-model path instead. The client speaks three verbs; retain writes knowledge, recall searches memories and reflect produces a topic-appropriate answer over a bank's accumulated knowledge. Each bank carries its own configuration, multi-tenant endpoints split by bank identity, and a two-line wrapper plugs memory into an existing assistant automatically. The Python example on GitHub shows the three calls set up in a few lines through the Alice example.

The liveliest stretch of the demo feeds a fictional Pulse Grid product brief into the system. The brief packs facts such as a no-rewrite database speedup, 14 large customers, 4 billion daily requests, a 38,000-dollar monthly server bill, three large customers whose contracts end next month and a half-second latency spike under heavy load. The assistant first extracts these facts into a memory bank, then runs the same fact set through three different persona filters. Results land both in a Markdown file and a plain HTML board showing the profiles side by side.

Kyra: three personas, three verdicts

The Kyra persona system tunes how the assistant treats evidence along three axes. The investor profile pushes suspicion to the maximum, drops emotional warmth to the floor and asks like a merciless vice president: if most revenue rests on three contracts ending next month, how long does the cash last? The senior engineer profile maximises literal rule adherence and refutes near-instant replication talk with the half-second measurement. The support profile damps suspicion, raises warmth and stresses that a no-rewrite speedup is genuine relief for tired developers. One memory yields three coherent yet different verdicts in three temperaments.

The translation to daily work is quick: spotting cash-burn risk early in startup briefs, collecting warning signs before subscriptions get cancelled, scanning thousands of project notes without paying per query, and scrubbing secret keys and personal data from incoming text before anything reaches the memory store. The presenter stresses the last one in particular; dangerous strings get filtered before they are written. Three verbs plus persona axes add up not to a single chatbot but to a memory infrastructure that can be reshaped per role.

The trade-offs are counted openly too. First setup pulls a few hundred megabytes of model weights, and every text chunk written to memory triggers one language-model call on the retain path, so cloud users are advised to batch chunks. Rather than evicting old knowledge at random, the system refines existing knowledge carefully so mid-session instructions stay as crisp as the latest messages. The ACL paper at aclanthology puts the retain, recall and reflect trio into a formal frame, and the component inventory on DeepWiki shows what runs where in production, from HTTP and MCP interfaces down to the Postgres schemas.