Google's Dream RSI: Turning History into a Simulator for Recursive Self-Improvement

In 'Google is SO back…,' Wes Roth frames Google's bet as dreaming RSI rather than living it: while many debate whether RSI should happen, Google puts the AI into a simulation and dreams through thousands of improvement pathways to see which branch gets there fastest. The paper at the center, Dream-RSI — the 'R' for simulation — does not try to replace the coding agent; it tries to improve the exploration policy that decides which experiment the agent should run next, turning the meta-decision into the main optimization target.

Why discovery behaves like a tech tree

The tech-tree metaphor borrowed from games makes the point intuitive: like crafting wooden tools before stone and diamond in Minecraft, science needs prerequisites. Wes sketches humanity's own tree from stone tools to polished blades, then wikis, DVDs, Wi-Fi, GPUs, and finally the 2025 stack of LM chatbot, reasoning model and AI coding agent as the current frontier. Some branches look promising and die, others look accidental and become leaps; the tree keeps branching, and choosing the branch is the real skill.

That history explains the pendulum swings. In the 80s and 90s many dismissed neural nets as dead ends until Geoffrey Hinton argued the bottleneck was compute and Jensen Huang's GPUs proved him right. The same split returned with large language models: one camp called it the wrong road to AGI and wanted to shut it down, the other scaled what seemed to work and was vindicated, if grudgingly. Dream-RSI systematizes that choice: even an excellent researcher can waste a week on the wrong chain, so the lever is not just better experiments but better decisions about which experiments deserve energy, compute and attention.

The answer: history is already a simulator

The proposal from Google and collaborators is disarmingly simple: all those past experiments, with the lineage of which finding depended on which, are already data — a map. The site distills it to three ideas: history provides the world for dreaming, dreaming powers the self-improvement step, and each successful lap contributes a new world. There is no learned world model and no approximation involved; the tree already grown by the agent functions as a precise replay engine for the territory it has covered, a world acquired for free as a side effect of doing the work.

In practice you roll history back to a point and let the agent walk the historical tree. Wes recalls Demis Hassabis's thought experiment of training up to the early 1900s and asking whether the agent would re-derive relativity, or more directly for this paper, whether it could speedrun discovery by finding faster branches through the same tree. Dream-RSI makes that concrete: dream thousands of candidate policies against that world at zero executions and only deploy the winner. The Mario analogy helps — a billion Marios converge on tactics that work — except here the Marios learn where to branch, what to parallelize and when to stop the search.

Closing the loop: a growing pool of worlds

The loop has three stages. Online exploration: the current policy steers the coding agent, expands the discovery tree and logs traces. Simulator construction: that tree, with its branching decisions and code-execution outcomes, becomes a reusable replay simulator pool. Dream-based policy improvement: a policy-development agent dreams up alternative exploration policies and replays them for rapid feedback, then ships the best scorer. Every winning deployment records a new tree, so the agent owns not one world but a growing pool; a policy scored over a larger set of worlds outperforms one fitted to the chance outcome of a single run and returns worlds no earlier strategy could have reached — the worlds evolve along with the agent.

The appeal is cost. Judging an exploration policy normally means watching it steer an entire long-horizon run to the end; feedback is delayed and expensive, and the meta-policy space is vast and mostly bad. A fixed, handwritten exploration script cannot learn from experience and keeps paying for directions that already failed. By reading history as a tree rather than as static text or training data, Dream-RSI turns past records into a simulator where a new strategy can traverse different branches in different orders with different concurrency and stopping rules, all over nodes that have already been executed. Evaluation returns immediately at zero extra executions, which is how one costly online run can pay for thousands of off-policy evaluations.

What the numbers show and how behavior adapts

Does it work? On dream-rsi.com's highlights across eight tasks in three domains, the controlled baseline is Recursive Fixed Exploration — identical agent, evaluator and budget, but the policy never changes, so round one is identical by construction. Dream-RSI reports 2.43× fewer generations on VGG16 at comparable performance, 2.09× higher score on ConvDiv at a comparable budget, and 162× fewer discovery-agent calls than SimpleTES on Lasso. The Lasso table is concrete: with Gemini-3.7-Flash, Dream-RSI averages 2,350.6 ms held-out runtime at 1,879 cumulative calls versus 2,516.7 ms at 3,200 calls for fixed exploration and 3,804.8 ms at 51,200 calls for SimpleTES with gpt-oss-120b; Gemini-3.1-Pro shows the same direction.

Two behavioral notes stand out in Wes's chart walk. The approach is adaptive: when progress is easy it spends less compute, when progress stalls it spends more. And trying to guide the agent with advice from its past makes things worse — open-ended exploration works best, which supports the idea of comparing whole routes on the map rather than copying the previous route. The video also notes that AlphaEvolve beats Dream-RSI on some mathematical optimizations, but that is not a contradiction; the tools target different slices of RSI and can be combined rather than ranked as direct substitutes.

The closing ties back to Google's broader RSI stack. AlphaEvolve's earlier win on Borg data-center scheduling — a non-theoretical optimization Wes recalls as saving millions by better organizing Google's global compute pool — is the complementary piece: one system runs experiments, another figures out how to schedule which experiment runs next, and together they improve the next generation of models, tools and processes. In that architecture Dream-RSI is the scheduler's dream trainer: a lightweight orchestration layer that makes branching, parallelism and stopping explicit and programmable while leaving the coding agent unchanged. That is why Wes calls it foundational rather than flashy — not a new model off the shelf but the layer that lets the models keep improving, a plausible next lever from the lab that invented the transformer.