Decisions as Retrieval: CLM-8B, the 13x Faster Architecture

Most decisions inside real software do not need a model to think for thirty seconds. Is this ticket urgent, which tool should run next, which link should the agent click: those answers are needed in milliseconds, not seconds or minutes. That is why the System One wave keeps growing; fast, non-reasoning models that return typed decisions instead of essays are suddenly hot again. The opening thesis here is exactly this: the heart of agent loops is no longer generation, it is selection speed.

This week the difference is architectural. A joint team from Stanford and Nvidia Research released CLM-8B, an open 8-billion-parameter model they call a contrastive language model . The sharpest analogy is CLIP for decisions: the joint embedding idea described in the Arxiv CLIP paper is applied here to situations and actions. A state encoder reads what is going on, an action encoder reads what could be done. In the Mario example the game screen is the state and left, jump, or run are the actions; every option is embedded, scored by closeness to the state, converted to probabilities with a softmax , and jump wins at eighty percent.

CLIP for Decisions: State and Action in One Space

The clever part of the design is that it speaks the existing habit fluently: the model uses the same interface as Jev. According to the Jevapi docs there are three primitives; a yes-no question, a multiple-choice question, and a score. According to the System-One docs all three are the same thing underneath: a state plus a closed candidate list. That closed nature cuts both ways. The model cannot invent an answer outside the list, echoing the no-hallucination promise, but the Jevals methodology reminds us what a System One model is by definition: no out-of-list generation, yet in-list mistakes always possible, and the model can pick the wrong option confidently.

How is a space built where the right action lands closest? The answer is contrastive training . Training starts from millions of examples, each a pair of a situation and an action somebody actually took; the model learns to pull each situation toward its own action and away from the rest. The neat trick is that nobody writes down wrong answers: in a batch of a thousand, if jump is right for one Mario screen, the other 999 actions automatically count as wrong for it. Positive and negative balance comes for free, and the difference between sounding right and being right gets carved into the geometry of the space.

The secret of the speed hides in that split. In a Jev-style model the state and all options are read together on every call; doubling the options doubles the reading load, plus the answer must be decoded token by token. CLM separates the two and exploits the typical agent loop: what changes across steps is the state, not the action list. Actions are embedded once and cached ; each new step costs one state embedding plus a nearly free dot product. On the team chart the CLM line barely moves as options approach the thousands: 44 milliseconds against 570 at 1024 options, roughly a 13x gap. According to VentureBeat the team reports up to 9x in open tests; whichever number is taken, the chart points the same way.

Three-Stage Training: World First, Fine Distinctions Second

Training follows a three-stage recipe, and the order is the whole trick. Stage one feeds 60 million question-answer pairs from Nvidia Nemotron data for general world knowledge, with questions as states and answers as actions. Stage two adds 30 million hard negatives written by a generative model, answers that sound right but are wrong, like the Venus trap for the planet closest to the sun. On a hard-negative test, pretraining alone scores 52 percent while the second stage lifts it to 69; starting with hard negatives from day one peaks at 62 and slides into memorization. The Notion official page frames this order as a pretraining plus post-training relationship: learn the world first, then the fine distinctions. Stage three converts the model into an agent with about one million real trajectory steps, replaying 40 percent of the original data; without that replay the hard-negative score falls from 69 to 56.

The most surprising part is what never moves: the 8-billion-parameter backbone stays frozen as a Qwen3-8B reader, receiving not a single gradient. Only small heads of about 20 million parameters on each side are trained. Because the backbone is fixed, the dataset is embedded once and full pretraining finishes in about an hour on a single RTX 4090. The Qwen3 repository on GitHub confirms the open-weight, 36-layer structure, and the model card on HuggingFace records 8.2 billion parameters under Apache 2.0. Loss follows a power law in compute, data, head size, and encoder size, with the encoder delivering the largest gains. Hence the team points to the bigger multimodal release expected next month.

After theory the focus moves to a real desk: a Qwen3-8B encoder served with vLLM exactly as the Qwen docs describe, topped with the released 75 MB CLM server, for roughly 25 GB of total memory. According to Nvidia the DGX Spark holds inference up to 200-billion-parameter models on the desk with 128 GB of unified memory, so the rig is ambitious yet reachable. The state is a support ticket from a customer charged twice and unable to reach anyone; is it urgent, which team, how angry: answers arrive as probabilities, 84 percent urgent and 99 percent billing. Once the ticket is seen, repeat questions return in under 2 milliseconds because everything is cached. In the Romeo and Juliet test raw embeddings rank Marlowe first while CLM pushes Shakespeare to 98 percent; twenty million parameters bend the space from similar to right.

Speed at a Thousand Options, Limits on Accuracy

The architecture shows off at extreme scale: with a list of 1080 tools from Stripe to Trello, the first call takes seconds for embedding, then every later request settles near 80 milliseconds. Picking among a thousand costs nearly the same as picking among one. In wiki racing, a page with 934 links is scored against the target in a single pass and reached in four clicks. But the same experiments document the ceiling: correct picks fall from 86 percent with 8 tools to 17 percent with 1080, as lookalike refund tools get confused. The cause is the blind spot of the design: the state is encoded alone, so options are never laid side by side and compared; each candidate is judged in isolation.

Hence the two-layer advice to builders: let CLM shrink a thousand options to a ten-item shortlist in milliseconds, with a joint-reading model issuing the verdict. Where the action set is small and stable, a game loop or a few routes, the model does fine on its own. According to the tool-use guide from MLflow, piling more than 8 tools onto one agent is a design smell and splitting into sub-agents by function is the fix; the accuracy curve of CLM independently confirms that rule. The closing big picture is nostalgic: the fast no-reasoning idea that Jev reheated meets the contrastive algorithms of 2017 and 2018 in the same pot. Decisions stay with retrieval, generation with slow thinking.