Building a Harness with Jev: LangChain and TypeSafe's System 1 Model

Jev is not a text generator but a System 1 engine that turns text into a typed decision, and the LangChain × TypeSafe webinar with Allie, Hunter and Sydney made that distinction crisp. It behaves like a panel of smart humans who can answer a well-scoped question in five seconds, returning a value your code can branch on directly instead of parsing prose.

The session framed LangSmith as the agent engineering platform: an agnostic stack for the agent development lifecycle — Build, Test, Deploy, Monitor. With endless inputs and nondeterministic outputs, teams now build with Deep Agents, LangChain or LangGraph in code, or with Fleet without code, testing via evals and experiments, shipping in one click, and observing every interaction from a single dashboard with governance and an LLM Gateway baked in.

System 1 vs System 2 and why code-first matters

Allie rooted the System 1 label in Daniel Kahneman’s Thinking, Fast and Slow. System 1 is fast intuition, System 2 is deliberate multi-step reasoning. Early LLMs gave quick but unreliable intuitive answers; recent reasoning models shifted toward System 2. Jev deliberately occupies the first camp — fast, scoped judgments — while, as she put it, “your code is often the System 2” that composes those judgments.

The code-first argument is pragmatic: most future automation is machine-to-machine, and APIs already run the world. Inserting a heavy prose-generating model just to output text that is then parsed into a tool call or a CLI invocation — itself a human interface — is an absurd bridge. If the consumer is software, the intelligence should be machine-native from the start.

Hunter’s hammer analogy landed the same point: once LLMs became popular, every problem looked like a nail. Tool calling and structured outputs were transformative prosthetics that made LLMs usable in software, yet running the same large engine for every tiny decision inflated cost and latency. Jev is an attempt to replace those prosthetics with a purpose-built decision.

At its core Jev exposes three question types evaluated in parallel against a shared state. A Choice picks one option from a list and returns per-option probabilities plus confidence. A Score rates the state along a single ordered axis from 0 to N with a continuous value. A Noul answers a yes-or-no statement with a probability between 0 and 1. Adding questions barely changes latency and costs only the tokens for the extra questions.

Choice is the workhorse and closest to a classifier. The example was routing a support ticket to Billing, Technical or Sales. It shines when options are distinct and a single dominant answer is expected; overlapping options signal that the question should be split.

Score’s power comes from crisp level definitions. In the demo a customer’s frustration is scored 0–2 as calm, frustrated, very angry, each level described semantically. Without that, what a “6 versus 7 out of 10” means is vague. When levels are well defined, a 1 means the same thing across thousands of states — whether chat messages, documents or log lines — enabling trustworthy sorting and filtering.

Noul is deceptively simple. “The message conveys urgency” returns a continuous probability, not a hard boolean, and any “or” inside the statement is a yellow flag that you are squeezing two axes into one. The rule is to never ask a compound question; open a separate Noul per axis and combine the results in code.

LangChain positioned Jev not as an LLM replacement but as the glue inside the harness. The classic agent loop is model → tool → evaluation → continue, and phrasing every micro-decision as an LLM call is expensive. LangChain’s harness philosophy is to use any model that fits the task; Jev handles the small, frequent junctions so the generative model is invoked only when prose or deep reasoning is truly needed.

Inside the harness: risk gating and model routing

The first live demo was AutoModeMiddleware. Adding a note to a CRM is benign; deleting a customer account is destructive. Previously that gate required another LLM call. With Jev, the middleware classifies the pending tool call just before execution: the safe call in Hunter’s run succeeded, the deletion was classified as risky and blocked with an error status. The middleware ships in langchain-typesafe’s experimental package and can be imported directly.

The second demo routed between models. A RouterMiddleware chose between a cheap fast model (Luna) and a deeper model (Sol). A trivial rewrite prompt was routed to the fast path; a longer research-heavy task went deep. Because the middleware writes every classification into LangSmith traces, you can audit exactly why each input took the path it did, including the state and question that drove the decision.

Observability got its own segment. Sydney stressed that evals and tracing remain essential for decision models, and Allie gave two practical tips: build an audit layer that records every Jev call with inputs and outputs, and centralize all instructions and criteria in one file. With vibe-coded questions the model often guesses the intent, but the wording is usually poor; a single human edit in that central file can lift accuracy dramatically.

On limits and roadmap, the team said Jev today allows 32K tokens for state plus the largest question, capped at roughly 64K for state plus all questions combined, splitting into multiple calls if needed. Multimodal is officially “not yet,” and the near-term focus is capacity, speed and cost rather than a promised date for larger contexts or image inputs.

Real-time unlocks were the most exciting part. Allie described filtering a Twitch stream with 100,000 concurrent viewers with only 100–150 ms added delay, scoring each message for interestingness and letting viewers slide a threshold to show only the best questions; every message can be classified as question, statement or hype in parallel without breaking the bank. Her personal favorite is Jev playing Doom live, picking each frame which monster or item to look at — possible only because parallel decisions are that cheap and fast.

On training, the team was candid but brief: a mixture of pretrained models plus fully in-house synthetic data, with calibrated decisions via reinforcement learning for calibrated decisions and breakthroughs beyond post-training that they keep as secret sauce. The LangChain blog pegs Jev at up to 200× faster inference and 400× lower cost than comparable LLMs on classification tasks, which explains why ten questions cost barely more than one.

Confidence was the subtlest topic. The confidence returned is a deterministic computation over the probability map, essentially measuring how concentrated the distribution is. It is a good proxy when you expect a single dominant answer, but it is naturally low when several good options compete — as in Doom’s “where to look” or a shopping cart with multiple viable items — and low confidence does not automatically mean “fall back.” Sometimes you want the raw top probability, sometimes top minus second, sometimes the ratio.

The last warning was about large inputs and context engineering. The Model Jaggedness doc lists known weak spots with honesty as a cultural value; dumping an entire world of context into the state can degrade accuracy. Allie showed how an overly granular “recent actions” array with every bullet of damage hurt intelligence in the Doom harness. The guideline is to include only what the decision needs, decompose multi-factor questions, and always test on your own distribution before production.