Jev Tested on 300 Trading Calls: What a Live Lab Revealed About AI Decisions

TypeSafe's Jev does not chat and does not write sentences; it returns typed, probabilistic decisions inside a schema the developer defines in advance. The Fintech Builder's lab takes that idea straight to markets: at every cutoff the model sees 90 daily candles, the task rules and 15 typed questions, answering them all in one parallel call. The company advertises 70–500 millisecond responses, parallel sampling and a price of $0.042 per million input tokens with free output, aiming to be two orders of magnitude faster and cheaper than chat models — yet the video shows speed does not automatically convert into edge.

How the lab is built: five markets, 300 cutoffs, 90 candles

The experiment spans 300 cutoffs over two years, 60 each in NVIDIA, JP Morgan, Exxon, Bitcoin and the gold ETF GLD. At each cutoff Jev sees only the 90 daily bars up to the close; everything after is hidden. Scoring is kept deliberately plain: entries trigger at the next open, stops, targets and holding periods come from the model's own answers and run up to eight bars. Costs are five basis points each way on $10,000, and one rule shapes the whole result — if the model returns 'not applicable' for the stop, the trade runs with no stop and nobody patches the answer. Every cutoff is also scored as a simple buy-and-hold over the same horizon, so active decisions can be judged against raw beta.

What Jev returns is a packet, not a single word. A choice question distributes probability across options — for example long 62%, no trade 31%, short 7%, with 44% confidence in the choice. A yes-no question returns the chance that the answer is yes, such as 'a setup exists, 59%'. A score question lands on a scale, like setup strength 3.65 out of six. The final five answers form a full plan: direction, entry style, stop, target and holding period. Because each answer is typed, it can be checked against what the market did next. The lab does exactly that, and no API layer forces the answers inside one call to agree.

The cleanest snapshot is the close of September 30, 2025. Jev leans long at 62% with 44% confidence, and the supporting reads align: firmly bullish view, strong bullish momentum, solid setup and a complete plan — 2% stop, 3% target, hold up to three bars. It buys the next open at 185.24 and hits the target two bars later for about 2.9% after costs, roughly $290, while holding the same eight-bar window would have lost 1.2%. The decision, the read and the plan point the same way, and the video keeps that picture in mind because most trades did not look like it.

Headline score: mostly flat, few trades, 50% hit rate

Overall Jev sits out 254 of 300 decisions, about 85% of the time. The remaining 46 trades split evenly, 23 wins and 23 losses. The net is +3.82%, $382 on the $10k-per-trade sizing, a profit factor of 1.06 and +0.08% per trade. For context, buying at every cutoff and holding eight bars generated more than $17,700 in total — not an apples-to-apples total because capital is put to work 300 times instead of 46. Even on a per-trade basis, buy-and-hold made 0.59% versus Jev's 0.08. The equity curve peaked in November 2025 on a single Bitcoin short and then gave back over $2,000, a reminder of how much variance a small trade count brings.

The market split is uneven. NVIDIA won four of seven trades yet lost $1,100, dragged down by two shorts. JP Morgan and Exxon left the run around –$1,400 after the first 108 decisions. Bitcoin is the only market where buying every cutoff lost money and Jev made money, contributing +$963. The gold ETF GLD helped close the gap with seven wins in ten trades. By outcome, 12 trades hit their target, 15 hit their stop and 19 expired on time. Longs earned about $900 while shorts lost $518. The distribution shows direction choice alone explains much of the return dispersion.

Why confidence ran backwards

Sizing by confidence would have hurt. Trades taken below 35% confidence averaged +1%, while the three trades above 50% averaged –3.1%, and Jev never took a trade above 55%. Winners and losers were nearly identical on the model's own diagnostics — confidence 32% vs 34%, 'setup exists' 47% in both groups — so those signals did not separate good from bad. The video pairs this with a small Apple pilot on 12 cutoffs: candles-only versus candles plus ten indicators. Adding indicators flipped four longs into shorts, all four hit their stop and the loss jumped from $118 to $840. The two harnesses agreed on only four of 12 decisions, all of them no-trade. It was a twelve-cutoff anecdote, which is why the full suite with indicators is now queued.

The sharpest finding is internal disagreement. Six of seven cross-checks that look for answers contradicting the trade found cases. 32 of 46 trades were taken while the model judged a setup probably absent, 21 while it signaled staying flat due to uncertainty, and nine had no stop at all. The most telling: 16 trades lacked a take profit — exactly the 16 shorts. Every short came with entry style, take profit and holding period marked not applicable, as if the plan itself said 'do not trade'. Direction always matched the side taken, but the rest of the plan fought the direction. The lab patches nothing, so the results stay raw.

Two trades show both extremes in the same template. On March 30, 2026 NVIDIA, a 40% confidence short with a strongly bearish read but no stop, no target and an eight-bar hold drifted as the stock rallied, producing the worst loss at –13.07%. The identical 'no stop, no target, high stay-flat signal' template on November 13, 2025 in Bitcoin was also a short, this time with a 70% stay-flat signal, and Bitcoin's near 15% slide over eight days turned it into the best win at +14.55%. The most confident short, JP Morgan on March 13, 2025 at 55% and a 96% bearish read, rose about 10% for a –10.19% loss. The 12 trades with no contradictions gained 6.36%, while the other 34 lost 2.54% — a clean separation that puts coherence at the center of performance.

The same tension appears on the other side. 45 times Jev chose no trade yet still picked a direction and a complete plan — on February 10, 2025 in NVIDIA a 52% no-trade call was paired with a 62% long plan, 2% stop, 3% target and a two-to-three-bar hold. Staying flat was not particularly selective: price moved on average 3.84% over eight bars after a pass versus 4.15% after a trade, and the lab counted 39 advances and 26 declines of 5% or more that were left on the table, including Bitcoin on August 18, 2026 where a 76% confidence pass was followed by a 22% rally over eight days. The video therefore labels Jev cautious but not well aimed and closes by promising two extensions: the full suite with indicators and a blind variant that hides symbols and dates to test whether the model reads the chart or recalls history.