Is Bonsai 27B Really 98% of Qwen3 27B? A Hands-On Ternary Test on a Single RTX 3090

Can a single RTX 3090 run a 27-billion-parameter model well enough to replace its full-precision twin? That is the question Digital Spaceport puts on the bench, using PrismML's ternary Bonsai 27B. Viewers keep asking what the best model is for one card or two small cards, and the episode tests that directly. The headline claim frames the test: does a nine-times smaller build preserve almost all of the Qwen3 27B capability?

What Ternary Means — Three Values, Nine Times Smaller

Ternary sounds simple: each weight lives in one of three states, minus one, zero, plus one, instead of two. That third state trims quantization error per scale group. In the numbers PrismML publishes, a 27B model that occupies about 54 GB in 16-bit drops to about 5.9 GB in ternary and 3.9 GB in binary. Roughly a ninefold shrink, which puts the full model comfortably inside a 24 GB card's budget.

The shrink is not a thin wrapper. Bonsai builds on Qwen3 27B, shifts around three-quarters of its attention layers to linear form, and keeps a 262K token window workable on device. The language stack is compressed end to end — embeddings, attention projections, MLPs, and the LM head share the same low-bit representation with no high-precision bypass. The vision tower ships separately in 4-bit HQQ so screenshots and documents travel with text. A DSpark speculative decoder sits on top as a lossless draft-and-verify boost.

The Rig — Proxmox LXC and a Single 3090

The setup is a familiar self-hosting rig: a Proxmox 9 LXC container numbered 118 on the Prox 2 host. The Hugging Face download is quick because the artifact is small enough to skip the key. Launch goes through PrismML's llama server starter from the demo repo. A key detail is that mainline llama.cpp does not yet carry these low-bit kernels upstream, so you must pull PrismML's fork for correct outputs. Environment wiring binds BONSAI_HOST to 0.0.0.0, leaves contact size on auto, and keeps speculative and KV-cache squeeze off for this run.

Once loaded, the card shows about 16.772 GB occupied, leaving headroom inside 24 GB. The server becomes reachable on 192.168.1.61:8080. Early chat throughput sits near 68 to 69 tokens per second and holds there for a while. For reference, the prior full-precision Qwen3 27B run on four RTX 3090s with vLLM landed near 40 tokens per second. Context is set at 131K; bumping it makes sense for Hermes Agent style multi-step loops. The vision path stays enabled.

Arcade Test — Same Inputs, Different Outputs

The core test is a fair head-to-head: the same arcade package previously generated with full-precision Qwen3 27B is generated again with Bonsai. The Hermes-style agent engages, thinking traces appear, prompt processing kicks around 1,250 tokens per second. One generation runs about 25,000 tokens, prompt work is negligible, and decode settles near 64.4 tokens per second. Results are slated for side-by-side play at arcade.digitalspaceport.com.

Each of the three mini games tells a different slice. The shooter moves only sideways, enemy flow feels excessive, score sits tiny at about 16,000 in the upper left, and the boss fires the wrong way. Grades there land in the C-minus to D-plus range. Space Racer feels stiff, lane changes are harsh, the tiny font does not help, and controls feel close to unplayable. Pixel Breaker lands better: the fence and cat vignette is decent, music feels corny yet gameplay is smoother, not full-screen but the breakable stars react as expected.

Is It 98 Percent — The Field Note

The final grade in the video is blunt: two fails and a D-plus, with quality clearly behind the full-precision run. The host says it does not feel like 98 percent, and the quirks in creative layers — level logic, music, small score differences — support that. The story lines up with lab numbers too: about 80.49 versus 85.0 on a 15-benchmark average, roughly 94.6 percent retained. Lab average and single creative generation are not the same; the latter is harsher.

Where Speed and Tool Use Shine

The bright spot is speed and tool use. Chat feels fluid, non-code tool calls land well, and no missed calls were seen in this run. Single-stream decode near 68 tokens per second on a 3090 translates to up to 134 tokens per second ternary and 163 in 1-bit on a 5090, and around 58 and 87 on an M5 Max in the published figures. DSpark's lossless 1.34x lift is part of that picture. It is enough to make a single card feel viable for a persistent single-thread assistant.

The boundary is drawn at sustained coding agents. The episode states it is fine for chat and single-thread agent flows, but not yet a good time for agentic code development. The reason is the quality drop seen in creative and structural code generation; even with long context and linear attention, coherence across levels slips. There is room to improve as the forked kernels land upstream and the context budget grows, and the two 3060 12 GB setup gets its own speed check.

Practical take ties to hardware prices. The channel's live board shows used RTX 3090s near 1,400 dollars and climbing, while a 3060 12 GB sits near 300 dollars. The dual 3060 configuration is tested separately for speed. A single 3090 already sits at the practical threshold for memory and throughput. Picking this build makes sense if your plan is an assistant, document reading, and light automation on one card; making fully autonomous code agents or vision-heavy game generation the main workload is early.