Qwen 3.8 27B on a Single 8GB Card: Better Than Expected

The opening claim is plain: Qwen 3.8 27B runs on a single RTX 4060 with 8GB, and every piece shown was produced in one shot with no agent looping. The author is upfront that the setup was slow and fiddly at first, yet still finds it remarkable that a model of this class produces finished work on such a modest card. The episode promises the full recipe: setup, harness and parameter choices, then final results next to Claude Opus.

The backdrop is the model's moment: Qwen 3.8 came out roughly two weeks earlier and quickly became the community's favorite topic. Despite its relatively small 27-billion-parameter size, it is described as matching Opus 4.6 on several coding and agent benchmarks, sometimes pulling ahead. Most guides recommend 24GB cards for it; this video probes the lower end instead.

The key insight comes from the quantization chart: the IQ4XS build stays within about six percent of the base model while shrinking it to around 14GB. That brings 16GB mid-range cards into play. The video asks one step further: what about 8 — too slow to bother? Time to test.

The configuration is frugal throughout: the Unsloth IQ4XS file, a 70K-token window judged plenty for a small project, and a 4-bit-quantized KV cache to save context memory. Roughly 26 layers go to the GPU, holding total use near 7GB with headroom left for the OS.

The fine tuning continues: the CPU thread pool is matched to the physical core count (six on the author's machine), and speculative decoding runs in MTP mode with at most two draft tokens. Thinking mode is switched on with the reasoning budget removed, for a concrete reason — the model can spend some 20K tokens just thinking through a mid-size website prompt, so a cap would abort it mid-thought. Sampling follows community-shared values: temperature 1, top K 20, repeat penalty 1, min P zero.

Harness choice is treated as first-order on 8GB: the tool with the leanest built-in system prompt wins, which is Pi. The logic is straightforward — with the card already at its memory limit, harness bloat taxes context and speed directly, so the lightest runner keeps the small card's burden from growing.

The first job is a Minecraft-themed 3D landing page. About an hour in, generation stalls in a loop while writing, and Pi cuts the write call short. Digging turns up two fixes: the HTTP timeout gets disabled because generation is so slow that normal limits kill the job, and the maximum output token count is raised so large files survive. The prompt runs again and lands after two and a half hours at about 4.2 tokens per second, starting near six and fading as context grows.

The other three jobs finish faster: a cyberpunk neon-city scene in Three.js in roughly an hour and a half at five tokens a second, a detailed personal-website prompt in about two hours at 4.6, and an interior-design landing page in an hour and a half at five. The same prompt goes to Opus 4.6 as well, and the video lines the two outputs up side by side each time.

The verdict is bold: Qwen 3.8 changes the game for local models. On 8GB the pace is still judged too slow for everyday use, but the output quality from a model this size is rated solid — in some tests better than the Opus 4.6 side. The projection offered is that a 16GB card lands around 12 to 14 tokens per second at 70K context, which counts as practical.

A closing question gets a straight answer: why not the 3-bit build on the 8GB rig? Under heavy CPU offload it ran slower than the 4-bit build, with quants not divisible by two paying a speed penalty. The takeaway is to stay in the 4-bit band for this kind of hybrid setup.