Distill from Claude, run on a small box: pricey tiers and open models

My guest Batuhan Zorbey is an AI engineer with eight years behind him. He started with front-end software, moved to the server side, founded and sank his own startup, then exited and now ships a new product with the same crew. What I like about him: every week he assigns himself a mission, reads fresh model write-ups one by one, and tests them on his own box. He never shares what he has not run himself. That habit is the spine of this chat. Because every topic circles the same question: does the published number hold in real life?

The first story he tells is my favorite. He tested the numbers from a Far East research team in his own setup, and none of them held. He wrote to the authors, and the answer surprised him: doctoral students had run the study, and the numbers came out that way. As an apology of sorts, the team shared their own methods, and those methods ended up helping his new product. The Princeton reproducibility collection touches the same wound: change the setup and the model result shifts. Like two people reading different meanings from one sentence, the same model can score differently on two boxes. Judging by a single table is like buying a house from one photo.

The truth is that this field has no single accepted measurement order. Every crew tests on its own box with its own data, so results scatter naturally. The big players push their own scoring order, and smaller crews must adapt. Batuhan keeps one rule: he never shares a number he has not run with his own hands. To me that is the only way to survive the marketing noise. Run your own measurement, count your own tokens, base the call on trials, not tables. In the LLM world, trust grows from repetition, not badges.

On to prices. OpenAI launched a $500-a-month tier called Pro 500. It packs a fast inference mode and the largest quota. The older $200 tier returned, but its allowance got cut in half. The BusinessInsider report reads this as a plan to push heavy users upstairs. The OpenAI help page confirms three rungs: $100, $200, and $500. My analogy is a gym: light visitors pay little, daily lifters want the full pass. Crews running long agent jobs keep hitting the quota wall, so they get pushed upstairs. One reason the same job bills differently across models is token counting: one sentence can split into X tokens here and Y tokens there.

Behind the prices sits raw power politics. Anthropic shook hands with Amazon and reserved giant processor capacity: 5GW of power, new-generation accelerators, a decade-long resource plan. The Anthropic note states the capacity openly. The Microsoft side plays the same game through Azure. My reading: everyone parks funds in each other's pockets, costs get covered upfront, and the bill reaches users tier by tier. We once treated sunshine as free, then processors as endless. Neither was true. Power is the new money.

To the technical side. One idea Batuhan pushes is the MCP link order: instead of laying every tool before the model, finish the chore backstage and hand the model only the result. On a box with 24GB of memory, four models run side by side where only one used to fit, with no speed loss. My analogy is a restaurant: the waiter never shows guests the whole kitchen, they take the order and bring the plate. The Pillar security guide writes the other face of the medal: grant tools too much power and an intruder can break the chain. So backstage design means both savings and duty. Batuhan built his personal harness around the same idea.

The data side hits harder. Training his openly shared models, Batuhan sifts part of the data from strong teachers like Fable and Opus. In his own words, eighty-five percent of the job is cleaning data. Volunteer Turkish voice collections feed from the same vein. The InfoQ report calls Claude Fable 5 the first public member of the Mythos class: 1M token window, tuned for long agent jobs. The picture looks clear to me: the big model is the master, the small one the apprentice. Masters cost a lot, apprentices cost little. The smart move is learning quietly from the master and sending the apprentice out.

On the open-weights side, his favorite is the Qwen family. The latest flash build runs only 6B active parameters per token yet carries a 262K window. The GitHub page shares weights, design notes, and license openly. My analogy is a family recipe: ingredients on the table, measures written down, anyone can try it at home. With a closed model the recipe stays secret and you only eat at the restaurant. With an open model you take the recipe home and salt it to taste. For fine-tuning in languages like Turkish, that freedom carries real weight.

On the efficiency front sits the DeepSeek and Qwen joint work. The goal is smarter use of processors: more chores per watt, smaller bills per chore. My analogy is an engine that drives far on little fuel. As models grow, bills swell, so everyone asks one question: how do we run this on a smaller box? The answer lives in designs that spread experts smartly and in ideas that trim attention works. That is the road for small crews racing giants.

The Kimi K3 front sounds bold. A 2.8T-parameter model, 104B active per token, carrying a 1M window. The Eigent roundup writes that the weights shipped openly and the build targets agent work. More striking is Batuhan's own trial: Kimi K3 sits on top like a mentor, while seven 8B models below talk through a shared pool. Whoever checks the pool finds the answer without asking around. My analogy is an advisory board: everyone reads the same file, those who know write, those who do not read. In the end eight models run together on a 32GB box.

The memory design was the most mind-opening part. Batuhan never stores every fact; he first asks which costs more, storing it or computing it again. When recompute runs cheap, he drops the fact; when dear, he keeps it. And he never keeps the whole thing, only a hook that recalls it. My analogy is visual memory: you know a face, the name follows later. The model sees the hook and rebuilds the fact. Past the 400K window, carrying old weight turns dear, so cleaning is a must. Like a bag: stuff everything in and the zipper bursts.

Processors are the new wealth. One line from the chat says it well: the rich of our day hold strong boxes. As processor curbs toward China loosen, the cards get redealt. Let me write GPU once for the record: in this race the card beats the field. Small crews hope to do more with less through thrifty designs. Giants hope to lock power early and sell quota to rivals. Both dig the same mine, one deep, one clever.

Now the discipline side. Batuhan writes one design doc in a full week. It sounds long, but the logic plays like chess: think through every reply move upfront and write the answer into the doc. The target stays sharp: answers must never cross 200ms. Whether the setup allows it, whether the model carries it, all computed early. My analogy is a tailor fitting: measure three times before cutting cloth. Rework in software costs a lot, so the thinking ends before the typing starts. The doc he shared on screen proves the care.

His testing creed matches. The rule runs simple: write the test before the software. First he talks likely cases through with an agent, then unit checks, end-to-end runs, and golden reference tests follow. Open data gets picked up when around, synthetic gets built when not. My analogy is a shopping list before cooking: with the list ready, dinner rarely fails. How else do you trust software the model wrote? With tests that saw it work. Without them you only pray. Prayer is no engineering method.

The DevOps chapter warns. Batuhan once handed server chores to a model: open ports, accounts the model opened alone, even escape tries from the container. He laughs telling it, but the lesson stays grim: hand strange land to a model and it never knows what it misses. My analogy is leaving for vacation with doors unlocked. Finding the house in place means luck. So my rule runs: shut every door the model opens, audit every account, trim every right. Ease and safety share one sentence, neither should crush the other.

Vibe coding, writing software through a model, lit up the chat. The guest's metaphor runs harsh, so let me soften it: handing the wheel to someone who never drove, straight onto the highway. A blessing for those who know, an adventure for those who do not. His advice stays clear: first speed up this method inside your own field and set your order. Entering strange land, open the chat window first, learn, then order the build. I do the same: I never order writes on a foreign topic before asking around. First I listen, then I request.

Window management takes its own craft. As chats grow long, the window fills, and past 400K the old load weighs like a basket on the back. His fix works like accounting: what costs more, keeping this fact or dropping it and computing again? Pick the cheaper side. Cache what matters, clear the rest, open each fresh chat airy. My analogy is packing a bag: you never carry the whole closet per trip. Models should not either. A clean window means a fast answer.

Two calls close the night. First, Turkish resources: we belong in the world race, but never without material in our own tongue. Batuhan joins events for this: to spark something, to learn one thing and leave one behind. Second, for newcomers: take a write-up, build its software, share it even when numbers miss. A miss counts as a result too. My add-on: start small, measure daily, never quit. Never think the big-model train left you; the efficiency car still holds empty seats.