TPU vs GPU: Google Chases NVDA's Crown

For a decade Google built a processor it never sold; the parts stayed inside its own buildings. Then Anthropic, the lab behind Claude, signed for up to a million TPUs in a deal worth tens of $B. Twenty-six days later came the second move: $30B of Azure capacity through Microsoft, plus NVDA. So the most-wanted customer of the AI race keeps both doors open. The question writes itself: what keeps that customer with NVDA?

The story starts with voice search in 2013. Engineers ran the numbers: three minutes of voice queries per person per day, and ordinary processors would force Google to double its data centers. So the TPU was born: a grid of small calculators where numbers wait inside cells, data enters once and passes cell to cell. The GPU is the opposite: thousands of flexible workers, each fetching its own numbers from memory while queues grow. The ant image is famous: GPU ants head for the pond one by one; TPU ants hand the water down the line. The team raised the first TPU in 15 months; by the 2016 announcement it already served Search and the AlphaGo matches against Lee Sedol. That same year Jensen Huang hand-delivered an AI system to OpenAI. One side built processors for its own pain; the other sold systems for everyone else's.

NVDA is the visible tree; the roots run underground, and their name is CUDA. Spread since 2006, that software layer grew together with PyTorch: millions of developers, years of accumulated practice. Moving to another processor means weeks of rewriting and re-testing. Google grew its own roots, JAX and XLA, where Gemini was trained. But most outside teams speak PyTorch. In 2026 TorchTPU arrived: PyTorch workloads run on TPUs with only small changes. Roots decide how easily teams move. A parallel step came through GitHub: a unified TPU backend for vLLM passes JAX and PyTorch model definitions through one lowering path. In Google's telling, Ironwood pairs 42.5 exaflops of FP8 compute for both training and serving.

When training ends, the real shift begins: serving models to users. Every reply puts the processors back to work. The TPU was born for this job, and Ironwood leans into serving: wider memory, faster roads to it, quicker links between processors. But NVDA never idles: grid-math units join its GPUs, and 72-unit racks sell as a single system. In November 2025 Gemini 3 lands, trained on TPUs alone; a week later Meta's TPU rental talks surface, and NVDA shares lose value. NVDA does something rare and comments on a rival: glad about Google's success, it says, and a generation ahead of the industry. A month later the $20B Groq licensing deal follows; founder Jonathan Ross started the TPU effort at Google twelve years earlier. The EconomicTimes summary notes Ross and his team joining NVDA while GroqCloud runs on alone.

Hourly price misleads: a cheap system running long can cost more by the finish. Google holds one genuine edge: a flagship NVDA processor costs about $6,400 to make and sells in the $30K–$40K band, a margin Google skips on its own parts. But savings inside Google do not land unchanged on the customer bill. And Google's claim of 80% better performance per dollar measures its new TPU against its own older one, not against NVDA. Only one test counts: same quality, same speed, lower bill — on the customer's own model.

Still, something shifted: Google now sells TPU systems, not only rents them. Per the Yahoo report, Meta discusses renting first and running TPUs inside its own buildings from 2027, with $B-scale figures on the table. Apple's case proves scale too: its server model trained on more than 8,000 TPUs. At Broadcom, expectations point to tens of $B in yearly TPU shipments, Anthropic the largest buyer. Huang pushes back: without Anthropic, where would TPU growth be? Yet the figures stand: the Yahoo page on NVDA's quarter lists $96.2B in revenue with $89B from data centers. For Google, winning need not mean pulling every GPU; it means earning the next order and a slice of serving work.

In 2013 engineers looked at three minutes of voice search and built the processor that changed their own math. Today Google offers other firms the same chance: to change theirs. Replacing every NVDA processor is unnecessary; giving the next order a reason is enough. The verdict lands on the bill, not in the lab.