The Engine Under the Hood

Look at the price-performance charts for large language models and one company keeps surfacing in the corner that matters most. Google. The same Google everyone wrote off is now quietly winning the combination that counts: models that are genuinely good and genuinely cheap.

The LMSYS Chatbot Arena ranks models by blind tests, and the Gemini family is all over it. The small Gemma models. Flash, which is fast and cheap. Pro at the top end. At every tier the pattern holds: Google’s models cost less than comparable alternatives. The whole lineup is laid out that way.

The obvious question is how. The answer sits underneath the models, in the chips they run on.

Run AI at any serious scale and you pay a tax to NVIDIA, because their GPUs are the only realistic option and they are priced accordingly. Their hardware is excellent. That’s exactly why the tax sticks.

Google chose a different route more than a decade ago: build its own chips. The Tensor Processing Units are vertical integration in the mold of Apple, where the people who design the hardware also design the software running on it. Google didn’t rent a factory, it built one, and it designed every machine inside.

A GPU is a general-purpose engine. Graphics, scientific computing, AI. A TPU does one thing: run neural networks. Neural networks at heart are a long series of matrix multiplications, tensors getting multiplied together over and over. Google’s TPUs strip out everything that isn’t that one operation and perfect the Matrix Multiply Unit at the center of the chip. It’s the difference between a sedan that hauls anything and a dedicated race car built for one track.

The specialization goes further. Modern models handle sparse tensors. Pull one user’s viewing history from billions of hours of YouTube and you get an enormous array where nearly every entry is zero, because they haven’t watched that video. A general-purpose chip multiplies those zeros anyway, wasting power and time. Google’s SparseCores are built to skip the zeros and compute only the non-zero entries. That efficiency is the reason Google Search and Ads can run at planetary scale without the compute bill eating the company.

The newer TPU generations, Trillium and Ironwood, push the same direction: more memory, more efficiency at the exact workloads that are becoming AI’s biggest bottleneck. They’re getting faster, and they’re also getting cheaper per unit of useful work, which is the number that decides what AI products can cost.

So while the world argues about which chatbot writes the better poem, the contest underneath is about economics. Most companies in the AI race buy their fuel from a single supplier. Google owns the refinery.