Inside Google AI Infrastructure: Chip Mix, Global Deployment, and How Agents Rewrite Datacenter Design
Google's head of AI infrastructure Amin Vahdat, in conversation with Sequoia's Sonya Huang, argues that building AI infrastructure has become an art of anticipation — from TPU design trade-offs to how long-horizon agents are reshaping datacenters.
The central claims: specialization is unavoidable, because general-purpose clouds lose to purpose-built systems; FLOPS are a vanity metric while effective throughput is the accountability standard; and what doubles every six months is token-generation capacity, not raw FLOPS. That mismatch is the hard part — hardware is procured and powered on multi-year cycles while models iterate every few months. The load is not isolated to accelerators: CPU, storage and network demand rise together, and optical circuit switching lets a failed rack be swapped out in milliseconds. Power is treated as a long-term constraint rather than an engineering detail. Performance left on the table has to be deliberately locked into the stack, layer by layer. Running through the whole conversation is one question: how do you build multi-year infrastructure to serve a demand curve that is still changing every quarter?
The full text of this article is currently available in Chinese only. Read the original