Google Cloud Launches Two New AI Chips to Compete with Nvidia

Google Cloud unveiled its eighth-generation custom artificial intelligence processors, dubbed Tensor Processing Units (TPUs), on Wednesday. The new lineup is split into two distinct variants: the TPU 8t, engineered primarily for intensive model training, and the TPU 8i, optimized for inference workloads.

Inference refers to the ongoing execution phase where AI models process user prompts and generate outputs after initial training.

Aligning with industry expectations, the tech giant reports substantial performance leaps over previous iterations. The latest TPUs reportedly accelerate AI training by up to three times while delivering an 80% improvement in price-to-performance ratio. Furthermore, the architecture supports seamless orchestration of over one million TPUs within a single cluster. This scalability empowers clients with enhanced computational throughput while simultaneously lowering energy consumption and operational expenses. The designation "TPU" rather than "GPU" highlights their custom-built, energy-efficient architecture, originally derived from tensor-based computing principles.

Despite these advancements, Google’s silicon does not yet pose an existential threat to Nvidia. Similar to rivals like Microsoft Azure and Amazon Web Services, Google is deploying its proprietary chips to augment, rather than replace, its Nvidia-powered infrastructure. In fact, the company has committed to integrating Nvidia’s upcoming Vera Rubin accelerators into its cloud offerings later this year.

Looking ahead, hyperscalers investing in in-house AI accelerators may gradually decouple from Nvidia’s ecosystem. As enterprises migrate their AI workloads to these cloud platforms and optimize applications for custom silicon, reliance on third-party hardware could diminish over time.

For now, however, counting Nvidia out remains a risky proposition. Renowned semiconductor analyst Patrick Moorhead recently quipped on social media about his 2016 forecast that Google’s inaugural TPU might undermine Nvidia and Intel. Instead, Nvidia has surged to a nearly $5 trillion market valuation, thoroughly disproving that early skepticism.

Should Nvidia’s strategic roadmap hold steady, Google’s expansion as a premier AI cloud provider will likely drive additional demand for Nvidia’s hardware rather than cannibalize it, even as a growing share of workloads shift to Google’s proprietary processors.

Moreover, Google revealed a strategic partnership with Nvidia to co-develop networking architectures that maximize the efficiency of Nvidia-based deployments in the cloud. A key focus of this collaboration is enhancing Falcon, a software-defined networking framework originally open-sourced by Google in 2023 under the Open Compute Project (OCP), the leading consortium for open-source data center hardware.