Google Unveils New AI Weapon: Cloud TPU v5p Chip and Cloud Supercomputer

Google announced more AI developments at the end of 2023. The company introduced its most powerful Tensor Processing Unit (TPU) to date, the Cloud TPU v5p, and an AI supercomputer from Google Cloud.

"In the past five years, the parameters of generative AI models have increased tenfold annually, which requires higher training, tuning, and inference," said Amin Vahdat, Google Fellow and Vice President of Engineering for Machine Learning, Systems, and Cloud AI.

Cloud TPU v5p is an AI accelerator for training and serving models. Google designed the Cloud TPU to work with large models that have long training times, primarily composed of matrix computations, and no custom operations in their main training loop, such as TensorFlow or JAX. Each TPU v5p pod has 8,960 chips with high-bandwidth chip-to-chip interconnects.

Cloud TPU v5p builds upon previous versions such as v5e and v4. Google states that TPU v5p has twice the FLOPs of TPU v4 and four times the scalability. It can also train LLM models 2.8 times faster than TPU v4 and embed dense models 1.9 times faster than TPU v4.

Next is the new AI supercomputer, which includes an integrated system with open software, performance-optimized hardware, machine learning frameworks, and flexible consumption models. The idea is that this hybrid approach improves productivity and efficiency compared to looking at each component separately. The performance-optimized hardware of the AI supercomputer utilizes Google's Jupiter data center networking technology.

In a change of pace, Google provides open software support for machine learning frameworks such as JAX, PyTorch, and TensorFlow. This announcement comes after Meta and IBM launched the AI Alliance, which prioritizes open-source (which Google did not participate in). The AI supercomputer also offers two modes: Flex Start Mode and Calendar Mode.

Google shared this news while introducing Gemini, a new AI model that the company claims is the "largest and most capable" and has been launched on Bard and Pixel 8 Pro. It comes in three sizes: Gemini Pro, Gemini Ultra, and Gemini Nano.