AMD Launches MI300X AI Chip as a "Generative AI Accelerator"

2023-06-14

Lisa Su stated that being able to run the entire 40 billion parameter model in memory actually reduces the number of required GPUs. On Tuesday, Lisa Su, CEO of AMD, unveiled a chip in San Francisco that is the core of the company's AI computing strategy and supports so-called generative AI tasks such as large language models with its massive memory capacity and data throughput. The component is called Instinct MI300X and is a follow-up version of the previously announced MI300A. The chip is actually a combination of multiple "chips" that are connected in a package through shared memory and network links. In front of the invited audience at the Fairmont Hotel in downtown San Francisco, Su referred to the component as a "generative AI accelerator" and stated that the GPU chip it contains, the CDNA 3 series, is "designed for AI and high-performance computing (HPC) workloads." MI300X is the "pure GPU" version of this component. MI300A is a combination of three Zen4 CPU chips and multiple GPU chips. However, in MI300X, the CPUs have been replaced with two additional CDNA 3 chips. MI300X increases the number of transistors from 146 billion to 153 billion and increases the shared DRAM memory from 128GB in MI300A to 192GB. At the same time, the memory bandwidth increases from 800GB per second to 5.2TB per second. "We strategically use chips in this product," Su said, "because it allows for mixing and matching different types of computing, such as swapping CPUs or GPUs." Su stated that MI300X offers 2.4 times the memory capacity and 1.6 times the memory bandwidth of Nvidia's H100 "Hopper" GPU. "Generative AI and large language models have changed the existing landscape. Whether it's training or inference, the demand for more computing is growing exponentially," Su said. To demonstrate the necessity of powerful computing, Su showcased the currently most popular large language model, the open-source Falcon-40B. Language models require more computing power as they are built with an increasing number of neural network "parameters." Falcon-40B consists of 40 billion parameters. She said that MI300X is the first chip powerful enough to run neural networks of this scale entirely in memory without having to move data back and forth between external storage. Su demonstrated how MI300X used Falcon-40B to create a poem about San Francisco. "A single MI300X can run a network model with approximately 80 billion parameters in memory," she said. She continued, "Compared to competitors, MI300X offers 2.4 times the memory capacity and 1.6 times the memory bandwidth, and all this additional memory capacity actually gives an advantage to large language models because we can run larger models directly in memory." Su stated that being able to run the entire model in memory means "for the largest models, it actually reduces the number of GPUs you need, greatly improving performance, especially inference performance, while reducing total ownership costs." "With MI300X, you can reduce the number of GPUs, which will become more important as the model size continues to grow," she said. "With more memory capacity, more memory bandwidth, and fewer GPU requirements, we can run more inference work on each GPU," Su said. She added that this will lower the total ownership cost of large language models and make the technology more accessible. To compete with Nvidia's DGX systems, Su introduced a series of AI computers called "AMD Instinct Platform." The first example is a combination of 8 MI300X with 1.5TB of HBM3 memory. The server complies with industry-standard open compute platform specifications. "For customers, they can use all this AI computing power in the standard industry platform and directly integrate it into their existing infrastructure," Su said. Unlike MI300X, MI300A is just a GPU. The existing MI300A will compete with Nvidia's Grace Hopper combination chip, which uses Nvidia's Grace CPU and Hopper GPU, and the company announced last month that the chip is in full production. Su pointed out that MI300A is being incorporated into the El Capitan supercomputer being built at the Department of Energy's Lawrence Livermore National Laboratory. Su said that MI300A is currently being showcased as a sample to AMD customers, and MI300X will begin sampling to customers in the third quarter of this year. She said that both products will be in mass production in the fourth quarter. Source: https://www.zdnet.com/article/amd-unveils-mi300x-ai-chip-as-generative-ai-accelerator/

RECENT AI NEWS

RECENT AI TOOLS