Lisa Su stated that being able to run the entire 40 billion parameter model in memory actually reduces the number of required GPUs.
On Tuesday, Lisa Su, CEO of AMD, unveiled a chip in San Francisco that is the core of the company's AI computing strategy and supports so-called generative AI tasks such as large language models with its massive memory capacity and data throughput.
The component is called Instinct MI300X and is a follow-up version of the previously announced MI300A. The chip is actually a combination of multiple "chips" that are connected in a package through shared memory and network links.
In front of the invited audience at the Fairmont Hotel in downtown San Francisco, Su referred to the component as a "generative AI accelerator" and stated that the GPU chip it contains, the CDNA 3 series, is "designed for AI and high-performance computing (HPC) workloads."
MI300X is the "pure GPU" version of this component. MI300A is a combination of three Zen4 CPU chips and multiple GPU chips. However, in MI300X, the CPUs have been replaced with two additional CDNA 3 chips.
MI300X increases the number of transistors from 146 billion to 153 billion and increases the shared DRAM memory from 128GB in MI300A to 192GB. At the same time, the memory bandwidth increases from 800GB per second to 5.2TB per second.
"We strategically use chips in this product," Su said, "because it allows for mixing and matching different types of computing, such as swapping CPUs or GPUs."
Su stated that MI300X offers 2.4 times the memory capacity and 1.6 times the memory bandwidth of Nvidia's H100 "Hopper" GPU.
"Generative AI and large language models have changed the existing landscape. Whether it's training or inference, the demand for more computing is growing exponentially," Su said.
To demonstrate the necessity of powerful computing, Su showcased the currently most popular large language model, the open-source Falcon-40B. Language models require more computing power as they are built with an increasing number of neural network "parameters." Falcon-40B consists of 40 billion parameters.
She said that MI300X is the first chip powerful enough to run neural networks of this scale entirely in memory without having to move data back and forth between external storage.
Su demonstrated how MI300X used Falcon-40B to create a poem about San Francisco.
"A single MI300X can run a network model with approximately 80 billion parameters in memory," she said.
She continued, "Compared to competitors, MI300X offers 2.4 times the memory capacity and 1.6 times the memory bandwidth, and all this additional memory capacity actually gives an advantage to large language models because we can run larger models directly in memory."
Su stated that being able to run the entire model in memory means "for the largest models, it actually reduces the number of GPUs you need, greatly improving performance, especially inference performance, while reducing total ownership costs."
"With MI300X, you can reduce the number of GPUs, which will become more important as the model size continues to grow," she said. "With more memory capacity, more memory bandwidth, and fewer GPU requirements, we can run more inference work on each GPU," Su said. She added that this will lower the total ownership cost of large language models and make the technology more accessible.
To compete with Nvidia's DGX systems, Su introduced a series of AI computers called "AMD Instinct Platform." The first example is a combination of 8 MI300X with 1.5TB of HBM3 memory. The server complies with industry-standard open compute platform specifications.
"For customers, they can use all this AI computing power in the standard industry platform and directly integrate it into their existing infrastructure," Su said.
Unlike MI300X, MI300A is just a GPU. The existing MI300A will compete with Nvidia's Grace Hopper combination chip, which uses Nvidia's Grace CPU and Hopper GPU, and the company announced last month that the chip is in full production.
Su pointed out that MI300A is being incorporated into the El Capitan supercomputer being built at the Department of Energy's Lawrence Livermore National Laboratory.
Su said that MI300A is currently being showcased as a sample to AMD customers, and MI300X will begin sampling to customers in the third quarter of this year. She said that both products will be in mass production in the fourth quarter.
Source: https://www.zdnet.com/article/amd-unveils-mi300x-ai-chip-as-generative-ai-accelerator/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.