NVIDIA recently announced its latest innovation, the next-generation NVIDIA GH200 Grace Hopper platform. This platform is powered by the innovative Grace Hopper superchip and features the world's leading HBM3e processor.
This new configuration is a game-changer, offering 3.5 times the memory capacity and 3 times the bandwidth of the current version. It includes a single server with 144 Arm Neoverse cores, providing 8 AI floating-point operations and 282GB of the latest HBM3e memory technology.
The technology is carefully designed to meet the demands of fast computing and exciting AI generation. The HBM3e memory is 50% faster than the existing HBM3, with a total bandwidth of 10TB/s, allowing the new platform to run models 3.5 times larger than previous versions and improve performance with 3 times faster memory bandwidth. NVIDIA expects to launch this new AI chip in the second quarter of 2024.
The new platform utilizes the Grace Hopper superchip, which can be connected to other superchips via NVIDIA NVLink, enabling them to deploy giant models for AI generation together. This high-speed, collaborative technology allows GPUs to fully access CPU memory, providing 1.2TB of fast memory in dual configurations.
Leading AMD and Intel
NVIDIA's next-generation GH200 surpasses the upcoming AMD 1300X. Recently, in order to compete with NVIDIA, AMD added an additional 64GB of HBM3 memory to the M1300X.
The M1300X combines CNA3 with industry-leading 192GB of HBM3 memory, providing a memory bandwidth of 5.2TB/s, while GH200 offers a bandwidth of 10TB/s. NVIDIA not only surpasses AMD but also leaves Intel far behind, as Intel has been striving to catch up in the race for training LLM GPUs.
GH200 far exceeds Intel's Gaudi2. Gaudi2's memory subsystem includes 96GB of HBM2E memory, providing a bandwidth of 2.45TB/s, along with 48MB of local SRAM, which has sufficient bandwidth to enable parallel operations of MME, TPC, DMAs, and RDMA NICs.
In the same trend, Intel announced its entry into the AI chip market this year, preparing to compete with NVIDIA and AMD. The upcoming Intel "Falcon Shores" chip is expected to showcase an impressive 288GB of memory capacity while supporting 8-bit floating-point calculations.
Falcon Shores will use conventional Ethernet switches, similar to the Intel Gaudi architecture designed for AI tasks. It will have various HBM3 memory options and adaptable I/O, potentially offering different memory options. Intel mentioned that it will provide up to 288GB of HBM3 memory and a total memory speed of 9.8TB/s.
Intel's Falcon Shores has been delayed and is planned for release in 2025, equipped only with GPU cores, with the introduction of CPU cores yet to be determined. During the latest earnings call, Intel CEO Pat Gelsinger stated that Gaudi3 will be a volume production product next year, and they are already working on Falcon Shores 2 for 2026.
Considering NVIDIA's GH200 plans being released earlier than Falcon Shore and Falcon Shore 2, it seems unlikely for Intel to surpass NVIDIA. Intel has formulated a series of ambitious strategies that must be successfully implemented in order to catch up with NVIDIA.
Software Aspect
An important aspect of NVIDIA's hardware success is CUDA, which enables parallel computing. CUDA can be considered as NVIDIA's moat in the AI field.
To compete with CUDA, AMD recently updated RocM. This is a significant step forward for AMD, as CUDA has always been one of NVIDIA's biggest advantages, especially in AI acceleration workloads.
RocM provides a large amount of memory bandwidth, allowing companies to purchase fewer GPUs, making AMD attractive to small companies with light to moderate AI workloads. Similarly, Intel has announced improvements to their CUDA alternative, oneAPI.
While NVIDIA takes a high-end market strategy, AMD and Intel can continue to focus on open-source solutions to compete with NVIDIA in the small company market.
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.