Qualcomm unveils Snapdragon 8 Gen3 chip capable of running large models and AI software

Qualcomm's new flagship mobile chip, Snapdragon 8 Gen 3, focuses on the device's internal AI capabilities from start to finish. The chip directly integrates generative AI capabilities, significantly speeding up the processing-intensive tasks typically outsourced to the cloud. This is a major advancement in the field of AI and is expected to be featured in smartphones with the new chip in the coming weeks.

Snapdragon 8 Gen 3 supports chatbots trained on Meta's Llama 2 and can accept text, image, and voice inputs. It can answer questions and generate images or text. The chip also runs Stable Diffusion, an AI image generator demonstrated by Qualcomm earlier this year, on the device. The company claims it can generate an image in less than a second, compared to the previous technology that took about 15 seconds, still much faster than a well-equipped laptop that takes several minutes.

All these features are integrated into Qualcomm's AI engine, utilizing the company's Hexagon neural processor. The perception hub uses OpenAI's Whisper for speech recognition. The perception hub also provides the AI engine with user information, including location, to provide more personalized responses. It even conveys your preferred activities, age, and "health level".

Generative AI also plays a significant role in the new image processing capabilities of Snapdragon 8 Gen 3. It supports generative filling for image expansion, allowing for scaling and cropping of photos on the device. The video capabilities are impressive as well, with an object erasure feature for videos - just a tap and unwanted subjects disappear! Additionally, it supports night mode recording on the device, up to 4K/30p. Google announced its Night Sight video feature will be introduced in the Pixel 8 Pro in the near future, but it will run in the cloud.

There is also a feature called Vlogger's View, which overlays selfie and rear camera videos in one view. It's not a picture-in-picture effect; this feature uses improved image segmentation technology to remove the background from the selfie video, making it look like what you see from the rear camera.

To address the potential misuse of tools, Qualcomm is partnering with a company called Truepic. Its technology aligns with the open standard of the Coalition for Content Provenance and Authenticity, similar to metadata used to ensure the authenticity of photos or videos. It binds authentication with digital assets (such as photos or videos) using cryptography, making it less susceptible to tampering compared to EXIF data.

Although Snapdragon 8 Gen 3 is not entirely AI-focused, Qualcomm claims that it offers a 10% improvement in energy efficiency compared to 8 Gen 2, thanks to the enhanced efficiency of the Kryo CPU and Adreno GPU. It supports Dolby's HDR photo format, aligning with Apple and Google's push for high dynamic range image capture in their latest devices. It is equipped with the X75 modem-RF, providing more support for 5G carrier aggregation. The GPU supports improved hardware-based ray tracing for more realistic light reflection effects in mobile games. The system will support refresh rates of up to 240Hz on compatible external displays.

Additionally, there is a new system called Snapdragon Seamless, aimed at facilitating pairing between devices from different manufacturers and operating systems. Qualcomm states that this will enable easier switching between devices, such as switching from a phone to a PC for video calls and then switching back to the phone for music playback.

This seems primarily aimed at making it easier for Android and Windows devices to interact with each other in the initial stages. Qualcomm states that this is an open platform that anyone can join, although initially, the technology will be primarily used by Qualcomm's original equipment manufacturers and operating system partners.