The discussion on the use cases of large-scale language models is rapidly growing, and the release of Llama 2 by Meta has further fueled this trend as a new open-source model. Despite the public release of Meta's Llama, there are still significant barriers to running it on local hardware. To address this issue and truly enable everyone to harness the powerful capabilities of Llama 2, Meta has partnered with Qualcomm to optimize the model based on the AI capabilities of their chips, enabling it to run on devices.
Industry experts predict that open large-scale language models may create new AI-driven content generation, intelligent assistants, and productivity applications. The ability to run large-scale language models locally on devices will undoubtedly create a powerful new ecosystem, similar to the app explosion triggered by the iPhone.
This initiative not only lowers the barrier to access the model but also unlocks various possibilities for AI processing on devices. Meanwhile, the consumer hardware and software industry is also beginning to recognize the potential of AI at the edge. Apple introduced a neural engine in its M1 chip, adding a new type of processor to computers, ultimately providing developers with tools to create truly mainstream AI.
Qualcomm is developing a new AI chip on the Snapdragon platform. The chip manufacturer equips its chips with various AI capabilities using the Hexagon processor. By using a method called micro-watt inference, Qualcomm can integrate tensor cores, dedicated processing for SegNet, scalar and vector workloads into the AI processor, and then integrate it into Snapdragon mobile chips.
As part of the collaboration with Meta, Qualcomm will leverage the capabilities of the new AI-supported Snapdragon chips to implement Llama 2 on devices. Running the model on devices allows developers to not only reduce the cloud computing costs of their applications but also provide users with a higher level of privacy protection as no data is transmitted outside the device.
Running the model on devices enables the use of generative AI without the need for an internet connection. Additionally, the model can be personalized based on user preferences as it "lives" on the device. Llama 2 can also seamlessly integrate with Qualcomm's AI stack, which is a set of developer tools for further optimizing AI models running on devices.
Durga Malladi, Senior Vice President and General Manager of Qualcomm's Technology, Planning, and Edge Solutions business, said, "We appreciate Meta's approach to open and responsible AI and are committed to driving innovation and reducing barriers to usage for developers of any scale by introducing generative AI on devices."
Qualcomm has previously collaborated closely with Meta, primarily manufacturing chips for its Oculus Quest virtual reality headsets. The company has also partnered with Microsoft to help expand AI workloads on devices. As part of collaborations with other chip manufacturers such as Intel, AMD, and NVIDIA, Microsoft has launched a new Mixed AI Loop toolkit to support AI development at the edge. Overall, the edge AI hardware and software ecosystem is evolving towards edge AI, and the role that Llama 2 may play is more significant than anyone anticipated.
It seems that Meta has learned a lot from the leakage incident of the first-generation LLaMA model. Although the first version of the large-scale language model was only available to researchers and academic institutions, the model and its weights leaked on 4chan. This sparked an explosion of open-source innovations based on the LlaMa-based model.
In less than a month since its release, the open-source community has made improvements to LLaMA in various aspects. Researchers at Stanford University have created a version of LLaMA that can be trained for only $600, which has further facilitated the development of faster and smaller versions. Most of these versions can run on devices, allowing people worldwide to access their own large-scale language models.
One developer has ported the large-scale language model to C++, resulting in a model version that can run on mobile phones. This project, called LLaMA.cpp, has gained momentum in the open-source community, quantizing the weights of the model. This innovation enables it to run on the Google Pixel 5, although it can only generate one token per second.
As part of the latest collaboration with Meta, Snapdragon may gain insights into the internal workings of the model. This will enable chip manufacturers to make certain optimizations to make Llama 2 perform better than other models. Considering the expected release in 2024, Qualcomm is likely to explore other partnerships to coincide with the launch of its Snapdragon 8 Gen 3 chip.
The open-source community will undoubtedly contribute to the fully open Llama 2. This move, combined with the momentum of the edge AI industry, is the first step in supporting a vibrant edge AI ecosystem.
Source: https://analyticsindiamag.com/meta-qualcomm-partnership-will-bring-llama-2-to-the-masses/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.