When Meta's LLaMA code leaked on GitHub, researchers worldwide gained access to the first GPT-level large language model (LLM). Subsequently, a series of LLMs were launched, injecting a new dimension into open-source artificial intelligence. LLaMA provided a stage for models like Alpaca and Vicuna-13B from Stanford University, becoming a representative of open-source technology.
However, a new competitor seems to have emerged - Falcon. Developed by the Abu Dhabi Technology Innovation Institute (TII) in the United Arab Emirates, Falcon outperforms LLaMA in terms of performance. It has three different variants: 1B, 7B, and 40B.
According to the institute, FalconLM is the most powerful open-source language model to date. The largest variant, Falcon 40B, has 40 billion parameters, which is smaller compared to LLaMA's 6.5 billion parameters. Faisal Al Bannai, Secretary-General of the Advanced Technology Research Council (ATRC), believes that Falcon's release will break the access restrictions of LLMs, enabling researchers and entrepreneurs to provide the most innovative use cases.
Top of the leaderboard
Two variants of FalconLM, Falcon 40B Instruct and Falcon 40B, rank first on the Hugging Face OpenLLM leaderboard, while Meta's LLaMA ranks third. Hugging Face evaluated the performance of these models based on four popular benchmark tests: AI2 Reasoning Challenge, HellaSwag, MMLU, and TruthfulQA.
Although the paper on LLM has not been published, we have learned that Falcon 40B underwent extensive training on a massive dataset, including one trillion tokens from a carefully curated refined-web dataset, ensuring its quality and relevance. Researchers revealed a particular focus on achieving high-quality data in large-scale scenarios. It is well known that LLMs are highly sensitive to the quality of their training data, so considerable effort was put into building a data pipeline capable of efficient processing across tens of thousands of CPU cores. The pipeline aims to extract top-notch content from the web, including extensive filtering and deduplication techniques.
TII released the refined-web dataset, which has been proven to be highly effective. Models trained solely on this dataset can match or even surpass models trained on meticulously curated corpora, showcasing their exceptional quality and influence.
The Falcon model also possesses multilingual capabilities. It can understand English, German, Spanish, and French, with limited abilities in some European languages such as Dutch, Italian, Romanian, Portuguese, Czech, Polish, and Swedish. Additionally, Falcon-40B is the second true open-source model to be released after H2O.ai. However, evaluating these two models has become challenging as H2O.ai has not undergone benchmark testing with other models on this leaderboard.
Commercial applications
Although the code for LLaMA is available on GitHub, its weights have never been publicly released, limiting its commercial applications. Furthermore, all variants depend on the original LLAMA license, making them unsuitable for small-scale commercial use.
In contrast, Falcon uses a modified Apache license, which means these models can be fine-tuned and used for commercial purposes. Falcon becomes the first open-source large language model that surpasses research restrictions. Initially, the license stipulated a default royalty of 10% on attributed revenue exceeding one million dollars. However, it was later announced that Falcon has transitioned to the Apache 2.0 license, eliminating the need for royalty obligations.
Compared to GPT-3, Falcon has achieved significant performance improvements, utilizing only 75% of the training compute budget and requiring only one-fifth of the computation during inference. Additionally, the model leverages 75% of GPT-3's training compute, 40% of Chinchilla's compute, and 80% of PaLM-62B's compute, achieving efficient utilization of computational resources.
Open-source vs. closed-source
While GPT-4 is the most advanced LLM to date, it is closed-source, and OpenAI has not disclosed any details about the model's architecture, size, hardware, training compute, dataset construction, training methods, etc. This prevents researchers and developers from understanding the technical details and internal workings of the model.
In contrast, open-source AI promotes collaboration, transparency, and greater innovation in the field. Open-source models allow for greater collaboration and knowledge sharing, leading to faster progress and innovation, as seen since the launch of LLaMA. The availability of LLaMA enables researchers and developers to access powerful language models without investing in proprietary solutions or expensive cloud resources. Some experts criticize closed-source models for their lack of transparency and potential biases, making open-source models an alternative to closed-source models.
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.