Adept AI Open Sources Large Language Model Persimmon-8B

2023-09-11

The San Francisco-based artificial intelligence startup Adept recently announced the open-source release of Persimmon-8B, their new language model with 8.3 billion parameters. Persimmon-8B is currently the most powerful fully open-source AI model with fewer than 10 billion parameters.

Persimmon-8B boasts a range of features that set it apart from similar products. The model is designed to be flexible and is released under the Apache License, making it the most powerful fully licensed language model with fewer than 10 billion parameters.

One of the most notable aspects is its context size of 16K, which is four times larger than LLaMA2 and eight times larger than well-known models like GPT-3. This increased context size allows Persimmon-8B to understand and generate text more coherently and effectively, making it applicable in a wide range of use cases. Despite training data comprising only 37% of its closest competitor LLaMA2, Persimmon-8B's performance is comparable, indicating its efficiency.

The 8B size is highly suitable for most users without access to large-scale computing power. It can be fine-tuned on a single GPU, run at faster speeds on modern MacBooks, and potentially even adapt to mobile devices.

Persimmon-8B represents an early achievement in Adept's model expansion plan, which will support the company's future AI products. Adept aims to develop an AI agent to assist people in various computer tasks.

In addition to the model, Adept has also released custom high-speed inference code that is on par with optimized C++ implementations like NVIDIA's FasterTransformer, while retaining the flexibility of PyTorch. This innovation could significantly accelerate application development.

The announcement also addresses the complexity of evaluating language models and discusses the limitations of commonly used methods that focus on internal knowledge rather than interactive capabilities. Adept emphasizes their use of direct answer generation for evaluation, which is closer to how users interact with language models.

In preliminary tests, Persimmon-8B outperformed other models in popular AI benchmarks such as Winogrande, HellaSwag, and ARC. Adept notes that as the original model is released, no measures have been taken to control harmful outputs.

This release is part of Adept's strategic plan for the coming months. By providing the public with this powerful and flexible language model, Adept hopes to drive greater innovation in the field of artificial intelligence.

Given its features and capabilities, it is certain that this model will serve as a cornerstone for numerous groundbreaking applications and research. The code, weights, and documentation for Persimmon-8B are now available on Adept's GitHub.

RECENT AI NEWS

RECENT AI TOOLS