Salesforce releases new LLM model trained on 8K input sequence length

2023-06-30

US SaaS giant Salesforce recently launched XGen-7B, a series of 7B large language models (LLMs) trained on 8K input sequence length. These models are released under the Apache 2.0 license. In standard NLP benchmark tests, XGen has achieved comparable or better results compared to other open-source LLMs such as Falcon, LLaMA, Redpajama, and OpenLLaMA. Traditionally, models like the ones mentioned above are trained on token sequences with a maximum length of 2K, which is a key limitation for modeling long sequences. Salesforce stated in a blog post, "Given this, we trained a series of 7B LLMs called XGen, using standard dense attention mechanisms capable of handling sequences up to 8K in length, with a training data size of 1.5T tokens. We also fine-tuned the XGen model with publicly available guidance data, creating a fine-tuned version (XGen-7B-inst)." Recently, Salesforce also announced the launch of Salesforce Starter for Micro, Small, and Medium Enterprises (MSMEs) in India. Starter is an easy-to-use CRM suite that includes sales, service, and email marketing tools to help businesses get started quickly, providing the necessary tools to enhance customer experience, reduce costs, and drive revenue growth. Earlier this month, Salesforce Ventures announced an expansion of its generative AI fund, increasing the fund from $250 million to $500 million to continue supporting the AI startup ecosystem and promote responsible generative AI development. Source: https://analyticsindiamag.com/salesforce-introduces-new-llms-trained-on-8k-input-sequence-length/

RECENT AI NEWS

RECENT AI TOOLS