Cohere Launches a Cost-Effective AI Model Running on Just Two GPUs

Cohere Inc., an artificial intelligence startup, has unveiled Command A today. This latest large language model delivers high-performance capabilities for commercial applications with significantly reduced hardware requirements compared to competing AI models.

The company claims that this LLM surpasses leading proprietary and open models such as OpenAI's GPT-4o and DeepSeek-V3. Cohere adds that in private deployments, the LLM can operate on just two graphics processing units using Nvidia's A100 or H100, whereas competing models may require up to 32 units.

This scale difference is significant as clients needing on-premise deployment, particularly in finance and healthcare sectors, typically must house their AI models within firewalls. This means purchasing expensive AI accelerator hardware and having high-performance models capable of running within enterprise environments.

"In head-to-head comparisons across business, STEM, and coding tasks, Command A matches or outperforms its larger, slower competitors - while delivering higher throughput and greater efficiency," says Cohere. The company specifies that Command A can deliver tokens at speeds up to 156 tokens per second, which is 1.75 times faster than GPT-4o and 2.4 times faster than DeepSeek-V3.

Designed with commercial applications in mind, the model also features a larger context window of 256,000 tokens, twice the industry average including Cohere's own Command R+ model. This means the model can process vast amounts of documents at once, or up to a 600-page book.

"We're simply training our models to make you better at your job," said Nick Frosst, co-founder of Cohere. "It should feel like putting a mech suit on your brain. So we're training it to augment your abilities. It should be particularly good at that."

The company states it's focusing on developing functionalities within the model to support scalable operations of AI agents. Agentic AI has recently become a notable trend in the industry, aiming to create artificial intelligence systems that can analyze data, make decisions, and execute tasks with minimal or no human intervention. In practice, this requires substantial computing power and well-trained AI models capable of executing efficiently and accurately based on corporate information.

Cohere says Command A will integrate directly with its secure AI agent platform, North, which enables enterprise users to fully leverage their company data. The platform is designed to allow enterprise AI agents to use customer relationship management, resource planning software, and other tools to automate tasks.