AI21 Launches Maestro AI Planning and Orchestration System

AI21 Labs has unveiled Maestro, a software platform designed to enhance the output quality of large language models (LLMs) significantly.

Based in Israel, AI21 is an artificial intelligence startup that has raised $336 million from investors such as Nvidia and Google. The company offers a series of enterprise-focused LLMs called Jamba. These models can handle prompts with up to 256,000 tokens and support Retrieval-Augmented Generation (RAG), a machine learning technique enabling AI to analyze information not included in its training dataset.

Prior to deploying LLMs into production, organizations often take steps to mitigate risks associated with output quality issues. This process typically involves setting up automated workflows to detect errors in prompt responses. While these workflows can reduce the risk of hallucinations, they are challenging to create and maintain.

The newly launched Maestro platform by AI21 aims to address this challenge. Described as an AI orchestration and planning system, it reduces the effort required to minimize LLM output errors while streamlining various related tasks.

To use Maestro, users need to provide a prompt along with specific requirements for processing it. For example, one could set a cost limit for generating the LLM response. According to AI21, Maestro automatically applies these customer-defined criteria, reducing the need for manual coding.

When handling complex prompts, Maestro breaks down the task into smaller sub-steps. This approach has been shown to improve the quality of LLM outputs. Afterward, the platform runs simulations to determine the most efficient method for delivering accurate results.

Maestro evaluates multiple processing methods and selects the one most likely to yield correct LLM responses. If necessary, it can extend reasoning time computations—a technique that enhances accuracy by increasing the time and infrastructure allocated to a task.

After generating a response, Maestro checks it for errors and creates a detailed log of each step taken during the process. Staff can review this log to verify the accuracy of the LLM output.

In internal tests, AI21 applied Maestro to several popular LLMs. The results showed that the platform increased model accuracy by up to 50% in some cases. According to AI21, this means that reasoning-optimized LLMs like o3-mini can correctly answer over 95% of prompts when connected to Maestro.

The company envisions customers using Maestro for various applications, including improving document analysis and answering user queries. It also suits automating repetitive business tasks like data entry.

"The widespread adoption of AI by enterprises represents the next industrial revolution," said Ori Goshen, Co-CEO of AI21. "AI21's Maestro is the first step toward this future—moving beyond the unpredictability of existing solutions to deliver reliable AI at scale."

Maestro is currently in early access, with AI21 planning to make the platform widely available later this year.