Galileo launches AI firewall Protect to defend against malicious input and output

Galileo Technologies recently released a new software tool called Protect, which promises to defend against harmful artificial intelligence inputs and outputs.

The company defines this product as a real-time illusion firewall, which is one of the top software tools in their latest lineup, aimed at helping businesses avoid inaccurate responses from AI models. Some of these tools are developed by AI startups like Galileo, while others come from large enterprises in other sectors of the technology market.

Atin Sanyal, co-founder and CTO of Galileo, said, "The rapid proliferation of artificial intelligence brings a series of new security and compliance challenges that businesses need to actively address. While protecting AI, companies also need to ensure that it does not affect the user experience."

Based in San Francisco, Galileo has received over $20 million in funding from investors such as Battery Ventures. Protect, their latest AI firewall, promises to filter malicious large-scale language inputs with millisecond-level latency. This means that businesses can successfully prevent network attacks without significantly slowing down the speed at which large language models answer user questions.

Many malicious prompts aim to induce large language models to perform actions they should not. These actions often involve the leakage of sensitive information. For example, hackers may attempt to construct a prompt that induces a large language model to leak records from its training dataset.

Filtering malicious prompts is just one method of preventing such network attacks. Another method is to detect whether a large language model has generated a response that may have been produced by malicious inputs and replace it. According to Galileo, their Protect tool also supports the latter application scenario.

Customers can set rules to specify the conditions that the tool should cover for the output of large language models. In addition to filtering prompt responses that contain sensitive data, developers can create rules to detect and replace illusory or inaccurate AI responses. Galileo states that the tool can also address more subtle issues, such as when the response style of a large language model does not align with a company's internal marketing guidelines.

This tool is integrated with two existing AI products from software manufacturers. The first is Observe, which alerts administrators when a large language model violates the output filtering rules set in Protect. The other product is called Evaluate, which helps AI teams troubleshoot the root causes of incorrect prompt responses.

Galileo is one of many startups helping businesses avoid inaccurate outputs from large language models.

Last month, Stardog Union launched Voicebox, an AI platform that promises to allow employees to query business data using natural language prompts while avoiding the risk of illusions. The company also introduced a local device that allows organizations to host Voicebox in their own data centers. Meanwhile, Gleen, based in Pleasanton, California, has built a dedicated AI data management tool to filter inaccurate LLM outputs.

Major market players have also joined this competition. A few weeks ago, Cloudflare Inc. launched Firewall for AI, a cybersecurity tool that prevents large language models from leaking sensitive data due to malicious prompt responses. Google LLC recently detailed its internally developed framework aimed at mitigating AI illusion problems.