Microsoft Unveils New Security System to Filter Malicious AI Outputs

Generating AI systems like ChatGPT are reshaping the internet, but they don't always bring positive impacts. As an early investor in OpenAI, the creator of ChatGPT, Microsoft has fully embraced this technology and offers Copilot AI in its numerous products and cloud services to help businesses build their own large language models. However, the specter of AI "illusion" still haunts these systems with uncertainty. Nevertheless, Microsoft states that users of its Azure AI platform will soon have new tools to regulate AI behavior.

Despite investing a significant amount of time in testing and refining large language models, the developers behind these systems are sometimes as surprised as we are when they go off track. Even if a company believes it has purified a model to the point where it doesn't output gender bias, falsehoods, or violent content, someone can still find new ways to chat with the bot and produce inappropriate results. Microsoft refers to these as "prompt injection attacks," which sounds like a fancy name for finding new query methods.

Microsoft has detailed five new features of Azure AI Studio. Three of them are currently available in preview, while the other two will be released later. The current features include Prompt Shield, Risk and Safety Monitoring, and Security Assessment. Prompt Shield aims to prevent users or external documents (indirect attacks) from attempting to deceive the model into generating malicious query outputs. Risk and Safety Monitoring is a set of tools that can detect and mitigate harmful outputs almost in real-time, and it also helps developers visualize the content filter status in the model. Security Assessment scans the content and security of model outputs while generating adversarial test datasets to improve manual "red team" model testing.

Azure's AI platform will also be able to generate secure system message templates in the coming months, which will help developers guide the model to produce safer outputs. The final piece of the puzzle will be Groundedness Detection, which is related to illusions. This feature analyzes outputs to ensure they are not "confidently wrong" or lacking basic common sense.

Microsoft will automatically add these security features to the GPT-4 model, but the Azure platform offers various AI options. Some less frequent users of LLM may need to adjust these tools and manually attach them to these models. As the rise of generative AI continues, Microsoft's fate also rises. By focusing on security and reliability, Microsoft hopes to help avoid the embarrassing mistakes that have plagued this technology since that decisive demonstration.