Research Reveals Security Vulnerabilities in Large Language Models

ChatGPT and Bard, among other LLMs (large language models), have sparked a wave of enthusiasm this year. Companies have invested millions of dollars in developing these AI tools, with some leading AI chatbots even reaching valuations in the billions of dollars.

The application of these LLMs in AI chatbots is growing rapidly. They learn and respond to specific user requests (known as "prompts") by scraping information from the entire internet.

However, computer scientists from AI security startup Mindgard and Lancaster University in the UK have demonstrated that parts of these LLMs can be replicated in less than a week for as little as $50, and the obtained information can be used for targeted attacks.

The researchers warn that attackers exploiting these vulnerabilities could leak private confidential information, bypass security measures, provide incorrect answers, or launch further targeted attacks.

This research will be presented at the 2023 Conference on Applied Machine Learning for Information Security (CAMLIS) and detailed in a new paper. The researchers demonstrate the feasibility of low-cost replication of essential content from existing LLMs and provide evidence of vulnerability transfer between different models.

This type of attack is known as "model parasitism," where the LLM is engaged in a conversation and prompted with a series of targeted prompts to reveal its workings.

The research team focused on studying ChatGPT-3.5-Turbo and used the acquired knowledge to create their own scaled-down replica model capable of replicating key content from the LLM.

Subsequently, the researchers used this replica model as a testing platform to identify and exploit vulnerabilities in ChatGPT without detection. They achieved an 11% increase in success rate by leveraging the knowledge obtained from the model to attack the vulnerabilities in ChatGPT.

Dr. Peter Gallahan, CEO of Mindgard and lead researcher of the project, said, "The scientific phenomenon we discovered is deeply concerning. This is one of the earliest studies that empirically demonstrates the successful transfer of security vulnerabilities between closed-source models and open-source machine learning models. Given the industry's heavy reliance on publicly available machine learning models hosted on platforms like HuggingFace, this is highly alarming."

The researchers state that their study highlights the hidden weaknesses of these powerful digital AI technologies and the potential for shared vulnerabilities among models.

Companies across various industries are currently investing billions of dollars in creating their own LLMs for various tasks, such as intelligent assistants. Financial services and large enterprises are adopting these technologies, but the researchers emphasize that these vulnerabilities should be a significant concern for any company planning to build or use third-party LLMs.

Dr. Gallahan adds, "While LLM technology holds transformative potential, businesses and scientists must carefully consider the network risks associated with adopting and deploying LLMs and conduct thorough understanding and evaluation."