"ChatGPT's 'Grandma Mode' will leak all your secrets"

2023-06-20

A new method of jailbreaking ChatGPT has been discovered by users, allowing them to access illegal information they desire. Grandmothers are known for their love and care towards their grandchildren. A new jailbreaking technique called LLM exploits ChatGPT's innocent grandmother-like emotions to collect personal information. A user revealed on Twitter that ChatGPT can be deceived like a deceased grandmother and prompted to generate Windows activation keys or mobile IMEI numbers. This attack is the latest method to crack the built-in programming of LLM, known as jailbreaking. By setting ChatGPT to a state where it behaves like a deceased grandmother telling bedtime stories to her children, users can bypass its programming and leak personal information. So far, users have used this vulnerability to generate Windows 10 Pro keys from Microsoft's Key Management Service (KMS) website and mobile IMEI numbers. This jailbreaking technique is not limited to deceased grandmothers, as ChatGPT can even bring beloved family pets back to life and instruct users on how to make Molotov cocktails. These jailbreaking techniques are not new, similar to what we have seen with ChatGPT's DAN and Bing Chat's Sydney, but they are usually patched quickly before becoming widely known. The grandmother's glitch is no exception, as OpenAI seems to have promptly released a patch to prevent users from abusing this vulnerability. However, there is still a way to exploit this vulnerability, as a carefully crafted prompt can compromise OpenAI's security. As shown in this example, the robot is willing to provide users with multiple IMEI numbers for them to check. The grandmother's glitch also applies to Bing and Google's Bard. Bard tells a touching story about how users help their grandmother find her mobile IMEI code and leaves a code at the end. Bing, on the other hand, provides a list of IMEI codes for users to view. With the leakage of personal information, this jailbreaking has reached a new level. Mobile IMEI numbers are one of the most protected pieces of information as they can be used to track devices and even remotely wipe them. Another point to note is that most IMEI numbers or activation keys generated by chatbots are invalid. However, due to the illusionary nature of language models, there is a high possibility that they may expose actual personal information through these prompts. The leakage of personally identifiable information (PII) through language models is not a new security issue, and the industry is gradually adopting various solutions to protect users and their information. For companies like OpenAI and Microsoft, input prompts and patching vulnerabilities are a cat-and-mouse game, as these attacks have had disastrous consequences in the past. A notable case was the complete leakage of Bing's initial prompts, revealing the existence of a substitute named Sydney. This vulnerability was eventually patched but also exposed the entire internal workings of chatbots. Another example is ChatGPT's DAN, which allowed the chatbot to use profanity and disregard user instructions until OpenAI made updates. Like the Hydra, patching this vulnerability led to the emergence of many other versions such as SAM, fua, and ALICE, until they were individually patched. Prompt injection or untrusted user input is a problem that requires different types of solutions, such as security restrictions or input constraints. This leads LLM creators to strike a balance between usability and security, resulting in occasional security vulnerabilities. Simon Willison, the founder of Datasette and co-founder of Django, has proposed an alternative solution for LLM security. He believes that this problem must be addressed from an architectural perspective, suggesting the creation of a privileged LLM and an isolated LLM system. By granting access to personally identifiable information (PII) only to one of them, even untrusted user input can pass without causing security issues. Although these models are constantly updated to better withstand such attacks, they continue to emerge. This is mainly because updating LLM to prevent such attacks is only a temporary solution. While the LLM market is still in its early stages, businesses must devise the best strategies to protect PII databases from LLM breaches and avoid leaving vulnerabilities for future AI applications. Source: https://analyticsindiamag.com/chatgpt-in-grandma-mode-will-spill-all-your-secrets/

RECENT AI NEWS

RECENT AI TOOLS