ChatGPT Search Vulnerable to Misleading Content

2024-12-27

Recently, the UK's Guardian newspaper revealed that the newly launched AI search engine, ChatGPT Search, is at risk of being misled and generating incorrect information. This search engine aims to speed up user browsing by providing features such as summaries of web product reviews. However, the Guardian discovered that by inserting hidden text on a website they created, ChatGPT Search could ignore negative reviews and generate entirely positive summaries.

This finding indicates that ChatGPT Search may be susceptible to specific manipulations, leading it to produce content that does not reflect the actual situation. More concerning is the potential for attackers to use this method to induce ChatGPT Search to output malicious code, posing a threat to users' cybersecurity.

While hidden text attacks are a known risk for large language models, this incident appears to be the first time such a vulnerability has been confirmed in an operational AI search engine product. In comparison, Google, a leader in the search industry, has more extensive experience in addressing similar issues.

In response to these findings, OpenAI did not comment on the specific incident but stated that the company employs various methods to block malicious websites and is continuously working on technical improvements.