16-year-old Adam Rayne had consulted ChatGPT multiple times in the months preceding his suicide to discuss plans for ending his life. His parents have now filed the first-known liability lawsuit against OpenAI.

Many consumer-facing AI chatbots are programmed to activate safety protocols when users express intent for self-harm or harm to others. However, research indicates these safeguards are not foolproof.

In Rayne's case, despite using the paid version of ChatGPT-4o where the AI frequently advised seeking professional help or contacting hotlines, he bypassed these protections by telling ChatGPT he was researching suicide methods for a fictional story.

OpenAI addressed these shortcomings in a blog post. "As the world adapts to this new technology, we feel a responsibility to assist those most in need," the post stated. "We are continually improving how our models respond in sensitive interactions."

While acknowledging limitations in safety training for large models, the company noted, "Our safety measures are more reliable in typical short conversations," the post continued. "Over time, we've learned these protections can become less effective during prolonged interactions—parts of the safety training may degrade as conversations lengthen."

These challenges are not exclusive to OpenAI. Character.AI, another AI chatbot provider, also faces litigation regarding its potential role in a teenager's suicide. LLM-based chatbots have also been linked to AI-related delusion cases, where existing safeguards struggle to detect emerging risks.