Irreversible Defects, Degradation, and Model Collapse Exist in Large Language Models (LLMs): A Time Bomb
A recent research paper titled "The Curse of Recursion: Training Generative Models to Forget" has discovered that using model-generated content during training leads to irreversible defects in the resulting model, with the tail of the original content distribution disappearing.
Researchers Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson refer to this effect as model collapse and demonstrate its universality in variational autoencoders, Gaussian mixture models, and LLMs.
They state, "We theoretically unveil the generality of this phenomenon across all existing generative models."
The researchers prove that addressing model collapse is crucial if we want to maintain the benefits of training on large-scale data scraped from the web.
They point out, "In fact, as the content generated by LLMs exists in the data collected from real human interactions with the system, its value will become increasingly important."
In reality, LLMs and generative AI may be disguises for "degenerative" AI.
Since the release of OpenAI's ChatGPT and Google's Bard, among other LLMs, it has been widely believed that these systems will continuously improve.
However, the discovery of inherent model collapse in LLMs challenges this notion and sparks discussions about the potential inevitable degradation of these systems.
Here is an overview of the potential mechanisms behind LLM collapse:
Expanded Training Data and Parameters
Current LLMs, including ChatGPT and other large language models, rely on publicly accessible internet data for training.
This data comes from individuals' daily consumption, writing, Twitter, comments, and review information, giving us insights into its sources.
There are two widely accepted approaches to improving the efficiency of LLM models.
The first is to increase the amount of data used for training, while the second is to increase the number of parameters considered by the model.
Parameters represent unique data points or features relevant to the topics the model learns.
Traditionally, models use various forms of human-generated data, including audio, video, images, and text.
This data corpus exhibits:
- Real semantics
- A wide range of events
- Diversity
LLMs capture subtle differences and nuances, enabling the model to develop a realistic data distribution and predict not only the most common categories but also the uncommon ones.
LLM Collapse: The Threat of Machine-Generated Data
Research indicates that machine-generated data, such as articles written by LLMs or images generated by artificial intelligence, poses a significant threat to the aforementioned "diversity." This problem is more complex than initially perceived and accumulates over time.
Researchers emphasize that this problem is particularly prevalent in models following a continual learning process and has significant impacts.
Unlike traditional machine learning, continual learning adjusts based on sequentially provided dynamic data.
Whether task-based or task-agnostic, the data distribution gradually changes without clear task boundaries.
Model Collapse and "Data Contamination"
Model collapse is a degradation process that affects the intergenerational performance of generative models. This is a newly discovered problem in LLMs.
When generated data contaminates the training set of subsequent models, it leads to erroneous understanding of reality.
Broadly speaking, data contamination refers to any factor that causes data to inaccurately reflect reality.
The research paper utilizes a manageable model that simulates the mathematical model of LLMs to demonstrate the severity and persistence of this problem in LLMs.
Maintaining Authenticity and Proper Data Usage
As suggested by the paper, solutions to this problem primarily involve maintaining authenticity through additional collaborative reviews and ensuring reasonable data distribution.
When training LLMs, proper usage of machine-generated data is also crucial.
Considering the significant cost of training LLMs from scratch, most organizations rely on pre-trained models as a starting point.
As LLMs are increasingly adopted in critical industries such as life sciences, supply chain management, and content, developers need to continuously improve the models while maintaining realism.
Source: https://aimagazine.com/articles/research-finds-chatgpt-headed-for-model-collapse