"Generating AI enhances everyone's creativity. Large language models like ChatGPT can generate high-quality papers and articles. Generative models like Stable Diffusion and DALL-E can create excellent images.
However, what happens when the internet is flooded with AI-generated content? This content will eventually be collected and used to train the next iteration of generative models. According to researchers from the University of Oxford, the University of Cambridge, Imperial College London, and the University of Toronto, machine learning models trained on AI-generated content will have irreversible flaws that worsen over generations.
The only way to maintain the quality and integrity of future models is to ensure they are trained on human-generated content. But as large language models like ChatGPT and GPT-4 are capable of creating content on a massive scale, human-generated data may soon become a luxury that few can afford.
Model Collapse
In their paper, the researchers studied what happens when subsequent models are trained on text generated by models like GPT-4.
"What happens to GPT-{n} as n increases?" the researchers wrote. "We find that learning from data generated by other models leads to model collapse—a degenerative process where the model forgets the true underlying data distribution over time, even when the distribution does not change."
Machine learning models are statistical engines that attempt to learn the data distribution. This applies to all types of machine learning models, from image classifiers to text and image generation models.
The closer the model's parameters are to the underlying distribution, the more accurate it is in predicting real-world events.
However, even the most complex models are only approximations of the real world. Therefore, they tend to overestimate events with higher probabilities and underestimate events with lower probabilities, albeit by a small margin. These errors accumulate as the models recursively retrain themselves, leading to model collapse. Eventually, later models in the sequence deviate from the original distribution of the natural data used to train them.
Model collapse is related to catastrophic forgetting, which occurs in models that continuously train on new data. Catastrophic forgetting causes machine learning models to gradually forget the information they were trained on in their early stages. Model collapse does not erase previously learned data but causes the model to interpret it incorrectly.
Model collapse is also related to data poisoning, where malicious actors attempt to manipulate the behavior of models by intentionally modifying the training data. Model collapse can be seen as a form of data poisoning. However, it is not intentional contamination of training data by individuals but rather contamination caused by the models and the training process itself.
Model Collapse in Generative AI
In their research, the researchers simulated the effect of training generative models on their own data. They tested three models: Gaussian Mixture Models (GMM), Variational Autoencoders (VAE), and large language models (LLM).
The task of GMM was to separate two artificially generated Gaussian functions. The model was initially trained on a dataset generated by a fixed function. It was then used to generate new data and retrain the next model. After 50 generations, the distribution of the data completely changed. By the 2000th generation, it lost all variance.
Using VAE to generate handwritten digits, the initial model was trained on real data. The next generation was trained on data generated by the previous model. The images gradually became blurry, and by the 10th generation, they became difficult to interpret.
The researchers then tested their hypothesis on OPT-125m, a small version of Meta's open-source large language model. They evaluated a common scenario where a pre-trained model is fine-tuned with recent data generated by another fine-tuned pre-training model.
They tested two different approaches. In the first approach, only data generated by the large language model was used for fine-tuning. In the second approach, a small portion of the original human-generated data was also added to the training mix.
"These two training approaches both lead to a decrease in the performance of our models, but we find that it is possible for the models to learn (partially) basic tasks using the generated data," the researchers wrote.
However, their findings also indicated that over time, the samples generated by these models were more likely to be produced by the original model.
"Simultaneously, we find that the generated data has a longer tail, meaning that some data is never generated by the original model due to the accumulated errors from learning generational data," the researchers wrote.
"What Will Happen to Future Generations of ChatGPT?"
The digital age has caused various forms of data pollution. Search engine algorithms heavily influence how people write online content. Bad actors employ various techniques to ensure their content ranks high on search engine result pages. We have seen the same effect on social media content recommendation algorithms, where controversial headlines are used to attract users and promote their content.
However, while the previous issues can be mitigated by changing ranking algorithms, the impact of large language models is difficult to detect and counteract.
"Our evaluation suggests that training patterns like large language models have a 'first-mover advantage,'" the researchers wrote. This means that platforms and companies that have access to genuinely human-generated text will have an advantage in creating high-quality models. After that, the internet may be flooded with AI-generated content.
The researchers suggest taking measures to retain access to the original data over time. However, it is currently unclear how to track and filter content generated by large language models on a large scale. This may become the focus of a new wave of innovation and competition among tech companies in the coming months or years.
Source: https://bdtechtalks.com/2023/06/19/chatgpt-model-collapse/"
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.