"Training Other AI Models with AI-generated Data"

2019-12-20

Generative Adversarial Networks (GANs) consist of a generator that generates samples and a discriminator that attempts to distinguish between generated and real samples. These two components of AI systems have countless applications, one of which is generating synthetic data. Recently, researchers at Uber utilized this concept in a paper titled "Learning to Accelerate Neural Architecture Search". The paper introduces a customized GAN called Generative Teaching Networks (GTNs), which can generate data or training environments for models to learn from before being tested on target tasks. According to the paper, compared to methods that only use real data, GTNs help accelerate search speed by a factor of 9. Furthermore, GTNs are more competitive compared to state-of-the-art architectures that achieve the highest performance with significantly less computational resources. As explained by the contributing author in a blog post, most model searches require a "substantial" amount of resources as they evaluate models by training them on datasets until their performance no longer improves. This process can involve repeating thousands or more model architectures within a cycle, making it computationally expensive and time-consuming. Some algorithms avoid this cost by training for short periods and using the results as an estimate of real performance. However, this training can be further accelerated by leveraging machine learning, and that's where GTNs come in. GTNs achieve success by creating unrealistic data that aids the learning process. It can combine information from various types of objects or focus training primarily on the most challenging examples, all while evaluating the model on real data. In experiments, the team reported that GTNs achieved an accuracy of 98.9% on popular open-source datasets in just 32 steps (approximately 0.5 seconds) of training. During this time, they consumed 4096 synthetic images, which is less than 10% of the MNIST training dataset images. When evaluated on another dataset, the model's learning speed was four times faster than actual data, even compared to optimized actual data learning algorithms, while maintaining the same performance level. Furthermore, performance on GTN data has been shown to be a predictor of real performance. Using only data generated by GTNs, the model achieved the same predictive capability as actual data in 128 steps, whereas it required 1200 steps on actual data. What do these steps and time mean? The answer is cost savings.

RECENT AI NEWS

RECENT AI TOOLS