OpenAI Releases Procgen Benchmark to Prevent AI Model Overfitting

2019-12-05

In the training of machine learning models, there is always a risk of overfitting (or being too close) to specific datasets. In fact, popular machine learning benchmarks, such as the Arcade Learning Environment, encourage overfitting and it is not impractical, simply because they do not prioritize generalization.This is why OpenAI has released the Procgen Benchmark today, which is a set of 16 procedurally generated environments (CoinRun, StarPilot, CaveFlyer, Dodgeball, FruitBot, Chaser, Miner, Jumper, Leaper, Maze, BigFish, Heist, Climber, Plunder, Ninja, and BossFight) to measure the speed at which models learn general skills. It is built on top of the initial CoinRun toolkit, which uses procedural generation to construct training and testing level sets.OpenAI wrote in their blog, "We wanted to have the best of both worlds: a benchmark with many different environments, each of which requires generalization at its core. To meet this need, we created the Procgen Benchmark, with a focus on experimental convenience, high diversity within environments, and high diversity across environments. CoinRun now becomes the inaugural environment for the Procgen Benchmark, contributing to its diversity."


According to OpenAI, the design of the Procgen environments provides a high degree of freedom (within basic design constraints), thus bringing meaningful generalization challenges to AI-driven agents. They have also calibrated them to ensure that benchmark agents make significant progress after training for 200 million time steps.In addition, the Procgen environments support two "well-calibrated" difficulty settings: easy and hard. (The former targets users with limited computational resources, as it requires approximately one-eighth of the resources for training.) And they have emulated the style of many Atari and Gym Retro games, staying consistent with precedents.


As the training set grows, the performance of AI models typically improves. The blog author explains, "We believe that the improvement in training performance comes from implicit curricula at different levels. If generalization is done across levels in the training set, then larger training sets can improve training effectiveness."


OpenAI writes: the huge gap between training and testing performance reveals a key hidden flaw in training environments that follow a fixed order of levels. OpenAI makes future work more complex, as it believes this will provide inspiration for more powerful and efficient AI models.OpenAI previously released Neural MMO, which is a "massively multi-agent" virtual training ground that allows agents to operate in RPG-like worlds, and Gym, which is a playground for reinforcement learning algorithms. Recently, it has provided SafetyGym, a set of tools for developing AI that adheres to safety constraints during training and is used to compare the "safety" of algorithms and the extent to which these algorithms avoid mistakes during the learning process.

RECENT AI NEWS

RECENT AI TOOLS