Nvidia's New AI Agent Based on GPT-4, Capable of Training Robots to Master Complex Skills

Nvidia Research announced today that it has developed a new artificial intelligence agent program called Eureka, supported by OpenAI's GPT-4, which can autonomously teach robots complex skills. In a blog post, the company stated that Eureka trained a robotic arm to rotate pen cores at a speed comparable to humans for the first time by autonomously writing reward algorithms. Eureka also taught robots nearly 30 tasks, including opening drawers and cabinets, juggling balls, and using scissors. "Reinforcement learning has achieved remarkable results in the past decade, but there are still many challenges, such as reward design, which remains an iterative process," said Anima Anandkumar, Senior Director of Artificial Intelligence Research at Nvidia and one of the authors of the Eureka paper, in the blog post. "Eureka is the first step in developing a new algorithm that integrates generative learning and reinforcement learning methods to solve difficult problems." Nvidia Research also released the Eureka AI algorithm library for experimentation with Nvidia Isaac Gym. Nvidia Isaac Gym is a physics simulation reference application for reinforcement learning research built on the OpenUSD framework. This work builds upon Nvidia's previous work in AI agents. The rise of multiple AI agents has sparked months of speculation, including Auto-GPT, BabyAGI, and AgentGPT in April of this year. This research builds upon previous work, including Voyager, a Minecraft-playing AI agent built using GPT-4. In an article this week about transforming chatbots into online agents, Jeff Clune, a computer science professor at the University of British Columbia (formerly a researcher at OpenAI), said, "This is a huge business opportunity with potential value in the trillions of dollars," but he also noted, "It has both tremendous upside and tremendous impact for society." Superior to expert-designed rewards In a new research paper titled "Eureka: Human-Level Reward Design via Large-Language-Model Programming," the authors state that Eureka leverages the impressive zero-shot generation, code writing, and context improvement capabilities of the latest LLMs like GPT-4 to evolve optimize reward code. They claim that the resulting rewards can be used to acquire complex skills through reinforcement learning. "In 29 open-source RL environments, including 10 different robot morphologies, Eureka's reward functions generated without any task-specific hints or pre-defined reward templates surpassed expert-designed rewards. Eureka outperformed human experts in 83% of the tasks, achieving an average specification improvement of 52%," they said. "Eureka is a unique combination of large language models and Nvidia's GPU-accelerated simulation technology," said Jim Fan, a senior research scientist at Nvidia and one of the contributors to the project, in the blog post. "We believe Eureka will enable dexterous control of robots and provide artists with a new way to generate physically realistic animations."