Robots Enhance Learning Speed through Eureka

Intelligent robots are reshaping our universe. At Robert Wood Johnson University Hospital in New Jersey, AI-assisted robots scan every inch of the building, searching for harmful bacteria and viruses, and use precise doses of germicidal ultraviolet light to disinfect them, bringing a new level of safety to doctors and patients.

In the field of agriculture, drone-powered robotic arms scan various types of fruits and vegetables and determine when they are perfectly ripe for picking.

The space intelligence system AI Flyways takes over the challenging and often nerve-wracking task of flight dispatchers who may need to make last-minute flight path changes due to sudden extreme weather, fuel shortages, mechanical issues, or other emergencies. It optimizes solutions, making them safer, time-saving, and cost-effective.

But let's not consider these achievements for now: Can robots perform pen spinning flawlessly?

A team of researchers at NVIDIA has developed a robot that can do just that. Although this task is impressive - some experts say it may take humans several months or even a year or longer to master challenging finger tricks such as Devil's Sonic, Backaround, Corkscrew, and Bust X2 - the focus of the NVIDIA project is that this pen spinning feat is taught to the robot through AI-generated instructions.

In a paper titled "Eureka: Human-Level Reward Design via Coding Large Language Models," the researchers describe an "evolutionary optimization of reward code" in which the robot learns complex fine motor actions through AI-generated instructions.

It promises to bring more efficient problem-solving, advanced physical manipulation, and increasingly intelligent machines to our future.

The team has developed an algorithm called Eureka, applied to GPT-4, to establish a reward system for LLM to learn advanced motor skills. These tasks are performed in the physics simulation application called Isaac Gym, developed by NVIDIA.

83% of the robots trained through Eureka outperformed human-designed instructions in the experiments. The pen spinning task is one of the 29 complex skills trained on the Eureka algorithm.

"The versatility and substantial performance improvement of Eureka suggest that the simple principle of combining large language models with evolutionary algorithms is a general and scalable reward design approach that may be applicable to difficult and open-ended search problems," said Anima Anandkumar, NVIDIA's Senior Director of AI Research and author of the Eureka paper.

Isaac Gym simulates physical activities in a three-dimensional environment. Massively parallel training generates a large number of possible manipulation solutions faster than humans or early computing systems. According to the researchers, this approach can speed up training by a factor of 1000.

Feedback from human manipulators can be incorporated into the training algorithm. The researchers state that it serves as a "powerful co-pilot" in particularly challenging tasks.

Eureka compiles progress statistics from each training session and adjusts the code to continuously improve results.

"Self-improving positive feedback loops may be just around the corner, allowing us to surpass human training data and capabilities," said Shital Shah, Chief Research Engineer at Microsoft Research.