DeepMind Transfers Multi-Dimensional Dataset Stacking Skill from Simulated to Real States

2019-10-23

What is the simplest way to teach a robot arm to stack blocks? In a recent study, researchers from DeepMind have been pondering a subtle question: AI methods for problem-solving always require a large amount of data, which is difficult to obtain in robotics. Even if there is enough data to train a machine learning model, it is almost useless without manual annotation.


However, scientists at DeepMind have found a solution to this problem from the observation learning system of the MuJoCo simulation environment and transferred their learning achievements to the real world. In a paper describing their research, they stated that this method can stack multidimensional datasets with only five hours of robot data.Last week, OpenAI published a relatedpaper, which studied the method of transferring skills from simulated robots to real-world robots to solve the Rubik's multidimensional dataset task. However, unlike the research that uses complex robotic hands for object manipulation, the DeepMind team used a basic Sawyer gripper.


The team proposed a two-step adaptation process. First, a strategy is learned using a simulation environment to solve the multidimensional dataset stacking task through synthetic images and proprioceptive perception (perception of position and motion). Two agents (a state agent with access to the simulator state and a visual agent using raw pixel observations) are trained simultaneously, with the state agent providing data for reinforcement learning of the visual agent.


In the second step, unlabeled real image sequences help to represent the state in the real domain, providing a general goal applicable to both simulation and reality. The researchers said that it leverages unlabeled data collected by the simulated robot to mitigate the negative impact of the gap between the simulated robot and the real robot.In the experiment, the researchers reported that their "from simulation to reality" transfer learning method showed significant improvement compared to domain randomization and other self-supervised adaptation techniques. On average, it successfully stacked the multidimensional dataset by 62%, far better than the baseline success rate of 12%.


The co-authors of the paper wrote, "Our agents interact visually with the real world, making our approach applicable to a wide range of manipulation tasks. The multidimensional dataset stacking task highlights a general approach to long-distance manipulation tasks. Most importantly, it is able to better utilize available unlabeled real-world data, thereby improving stacking performance."

RECENT AI NEWS

RECENT AI TOOLS