Robot "Chef" Learns to Reproduce Recipes by Watching Food Videos

2023-06-06

Researchers have trained a robot "chef" that can watch and learn from cooking videos to create dishes on its own. Researchers from the University of Cambridge have written a cookbook for their robot chef, containing eight simple salad recipes. After watching a video demonstration of one of the recipes, the robot is able to recognize and recreate the dish. Furthermore, these videos have helped the robot gradually expand its repertoire. By the end of the experiment, the robot came up with its own ninth recipe. Their research, published in the IEEE Access journal, demonstrates how video content can serve as a valuable and rich data source for automated food production, potentially making the deployment of robot chefs easier and more cost-effective. For decades, robot chefs have been a staple in science fiction novels, but in reality, cooking has been a challenging task for robots. Some commercial companies have developed prototypes of robot chefs, although these robots are not yet commercialized and are far behind human chefs in terms of skills. Human chefs can learn new recipes by observing, whether it's watching others cook or watching videos on YouTube. However, programming robots to make a variety of dishes is both expensive and time-consuming. "We wanted to see if we could train a robot chef to learn in a progressive way, by recognizing ingredients and their combinations in dishes, similar to humans," said Grzegorz Sochacki, the first author of the paper and a doctoral candidate in the Department of Engineering at the University of Cambridge. Sochacki, from the Biorobotics Laboratory of Professor Fumio Hara, and his colleagues designed eight simple salad recipes and filmed the process of making the salads. They then used an open-source neural network to train their robot chef. The neural network was programmed to recognize a range of different objects, including the fruits and vegetables used in the eight salad recipes (broccoli, carrot, apple, banana, and orange). Using computer vision techniques, the robot analyzed each frame of the video and was able to identify different objects and features, such as knives and ingredients, as well as the demonstrator's arms, hands, and face. The recipes and videos were converted into vectors, and the robot performed mathematical operations on the vectors to determine the similarity between the demonstration and the vectors. By correctly identifying the ingredients and the actions of the human chef, the robot could determine which recipe was being made. The robot could infer that if the human demonstrator held a knife in one hand and a carrot in the other, the carrot would be chopped. In the 16 videos it watched, the robot identified the correct recipe 93% of the time, even though it only detected 83% of the human chef's actions. The robot was also able to detect subtle variations in the recipes, such as making a double portion or a normal human error, indicating a change rather than a new recipe. The robot correctly identified and added the demonstration of the ninth new salad to its recipe and made it. "It's amazing that the robot can detect such subtle differences," said Sochacki. "These recipes are not complex, they are essentially chopped fruits and vegetables, but it effectively recognizes that, for example, two chopped apples and two chopped carrots are the same recipe as three chopped apples and three chopped carrots." The videos used to train the robot chef are not like the food videos made by some social media influencers, which are filled with fast cuts and visual effects, and quickly switch between the person preparing the food and the dish they are preparing. For example, if the human demonstrator wraps the carrot with their hand, it would be difficult for the robot to recognize the carrot. To make the carrot recognizable to the robot, the human demonstrator must lift the carrot so that the robot can see the whole carrot. "Our robots are not interested in the trendy food videos on social media, they are too difficult to understand," said Sochacki. "But as these robot chefs become better and faster at recognizing ingredients in food videos, they may be able to learn a variety of recipes from websites like YouTube." Source: https://techxplore.com/news/2023-06-robot-chef-recreate-recipes-food.html

RECENT AI NEWS

RECENT AI TOOLS