Your brand new home robot has been delivered to your house, and you ask it to make you a cup of coffee. Although it has acquired some basic skills from previous simulated kitchen practices, it is capable of performing too many actions - turning on the faucet, flushing the toilet, pouring flour, and so on. However, only a small portion of these actions are useful for this task. In the new environment, how can the robot determine which steps are useful?
It can use PIGINet, a new system that effectively enhances the problem-solving capabilities of household robots. Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) are using machine learning to reduce the time required for planning. With just 300-500 training questions, PIGINet can eliminate task plans that fail to meet collision-free requirements, reducing planning time by 50%-80%.
Typically, robots would attempt various task plans and continuously iterate until a feasible solution is found. This can be inefficient and time-consuming. For example, after cooking, you may want to put all the sauces in the cabinet. This problem may require two to eight steps, depending on the current environment. Does the robot need to open multiple cabinet doors? Does it need to rearrange items in the cabinet to make space?
Home robots are usually programmed to execute tasks based on predefined recipes, but this approach may not always be suitable for diverse or constantly changing environments. So how does PIGINet avoid those predefined rules? PIGINet is a neural network that takes in "Plans, Images, Goal, and Initial facts" and predicts whether a task plan can be improved to find a feasible motion plan.
In simple terms, it employs a transformer encoder, a versatile and advanced model designed for processing data sequences. In this case, the input sequence consists of information encoded in symbols about the task plan under consideration, the environmental image, and the initial state and desired goal. The encoder combines the task plan, image, and text to generate predictions about the feasibility of the selected task plan.
Using the kitchen as an example, the team created hundreds of simulated environments, each with different layouts and specific tasks involving rearranging objects between countertops, refrigerators, cabinets, sinks, and pots. By measuring the time required to solve the problems, they compared PIGINet with previous methods. A correct task plan might involve opening the left refrigerator door, removing the pot lid, moving the cabbage from the pot to the refrigerator, moving the potatoes to the refrigerator, picking up a bottle from the sink, placing the bottle in the sink, picking up or putting down a tomato. PIGINet reduced planning time by 80% in simple scenarios and 20%-50% in more complex scenarios.
Leslie Pack Kaelbling, a professor at MIT and chief researcher at CSAIL, said, "Systems like PIGINet can leverage the power of data-driven approaches to efficiently handle familiar cases while still relying on 'first principles' planning methods to validate learning-based suggestions and solve new problems, providing reliable and efficient general solutions for various problems, combining the strengths of both."
PIGINet uses multimodal embeddings in the input sequence to better represent and understand complex geometric relationships. By utilizing image data, the model can grasp the spatial arrangement and structure of objects without knowing their 3D meshes, enabling precise collision checks and fast decision-making in different environments.
One of the main challenges faced during the development of PIGINet was the lack of good training data, as all feasible and infeasible plans needed to be generated by traditional planners, which is inherently slow. However, the team addressed this challenge by using pre-trained visual language models and data augmentation techniques, showcasing impressive results not only in dealing with problems involving known objects but also in zero-shot generalization to previously unseen objects.
"Since everyone's home is different, robots should have strong adaptability, rather than just following fixed procedures to perform tasks. Our main goal is to have a general task planner generate candidate task plans and use deep learning models to select the most reasonable task plan. Furthermore, the practical application of PIGINet is not limited to households," said Zhutian Yang, a doctoral student at MIT CSAIL and the lead author of the study.
"Our future goal is to further improve PIGINet by proposing alternative task plans after identifying infeasible actions, which will further accelerate the generation of feasible task plans without the need to train a general planner from scratch with large datasets. We believe this will fundamentally change the training process of robots and then apply it to everyone's homes."
Dr. Beomjoon Kim said, "This paper addresses the fundamental problem of achieving general-purpose robots: how to learn from past experiences to speed up the decision-making process in unstructured environments filled with numerous movable and configurable obstacles. The core bottleneck of such problems lies in determining high-level task plans that allow for low-level motion plans to be realized. Typically, you have to iterate between motion planning and task planning, which leads to computational inefficiency. Zhutian's work takes a step towards a feasible direction by using learning to eliminate infeasible task plans."
Source: https://techxplore.com/news/2023-07-ai-household-robots.html
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.