Although large language models (LLMs) have made progress in reasoning capabilities recently, they still struggle with hierarchical multi-step reasoning tasks such as developing complex programs. In contrast, human programmers (typically) learn to decompose difficult tasks into manageable components that can work independently (modularity) and collaboratively (compositionality). Additionally, if there is an issue with the code generated by humans in a certain function, that part of the software should be rewritable without affecting other parts of the application. In contrast, people naively expect code LLMs to generate error-free code sequences.
This has prompted a recent study at Stanford University to explore how LLMs can be used for problem decomposition and compositional solution building. They propose Parsel, a compiler that takes a specification containing function descriptions written in natural language and constraints defining the expected behavior of the implemented functions. By using Parsel, programmers can write programs in natural language to solve competition-level programming problems, surpassing the previous state-of-the-art by over 75%.
A code LLM is given a description of a function and the signatures of the functions it depends on, and is tasked with generating the implementation of the function. When a constraint is added, the compiler searches for possible combinations of implementations until it finds a valid one.
Previous research has shown that code language models, unlike humans, cannot develop programs that can sequentially execute multiple small tasks. Parsel addresses this issue by separating the decomposition and implementation processes. While they intended to achieve natural language programming, they found that LLMs also performed well in Parcel encoding.
Decomposing an abstract plan to the extent that it can be automatically solved is a common pattern in human reasoning, which is reflected in the generation and implementation of Parsel; this compositional structure is also useful for language models. In this study, the team demonstrated that LLMs can create Parsel from a small number of instances, and their solutions outperformed state-of-the-art methods on competition-level problems from the APPS dataset. Excitingly, using Parsel, LLMs wrote plans for step-by-step robot planning from high-level tasks, achieving an accuracy increase of over two-thirds compared to the zero-shot planner baseline.
To evaluate the effectiveness of Parsel, experienced competition programmer Gabriel Poesia used it to solve a series of APPS challenges that typically appear in programming competitions. Within 6 hours, he found solutions for 5 out of 10 problems, including 3 that GPT-3 had previously failed on.
The researchers demonstrated that Parsel can be used for theorem proving and other activities that require algorithmic reasoning by formulating it as a general framework.
They plan to implement automatic unit test generation in the near future. One approach they mentioned is to search for special cases and see if there is a set of functions that agree on all existing tests and any new tests as well. This avoids the issue of exponential growth in implementation combinations, which could make automatic decomposition possible. They also intend to adjust the "confidence threshold" of the language model, as maintaining clear and concise descriptions is necessary for more important or critical parts of the program.
Source: https://www.marktechpost.com/2023/07/04/researchers-at-stanford-introduce-parsel-an-artificial-intelligence-ai-framework-that-enables-automatic-implementation-and-validation-of-complex-algorithms-with-code-large-language-models-llms/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.