OpenAI Finally Cracks the Mathematical Code

2023-06-02

OpenAI announced their new "process supervision" training mode: improving mathematical proficiency and bringing more consistency. After a long period of poor performance in mathematics, OpenAI announced yesterday that they have introduced a new technique called "process supervision" to enhance mathematical reasoning abilities. This new approach involves rewarding each step of reasoning instead of rewarding the correct final answer as in "result supervision". It is said that process supervision can improve performance and achieve consistency by training models to generate "chains of reasoning" that are closer to human thinking. Reducing illusions OpenAI believes that with process supervision, illusions will be minimized to some extent. It states that each step in the process is closely supervised, resulting in better outcomes. Furthermore, OpenAI states that rewarding consistent processes will lead to more "interpretable reasoning" as models are encouraged to follow human-approved processes. In simple terms, interpretable reasoning focuses on creating transparent models and providing clear explanations for their outputs. Result supervision may reward inconsistent processes that are difficult to detect. In contrast, process supervision does not face such issues. The company also states that addressing the problem of illusions is a crucial step towards building a consistent general artificial intelligence. Improving math scores OpenAI states in its blog post that it uses the MATH test set to evaluate the reward models of process supervision and result supervision. Additionally, multiple solutions are generated for each question, and the highest-ranked solution in each reward model is selected. We observed that the reward model of process supervision not only outperforms the reward model of result supervision but also exhibits a widening performance gap when considering more solutions to problems, indicating the reliability of this model. By reducing illusions and making their models more consistent, OpenAI is striving to make their chatbots closer to perfection through this approach. However, the applicability of these results to domains other than mathematics remains unknown. OpenAI believes that if process supervision is also applied to other domains, we may obtain a superior and more consistent approach than result supervision. Source: https://analyticsindiamag.com/chatgpt-finally-gets-better-at-math/

RECENT AI NEWS

RECENT AI TOOLS