Sophia: A Novel Approach to Training Large Language Models

2023-07-04

A team from Stanford University has developed Sophia, a new method for optimizing large-scale language model pre-training that is twice as fast as existing methods. ChatGPT and other applications that rely on large language models (LLMs) are gaining widespread use and attracting significant media attention. However, a few large tech companies dominate the field of large language models because pre-training these models is very expensive, with costs estimated to start from $10 million and potentially reach several tens or hundreds of times that amount. "Hong Liu, a computer science graduate student at Stanford University, said, 'Small organizations or academic groups find it difficult to use large language models.'" To change this situation, Liu and his colleagues started improving the current methods for optimizing large language models. The result is a method called Sophia, which cuts pre-training time in half. The details of this method were published on the arXiv preprint server. Optimization: To better optimize large language model pre-training, Liu and his colleagues, including Stanford University postdoctoral fellow Zhiyuan Li, Stanford University research engineer David Hall, computer science assistant professor Tengyu Ma, and associate professor Percy Liang, used two techniques. The first technique is called curvature estimation, which is not new, but the Stanford research team found a more efficient way to implement it. To understand their method, let's take a factory assembly line as an example. To be effective, a factory manager needs to optimize the number of steps required to transform raw materials into the final product and appropriately allocate the workload for each step on the production line. The same applies to pre-training large language models. These models have millions or even billions of parameters, and Liu compares them to factory workers striving towards the same goal. One characteristic of these parameters is their curvature, which Liu believes represents the maximum speed they can achieve towards the ultimate goal of pre-training large language models. In the factory analogy, curvature is similar to the workload of factory workers. If an optimization program can estimate curvature (workload), it can make large language model pre-training more efficient. The problem is that estimating curvature with existing methods is very difficult and expensive. "In fact, it is more expensive than working directly without curvature prediction," Liu said. That's why the state-of-the-art methods for optimizing large language model pre-training (such as Adam and its variants) have abandoned the curvature estimation step. Nevertheless, Liu and his colleagues noticed that previous methods using parameter curvature estimation could lead to inefficiency: previous researchers updated their curvature estimation at every optimization step. The Stanford research team wondered if they could improve the efficiency of this process by reducing the number of updates. To test this idea, the Stanford research team designed Sophia to estimate parameter curvature every 10 steps. "It turned out to be a huge success," Liu said. The team's second optimization technique, called pruning, addresses the issue of inaccurate curvature estimation. "If the estimation is wrong, it's like giving more work to those who are already struggling. That's worse than not estimating at all." Pruning prevents this situation by setting a threshold or maximum curvature estimation. "In our factory analogy, it's like setting workload limits for all employees," Liu said. Another analogy frequently used in optimization is the landscape of hills and valleys, where the goal is to end up in the lowest valley. Liu said that without pruning, it's possible to land in a saddle between two hills. "That's not what you want in optimization." Testing Sophia: Liu and his colleagues used Sophia to pre-train a relatively small large language model with the same size and configuration as OpenAI's GPT-2. Sophia combines curvature estimation and pruning, allowing for smooth optimization of large language model pre-training with half the number of steps and time required by Adam. "Sophia's adaptability sets it apart from Adam," Liu said. "Adam struggles with parameters that have heterogeneous curvature because it cannot predict them in advance." Liu said this is the first substantial improvement in language model pre-training compared to Adam in nine years. "This could mean a significant reduction in the cost of training real-world large models," he said, adding that as models scale up, Sophia's advantages will only increase. Next, Liu and his colleagues hope to use Sophia to develop an even larger large language model. He also wants to see Sophia applied to other fields of machine learning, such as computer vision models or multimodal models. "Transferring Sophia to a new field will take some time and resources, but since it is open-source, the community can certainly do it." Source: https://techxplore.com/news/2023-07-team-faster-cheaper-large-language.html

RECENT AI NEWS

RECENT AI TOOLS