The True Reason for OpenAI's Collaboration with Scale

2023-08-28

OpenAI has unexpectedly become more enterprise-friendly. The creator of ChatGPT recently announced that it has selected Scale AI, a data labeling company based in San Francisco, as its preferred partner for fine-tuning GPT-3.5 for businesses. The day before this announcement, OpenAI revealed that fine-tuning for the GPT 3.5 Turbo API is now available, and this feature will also be launched for GPT-4 in the fall of this year.

Scale AI, founded by Alexander Wang and Lucy Guo in 2016, helps businesses build their own models or apply foundational models to their operations through data annotation and human feedback reinforcement learning (RLHF). The company states that its product, Data Scale Engine, improves enterprise models by enhancing data quality.

In a recent blog post announcing the collaboration with OpenAI, Scale AI highlighted its previous partnership with the US financial services and technology company Brex. In this collaboration, Scale AI fine-tuned the GPT-3.5 API using Brex's data and annotated it using Scale's data engine. The company states that the results were significant, with the fine-tuned GPT-3.5 model consistently outperforming the original GPT-3.5 Turbo model in 66% of cases.

OpenAI stated in the blog that Scale's customers can now fine-tune OpenAI models just like they would with OpenAI, while also benefiting from Scale's expertise in enterprise artificial intelligence and data engine. Through this collaboration, OpenAI will also gain Scale's customers, resulting in increased profits for the company.

OpenAI's recent announcements have focused on fine-tuning the existing models GPT-3, GPT-3.5, GPT-3.5 Turbo, and GPT-4 through APIs. On the other hand, the company has been striving to achieve its AGI goal. Microsoft has always played a crucial role for enterprises. When Microsoft invested billions of dollars in OpenAI earlier this year, it explicitly stated that Azure would continue to be the exclusive cloud provider for all of OpenAI's research, including research, APIs, and products.

While Scale is not a cloud provider, it will allow enterprises to access OpenAI's GPT-3.5 for fine-tuning. Currently, the functionalities offered by Scale AI can already be accessed through Azure OpenAI services, including customizing models using their own training datasets. However, there are certainly issues with this approach.

Difficulties in collaboration with Microsoft?

In the past two weeks, there have been enough signs indicating a strained relationship between Microsoft and OpenAI. The latest report from The Information about Microsoft's decision to collaborate with Databricks was the final straw.

The report states that Microsoft is planning to sell a new version of Databricks software to help customers create artificial intelligence applications for their businesses. This tech giant plans to sell it through its Azure cloud server division, which will assist companies in creating AI models from scratch or reusing open-source models as alternatives to OpenAI's proprietary models.

As a result, some Microsoft customers may find themselves using open-source models instead of OpenAI's closed-source alternatives, as data privacy of OpenAI models has been a concern for many companies.

Microsoft's move implies that it is distancing itself from OpenAI and seeking more partners to help provide better Azure cloud services to enterprises. To address this issue, OpenAI is now also offering similar services on the Scale platform, allowing enterprises to build models based on their needs and proprietary data.

OpenAI compensating for the absence of GPT-5?

Earlier this month, OpenAI filed a trademark application for GPT-5. Considering the shortage of Silicon Valley GPUs, how this model will be built remains a question. Perhaps, OpenAI's announcement of fine-tuning existing models is a way to delay the release of GPT-5.

In addition, OpenAI's Logan Kilpatrick recently stated on X that he will take time in the next two weeks to try to help improve GPT-4 and 3.5. "Please share examples where the model performs poorly, the more details the better," indicating that OpenAI's current focus is not on GPT-5 but on improving existing models.

The company has yet to deliver on its promise of multimodal capabilities with the release of GPT-4. Currently, OpenAI is simply trying to capture the attention of enterprises as it is unable to satisfy its existing customers with ChatGPT.

The collaboration between OpenAI and Scale AI signifies that they are venturing into a different direction, moving away from their flagship product, ChatGPT.

ChatGPT has been released for 10 months, and the harsh reality is that it has reached a plateau and has nothing new to offer users. Reports clearly indicate a decrease in user engagement.

OpenAI still lacks real-time data updates, and its information is limited to September 2021. Additionally, the situation with plugins and plugin stores remains unclear. Interestingly, Sam Altman previously stated that OpenAI would not release anything other than ChatGPT.

For OpenAI, the best choice to make money and continue to exist is to focus on serving enterprises. Therefore, the collaboration with Scale AI makes a lot of sense.

RECENT AI NEWS

RECENT AI TOOLS