Why the Timing is Right for the "Real" GPT-4

We previously asked the question "What happened to Multimodal GPT-4?" Six months later, Google's Gemini seems to have forced OpenAI to seriously consider speeding up the release of GPT-4 with multimodal capabilities. According to reports, Google will soon launch Gemini, and OpenAI must be prepared.

OpenAI is currently integrating GPT-4 with multimodal capabilities, similar to Google's plans with Gemini. According to recent reports, this integrated model is expected to be named GPT-Vision. Gemini and GPT-Vision are expected to enter the market this fall and may compete with each other.

Although Sam Altman previously made it clear that people should not expect GPT-5 or GPT-4.5, according to an article by Information, OpenAI may release a more powerful multimodal model, codenamed Gobi, after GPT-Vision. Unlike GPT-4, Gobi is designed as multimodal from the beginning.

Many people are eagerly anticipating the possibility that OpenAI will unveil Multimodal GPT-4 at their inaugural developer conference. OpenAI Developer Day is scheduled for November 6th in San Francisco.

The reason OpenAI has not released multimodal capabilities may not be because they are unable to develop them. The creators of ChatGPT are working with a company called Be My Eyes to develop an application that describes images to blind users, helping them interpret their surroundings and interact with the world more independently.

In this collaboration, OpenAI realized that it might be too early to add multimodal capabilities to GPT-4 at this stage, as integrating images could raise privacy concerns. Additionally, there is a risk of misinterpreting facial features, such as gender or emotional state, which could lead to generating harmful or inappropriate responses.

Meanwhile, OpenAI is already prepared. Several months ago, there were reports that OpenAI is developing Dall E-3. Early samples leaked by user MattVidPro show that this model performs much better than other image generators, including the commonly regarded best, Midjourney.

Interestingly, in a recent interview, Google CEO Sundar Pichai, when asked about the advantages of Gemini compared to ChatGPT, replied, "Today, you have separate models for generating text and images. With Gemini, these models will be fused." This suggests that our biggest expectation for Gemini is that it can generate both text and images based on user prompts.

If OpenAI combines the capabilities of Dall E-3 and ChatGPT Plus, it would be even more superior compared to Gemini.

To surpass GPT-4, Gemini is being trained on YouTube videos, making it the first multimodal model trained not just on text (or GPT-4's text plus images) but also on videos. Additionally, Demis Hassabis recently claimed that engineers from DeepMind are using AlphaGo's techniques for Gemini.

On the other hand, Google's Bard has not made a strong impression and falls short in generating text compared to ChatGPT. Therefore, placing hope on Gemini to reverse Google's fate is a huge gamble.

OpenAI Can Take Risks

OpenAI's product release process is different from Google's. Google, being a long-established and reputable company with 4.3 billion users worldwide, carefully considers before launching any product. They must ensure that the product is complete and flawless, without any loopholes.

On the other hand, OpenAI has already released products, albeit imperfect ones, and relies on consumer feedback to make necessary changes.

Take GPT-4 as an example. When OpenAI initially launched it, they mentioned that it would be multimodal. However, that was not the case. Additionally, OpenAI openly acknowledges the limitations of GPT-4 and states that it is still not fully reliable, often producing inaccurate information and leading to reasoning errors.

Pichai expressed similar views in a recent interview, stating that the success of the ChatGPT product proves that LLM technology is well-suited for the market.

He said, "Thanks to OpenAI for launching ChatGPT, which demonstrates the alignment of the product with the market. People are ready to understand and use this technology."

It can be said with certainty that as Google and OpenAI strive to take the lead in the multimodal war, this fall will certainly become more interesting.