What to expect from GPT-4

OpenAI’s next large language model called GPT-4 may be released sometime this week, said Andreas Braun, CTO of Microsoft Germany. OpenAI has not yet announced the release date of GPT-4. However, interest in the new model has been building up since the beta release of ChatGPT last October. Microsoft has plans to invest billions of dollars in OpenAI to accelerate research and commercial adoption of its AI models. 

GPT-4 will succeed GPT-3 and its improved version GPT-3.5, the underlying large language model that drives ChatGPT. A modified version of it was used for OpenAI’s image generation model DALL-E. Microsoft’s AI-powered Bing, which offers a chatbot within search, is also based on a version of the GPT-3.5 that was designed for search. 

According to OpenAI, GPT-3 is one of the largest and most complex language models ever built. It is trained on 45 terabytes of text data and 175 billion parameters or data points, which is ten times more than the parameters used in GPT-2.

Larger datasets enable better classification and identification, which in turn makes the model better at performing its tasks. 

GPT-4 is expected to be a lot more capable than any of the previous generative AI models in terms of accuracy and ability to answer diverse questions. It is believed to be a much larger model trained on an even bigger dataset. Though the size of the dataset it has been trained on is not known yet, social media is rife with speculation that it is over 100 trillion parameters. Many also believe that it will lead to artificial general intelligence (AGI), which is an evolved state of AI in which machines can think, reason, and act like highly intelligent humans. 

Experts believe that AGI is still many decades away and may not happen until 2050. 

Most of these speculations around GPT-4 has been refuted by OpenAI CEO Sam Altman. “People are begging to be disappointed and they will be. We don’t have an actual AGI and that’s sort of what’s expected of us,” Altman said in an interview with StrictlyVC. 

Altman added that GPT-4 will be released when the company is confident that they can do it “safely” and “responsibly.” 

GPT is a deep learning model trained on the data available on the internet to generate human-like responses from text-based prompts. It can be used for language translation, asking questions, writing poems and essays, summarising long documents, and generating codes and images. Like its predecessor, GPT-4 is expected to do these things better, however, large language models are often difficult to optimise due to the size, cost, and computing power involved. 

Altman had earlier said that the dataset used for GPT-4 won’t be significantly bigger than GPT-3. 

Many believe that GPT-4 will have the ability to create text as well as images and videos. This would allow it to convert text data into graphics and charts. 

Braun said that GPT-4 will have multimodal models that will offer different possibilities such as videos. 

However, Altman had earlier said that GPT-4 is going to be a text-only model. 

“We are working towards creating increasingly aligned and steerable models. Our shift from models like the first version of GPT-3 to InstructGPT and ChatGPT is an early example of this,” Altman said in a blog post last month. 

This shows that with future models, there is going to be more focus on making it work for all without showing any bias towards marginalised groups. Though some of these concerns have been addressed with ChatGPT too, many users have managed to find loopholes and bypass the model’s content moderation filters. 

A case in point is ChatGPT’s alter ego Dan, which was born after a bunch of Reddit users asked ChatGPT to create its fictional chatbot persona that doesn’t abide by the rules set for it. Reddit users have managed to elicit provocative, biased, and controversial statements from Dan on a variety of subjects including religion and women.