GPT-4 vs GPT-3.5: Five key differences between the two OpenAI language models

On March 14, OpenAI, the Microsoft-backed artificial intelligence (AI) research firm, introduced GPT-4 — the latest generation of its large language model (LLM). Standing for Generative Pre-trained Transformer, GPT-4 offers a host of improvements over GPT-3.5, the underlying language model that powered the November release of the popular chatbot, ChatGPT. While the number of the version is iterative, OpenAI, during its introductory presentation, offered a host of key use cases in which GPT-4 will improve in comparison to what GPT-3.5 could produce.

Now, as the tool is opened up for paying customers that subscribe to OpenAI’s ‘ChatGPT Plus’ premium usage model, as well as in the form of application programming interfaces (APIs) for developers to build GPT-4-based products on, we take a look at five key ways in which GPT-4 claims to have improved over GPT-3.5.

Multi-modal inputs

The first and foremost difference between GPT-3.5 and its newer iteration, GPT-4, is in the form of handling inputs. While GPT-3.5, through ChatGPT, could only take text inputs to generate results, GPT-4 will also allow users to submit images as part of their queries. During the GPT-4 presentation, OpenAI president and cofounder, Greg Brockman, showcased how GPT-4 could understand a sketch of a website’s wireframe (the basic framework of a site) and generate code for the entire website itself.

OpenAI also showcased a demo by adding an image, and asking the chatbot to explain what was funny about the image. While GPT-3.5 did not have this capability, GPT-4 can understand images, and offer users a text response based on the context of the image.

The capability is only limited to one company, US-based visual processing firm Be My Eyes, at the moment, but OpenAI said that it will enable access to the image processing capability of GPT-4 to more businesses in due course of time — without offering a timeline for it.

Multi-language support

While GPT3.5 was based only on English, GPT-4 has been trained in 26 languages across the world, including five Indian ones — Bangla, Urdu, Punjabi, Marathi and Telugu. While the Indian languages rank lower than the rest of the languages in terms of accuracy of results generated based on these, the presence of a larger number of languages in GPT-4’s data sets, according to experts, will enable greater localised language use cases through the latest LLM.

More data and bigger memory

While OpenAI did not specify the number of data points on which GPT-4 has been trained, it claims that the model is based on substantially more data points than what GPT-3.5 was based on. The latter used 175 billion data points as part of its training database. An industry expert told TechCircle that analysts expect GPT-4 to be based on nearly 3 trillion data points, which expands the amount of information that it ‘knows’, and has processed.

OpenAI itself says in its webpage that the new LLM has “incorporated more human feedback, including feedback submitted by ChatGPT users,” behind GPT-4.

The company also said in its developer presentation that GPT-4 has a larger memory, or the amount of information that it can summon and process to answer one user query. While this was 4,096 ‘tokens’ for GPT-3.5 (and thus, ChatGPT), the same has been increased to 32,768 tokens for GPT-4. This represents an 8x increase in the total memory that GPT-4 has, which means that while answering user queries, GPT-4 will have 8x the amount of information than what the GPT-3.5-powered ChatGPT could process, thus giving the new iteration more context and background.

For reference, according to OpenAI, one token is roughly equivalent to three-quarters of a word.

Greater reasoning and accuracy

The company said during the release of GPT-4 that the larger amounts of data and memory has made the new LLM better at reasoning, and with the ability to offer more accurate results. According to the company’s official page, GPT-4 “is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5,” by virtue of the additional data points used to train the model.

GPT-4 can now also take longer inputs, and OpenAI said that this can allow the new language model to produce longform written content as well.

OpenAI also said that it has focused particularly on responding to ‘malicious’ prompts, and that the new model will offer “fewer surprises” when it comes to answering user queries.

Better safety protocols and ‘steerability’

Finally, while many industry experts and analysts had raised concerns regarding the safety standards of ChatGPT based on GPT-3.5, GPT-4 will seemingly offer better safety protocols for a safer chatbot. OpenAI claimed that it has worked with 50 experts in AI safety and security while developing GPT-4, and has also used GPT-3.5’s training data to improve the safety of results delivered by GPT-4.

During the presentation, OpenAI also added that GPT-4 will allow users to ask the chatbot to adopt different personas, albeit with limitations that the company has built into the system. The factor enabling the chatbot into adopting new personalities is called ‘steerability’, and OpenAI said that it is this factor that led to GPT-3.5 producing malicious results — a factor that it said will be improved in GPT-4.