How Google PaLM 2 betters predecessor and rivals OpenAI’s GPT-4

On Wednesday, Google unveiled the second generation of its Pathways Language Model (PaLM), called PaLM 2. The new large language model (LLM) will power the latest version of the company’s ChatGPT-rivalling artificial intelligence (AI) chatbot, Bard, and Google has claimed to have significantly improved the capabilities of its latest AI model over its predecessor. The list of upgrades to PaLM is similar to the changes that OpenAI announced with the release of its latest LLM, Generative Pre-trained Transformer (GPT)-4, but with a few key differences.

What is Google PaLM 2?

In a blog post announcing the rollout, Zoubin Ghahramani, vice-president at Google’s AI research division DeepMind, said that PaLM 2 is a “state-of-the-art language model with improved multilingual, reasoning and coding capabilities.”

According to the post, while the first generation of PaLM was trained only on English language data, PaLM 2 is “more heavily trained on multilingual text” — Google claims that it has over 100 languages in the latest LLM. This will seemingly enable PaLM 2 to understand hidden meanings within poems, and even complex riddles and idioms. Other new capabilities include its ability to contextually understand images that are submitted to it in queries, process mathematical equations for logical reasoning, and write code in languages such as Python, JavaScript, Prolog and Fortran.

PaLM 2 is the successor to PaLM, which was first detailed by Google researchers in April last year. The LLM was claimed to have 540 billion parameters in its training dataset, and was first released in public domain when Google and Alphabet chief, Sundar Pichai, unveiled the company’s AI-powered chatbot platform, Bard, on February 7. In the original PaLM paper, Google said that smaller models based on PaLM would feature 8 to 62 billion data points, depending on their use cases.

Put simply, while Bard is equivalent to the popular AI chatbot ChatGPT — made by Microsoft-backed OpenAI — PaLM is the technology that powers the chatbot. This makes it equivalent to the GPT LLMs that OpenAI uses to power its chatbots.

How large and diverse in terms of languages is PaLM 2?

Google has not disclosed the size of the dataset that PaLM 2 has been trained on, but said in a 91-page research paper that the training data used in its new LLM is “significantly larger”. However, it confirmed that PaLM 2 will be used in different sizes, including smaller models, to test various use cases. These models will be called Gecko, Otter, Bison and Unicorn, with Gecko being the smallest. However, it did not specify exactly how small Gecko would be, with Ghahramani’s blog describing the model stating that “it can work on mobile devices and is fast enough for great interactive applications on-device, even when offline.”

Google also did not provide a breakup in terms of which language constituted what percentage within its 100-plus languages used in training it.

How does it stack up against GPT-4?

Real-world comparisons are likely to surface soon, with Google releasing Bard as an open-for-all chatbot that is now available in English across 180 countries.

The first two obvious similarities include the ability of both the LLMs to be multilingual, and their ability to process images as part of user query inputs. Interestingly, OpenAI also did not disclose how much bigger the dataset of GPT-4 is, in comparison to GPT-3.5 — a factor that Google emulated with PaLM 2’s release.

Both OpenAI and Google have now claimed that it is not necessarily a case of “larger, the better” with LLMs, which is shaping up to be a cross-industry approach as enterprises seek to find easily deployable and workable LLMs for their use cases.

GPT-4 also opened up a paid ChatGPT tier for OpenAI, although Google has refrained from introducing a paid usage model for Bard at the moment.