What is PaLM 2: Google’s new large language model explained

Even though Google was one of the first adopters of generative AI, it has now found itself blindsided by the explosive growth of rivals like ChatGPT and Bing Chat. So in response, Google launched its Bard AI chatbot to mixed reception. More recently, the company also started experimenting with showing AI responses alongside search results. But what’s even more impressive is that Google has developed the machine learning models in-house. You may have heard about Google LaMDA as early as 2020, but the company has now switched to the more advanced PaLM 2 for its AI products.

So in this article, let’s take a closer look at PaLM 2 and how it differs from previous Google models. Later, we’ll also see how it stacks up against GPT-4, which is regarded as the best large language model available today.

PaLM stands for Pathways Language Model, which signals the use of Google’s Pathways AI architecture that helps train machine learning models to perform a variety of tasks. It’s a big part of Google’s vision to reach general artificial intelligence or AGI.

Google believes the Pathways architecture will pave the way for multi-modal AI beyond text. In a blog post, the company said,

Pathways could enable multimodal models that encompass vision, auditory, and language understanding simultaneously. So whether the model is processing the word “leopard,” the sound of someone saying “leopard,” or a video of a leopard running, the same response is activated internally: the concept of a leopard. The result is a model that’s more insightful and less prone to mistakes and biases.

While Google first spoke about PaLM in 2022, the company never launched a product utilizing it. However, with mounting pressure from the competition, Google has now upgraded its Bard chatbot to use PaLM 2. Unlike the LaMDA model it replaces, PaLM 2 was trained on over 100 languages and has even better domain-related knowledge in areas like coding. It also has much better logical reasoning and mathematical capabilities.

From a technical perspective, Google’s first-generation LaMDA machine learning model boasted 137 billion parameters and a training size of 1.56 trillion words. This time around, Google says that PaLM 2 uses a new technique known as compute-optimal scaling to produce better results with fewer parameters and a smaller training dataset.

Google hasn’t committed to bringing multi-modal abilities to its AI products like Bard just yet. However, its future Gemini language model will likely bring support for images, audio, and more.

PaLM 2’s capabilities have given Google Bard a pretty big boost in terms of capabilities. When it first launched in early 2023, the chatbot used the LaMDA language model under the hood. And with the move to PaLM 2 now, Google catapulted past the first-generation PaLM model.

Google has published a 92-page technical report detailing how PaLM 2 improves upon its predecessors, but let’s cut to the chase. Here’s a quick summary of the improvements over the company’s previous models:

1. PaLM 2 was trained on a mix of languages and its training data included many non-English text samples. It also performs better at foreign language tests than LaMDA and first-gen PaLM.
2. It sports better logical reasoning capabilities than its predecessors and even manages to match GPT-4 in the tests Google performed.
3. Google’s Bard chatbot can now generate and debug code. That wasn’t possible with the company’s previous language models.
4. PaLM 2 can translate between languages better than previous language models. In fact, it can also surpass Google Translate in certain languages like Portuguese and Chinese.
5. With the new Pathways architecture, PaLM 2 is more efficient to train and use. This makes it more economical and feasible to include in Google services like Gmail.
6. Google can fine-tune the PaLM 2 model for specific use cases. For example, Sec-PaLM is optimized to detect cybersecurity vulnerabilities, while Med-PaLM provides answers to medical queries.

How does PaLM 2 compare to OpenAI’s GPT LLMs?

OpenAI, the creator of ChatGPT, has so far managed to surpass other language models in terms of capabilities. But with the release of PaLM 2, can Google take the crown? According to the company’s technical report, it scores better on common-sense tests. However, synthetic benchmarks don’t necessarily translate to real-world performance.

So to find out which one comes out ahead, I asked Google Bard and Bing Chat a couple of riddles. Microsoft uses OpenAI’s GPT-4 as the foundation for Bing Chat and I used a bit of creative prompt engineering to keep it from searching the internet for a response. Here’s the result of the first riddle I asked.

As you can see in the above screenshots, both Google Bard and GPT-4 managed to solve the riddle correctly. In fact, PaLM 2 also gave me an alternative solution that Bing Chat did not offer up.

However, Google’s Bard chatbot refused to solve another riddle I threw at it. Bing Chat, meanwhile, gave in-depth detailed reasoning and the correct solution when asked the same riddle.

While these two examples don’t test PaLM 2’s full capabilities, they prove that Google has started to catch up with the competition but still has a fair bit of work left to do.

At Google’s I/O developer conference, we got a glimpse at the company’s plans for the future of PaLM 2 and its AI developments in general. The latest language model will power everything from a new “Help me Write” feature in Google Docs and Gmail to AI image generation in Slides. Broadly, these features fall under Google’s Duet AI for Workspace umbrella and we should see them roll out in the coming months.

Google has also developed a version of PaLM 2 to fit on smartphones. Codenamed Gecko, this condensed language model could someday make it possible to use generative AI completely offline on a typical Android smartphone.

Finally, software developers can now access PaLM 2 programmatically via APIs. This means we may see other companies rely on Google’s large language model, just like we’ve seen Duolingo and Khan Academy adopt GPT-4 for their respective AI-powered features.