Member-only story
PaLM 2 was announced by Google on 10th May 2023 as an answer to GPT-4. PaLM (the predecessor of PaLM 2) had 540 Billion parameters but Google has not disclosed the number of parameters in PaLM 2. The technical report on PaLM 2 simply states:
The largest model in the PaLM 2 family, PaLM 2-L, is significantly smaller than the largest PaLM model
The technical report also shows that in some cases PaLM 2 can outperform GPT-4 but others people have hinted that in many cases PaLM 2 falls behind GPT-4. In the context of this blogpost I am not evaluating performance in terms of the quality of the response. This is highly dependent on the particular case you are trying to use the LLMs for. I am just going to evaluate the response time performance of PaLM 2 vs GPT-3.5-turbo (which is faster than GPT-4). Maybe I will add the performance numbers of GPT-4 just to put things into perspective at a later date.
Currently, only the second largest version of PaLM 2 (labeled Bison) is generally available for public access on Google Cloud. So, I will evaluate that against GPT-3.5-turbo (0613) for the same input prompt under three different traffic workloads.
Prompt used
Translate the source text from English to Russian. \n Example 1: Input: Oktyabrskiy District (Kursk Region) 471583321 * Phone \n Translation: Октябрьский Район (Курская Область) 471583321 * Телефон \n Example 2: Input: It also has a large amount of furniture and more than 200…
