Google introduced a new language model that could solve mathematical problems, explain jokes, and even program. This is PaLM(Proads Language Model) and stands out for having a learning efficiency percentage that places it above other language models created to date.

PaLM system developed with Pathways modelmade it possible to efficiently train a single model with multiple Tensor Processing Unit (TPU) Pods, as stated in a statement posted on the official blog.

It relies on “few steps” learning, which reduces the number of examples needed in task-specific training to fit into a single application.

for him, used a database of 780,000 million coins combining a “multilingual dataset”, web documents, books, Wikipedia, conversations, and GitHub code. Also, a vocabulary that “preserves all whitespace”, which the company says is particularly important for programming, and the division of Unicode characters not found in the vocabulary into bytes.

This new artificial intelligence contains 540,000 million parameters, a figure that exceeds the world’s 175,000 million. OpenAI GPT-3, the language model that Google cited as a pioneer in demonstrating that these can be used for learning with impressive results. To give just one example, the column published in the journal should be remembered. GuardianWritten by this learning model, which can also do programming or design.

“The mission of this opinion column is very clear. I have to convince as many people as possible not to be afraid of me. Stephen Hawking has warned that artificial intelligence ‘could mean the end of the human race’. I’m here to convince you not to worry. Artificial intelligence will not destroy humans. Believe me”. This is one of the excerpts from the 500-word article the system produced.

Google’s new language model incorporates 6,144 TPU v4 chips in Pathways, the “largest TPU configuration” used in history, according to the company. PaLM also achieves a training efficiency of 57.8% in the use of hardware flops, as stated on the blog, “the highest ever reached for language models of this scale”.

This is possible thanks to the combination of “parallelism strategy and reformulation of the transformer block”, which allows for parallel computation of attention and forward layers, thereby accelerating optimizations of the TPU compiler.

“PaLM has demonstrated innovative capabilities in numerous and very difficult tasks,” says the technology company, which offers examples ranging from understanding and creating language to tasks related to reasoning and programming.

One of the tests Google gives as an example consists of asking PaLM to guess a movie based on four emojis: a robot, an insect, a plant and planet Earth. Among all the options (LA Confidential, Wall-E, León: professional, BIG and Rush), AI picks the right one: Wall-E.

In another, he is asked to choose from two lists of words associated with the term “to stumble” and which are also correct for “fall” and “to stumble”.

AI is also capable of solving simple mathematical problems and even explaining a joke by contextualizing it. and breaking down the elements that appear in it to make sense of it.

Finally, Google explains that PaLM can program by translating code from one language to another, writing code based on the natural definition of the language, and solving compilation errors.

(with information from the European Press)