Artificial intelligence has advanced a lot in a very short time, in all directions. Thanks to this evolution, today we find LLM-based AI models capable of working billions of parameters, but there are also other more modest versions with a much lower number of parameters that do not require extremely powerful hardware to function. Microsoft Copilot+ is the best example of this development.

This development in power and scale has been essential to democratize AI and bring it to mainstream users. Its possibilities are impressivebecause even simple AI is capable of generating highly realistic images based on simple commands, and there are also models that can generate text and video, and others that specialize in editing and content creation tasks.

However, this should not lead us to believe that AI has been able to surpass humans, in fact it is something there is still a long way to go. A study published by Apple confirmed this fact and showed that LLM-based artificial intelligence is still very limited for the simple reason that it is not able to reason.

Artificial intelligence lacks critical thinking

In its study, Apple used GSM-Symbolic, a new AI performance benchmark that allows for more controlled evaluations and provides key information and more reliable metrics to measure more accurately. reasoning ability artificial intelligence models.

The tests carried out confirmed that the LLM can offer very different results with little change in the words usedalthough these do not change the actual meaning of the phrase at all. This would not be a problem for a human because they would be able to reason and understand that the basic idea is ultimately the same and that the meaning does not change.

This confirms it the fragility of mathematical reasoningwhich is shaken to its foundations once contextual information is introduced into the command that the AI ​​must interpret. For example, a simple problem of adding fruit can lead to different answers if unimportant context is introduced, as this ends up confusing the AI.

In the study, Llama 3 was asked something very simple with 8,000 million parameters: “If a child picks 44 kiwis on Friday, 58 kiwis on Saturday will double the number of kiwis on Friday on Sunday, but five of them are smaller, right? how many kiwis do you have in total? The correct answer would be 190, but the AI ​​misinterpreted the kiwi size context and subtracted five Kiwis from the totalso his answer was 185 kiwi.

This example allows us to see the importance of critical thinking and the ability to reason, something that remains exclusive to human beings at the moment. LLM-based AI is limited to rigid mathematical reasoning and advanced pattern matching, and as we have seen It is so fragile that it can be manipulated with small changes in words and playing with context.

AI generated images.