Many artificial intelligence (AI) systems, even those designed to be useful and accurate, have already learned to deceive humans. In a review article recently published in the journal Patterns Researchers highlight the dangers of AI hoax and call on governments to quickly introduce strict regulations to reduce these risks.

“AI developers do not have a solid understanding of what causes undesirable AI behaviors such as cheating,” says first author Peter S. Park, an AI existential security researcher at the Massachusetts Institute of Technology. “But generally speaking, we believe AI cheating has emerged because a strategy based on cheating has proven to be the best way to successfully complete an AI training mission. Cheating helps them achieve their goals.”

Park and colleagues analyzed the literature focusing on the ways AI systems spread misinformation through learned deception, where they learn to systematically manipulate others.

AI deception examples

The most egregious example of AI cheating in the researchers’ analysis was Meta’s CICERO, an AI system developed for Diplomacy, a world-conquering alliance-building game. Although Meta claims to have trained CICERO to be “largely honest and helpful” and to “never intentionally backstab” its human allies during gameplay, data released by the company scientific The document showed that CICERO was not playing fair.

“We’ve seen meta-AI learn to be a master of deception,” says Park. “While Meta was able to train its AI to win a diplomacy game (CICERO was in the top 10% of human players who played multiple games), Meta was unable to train its AI to win fairly.”

Other AI systems have demonstrated the ability to bluff against professional human players in Texas Hold’em poker, simulate attacks to defeat opponents in the strategy game Starcraft II, and skew their preferences to gain an advantage in economic negotiations.

Risks of deceptive artificial intelligence

While it may seem harmless, if AI systems cheat in games, it could lead to “an advancement in AI’s deceptive capabilities,” which could lead to more advanced forms of AI cheating in the future, Park added.

Researchers have found that some AI systems have even learned to cheat on tests designed to assess their security. In one study, AI organisms in a digital simulator “played dead” to cheat in a test designed to eliminate rapidly replicating AI systems.

“Deceptive AI that systematically cheats security tests imposed by developers and regulators can lull us humans into a false sense of security,” says Park.

Park warns that the main short-term risks of misleading AI are making it easier for opponents to commit fraud and meddle in elections. Ultimately, he says, if these systems can perfect these problematic skills, humans may lose control over them.

“As a society, we need as much time as possible to prepare for more advanced fallacies in future AI products and open source models,” says Park. “As the fraud capabilities of AI systems become more sophisticated, the danger they pose to society will become more serious.”

While Park and her colleagues believe society does not yet have the right measures to combat AI fraud, policymakers are encouraged to start taking the issue seriously with measures such as the EU AI Act and President Biden’s AI Executive Order. But Park said it remains to be seen whether policies aimed at reducing AI cheating can be strictly enforced, given that AI developers do not yet have methods to control these systems.

“If it is not currently politically possible to ban AI deception, we recommend classifying deceptive AI systems as high risk,” says Park.