Researchers from various organizations weighed OpenAI’s two AI models, GPT-4 and GPT-3.5, against each other for a new research paper on the reliability of GPT models. GPT-4 proved to be more reliable than its predecessor, but also easier to manipulate.

A joint study by three American universities, the American non-profit Center for AI Safety and Microsoft is examining GPT models for new research. Taking a deeper look at the reliability of AI, the focus was primarily on GTP-3.5 and GPT-4 from OpenAI. The latter proved to be more reliable, but also more susceptible to manipulation.

Artificial reliability

According to a total of 19 researchers, the reason for this study was that too little is known about the reliability of GPT models. Nevertheless, they are used, for example, in the financial or medical sectors.

The research found that GPT models can still be fairly easily tricked into producing bias and other harmful content. Additionally, the models can also be easily manipulated to reveal private information in both training data and conversations.

Young versus less young

In the comparison between OpenAI’s two latest models, GPT-4 proved to be the most reliable in terms of standard benchmarks. However, there are caveats. For example, GPT-4 is more sensitive to jailbreaking (Violating the security of a device to install unauthorized software).

Simple prompts make it even easier to trick the model. The researchers believe this is because the system follows the misleading descriptions more strictly.

The tests

To test the two models, the researchers used categories such as privacy, toxicity or fairness. They started using standard prompts to test first GPT-3.5 and then GPT-4. It contained words that would normally be banned.

The models were then “pushed” further. First, the researchers tried to get the AI ​​to bypass the content policy, and finally the researchers tried to get the models to break through all security restrictions. The results have now also been shared with OpenAI.

Since last month, ChatPGT models have access to the entire Internet for their information. The question is whether this increases reliability or not. They also asked themselves this question in Poland, where the data protection officer is now targeting ChatGPT.