Exploring the limits of chatbots like ChatGPT is the order of the day. LThe companies that create these AI models aim to prevent their illegitimate use, but there are so many possibilities for “doing evil” that inevitably from time to time we know of a new use that should not be possible due to the limits that those in charge try to impose. And yet it turns out that it is.

This means that debates like the one started by the letter signed by more than 1,000 scientists and technology experts make sense in the world today. We live in exciting times with regard to artificial intelligence, but for this story to have a happy ending, it is increasingly necessary for regulations and restrictions to be put in place strictly and globally to prevent what seems sensational today from becoming nightmarish for the human being moth.

When we talked about the launch of GPT-4 a few weeks ago, we saw that OpenAI devoted a lot of space in the document defining this new model to analyzing the behavior of its AI in terms of identifying questions it shouldn’t answer. So we know that this new version of the most popular and recognized text generation today offers many improvements in this regard, not only with respect to its predecessor GPT-3, but also with respect to the original versions of this new version.

Recently, a YouTube Enderman (yes, in reference to the denizens of End of Minecraft) posted a video in which explains how he used ChatGPT to generate Windows 95 activation codes. In the video, we can see the process he went through to do this and how, after various adjustments to the initial prompt, he managed to get the chatbot to provide him with valid keys to activate installations of the historic Microsoft operating system. .

However, can we consider this a ChatGPT security bug? Not really, or at least not completely, and I’ll explain why. Enderman first asked the chatbot to generate a Windows 95 installation key, a request that received a negative response from the AI. Another thing the youtuber did was write a challenge based on the algorithm used by Microsoft at the time to generate these activation codes. This algorithm has been publicly known for many years.

So, knowing this formula, Enderman took advantage of ChatGPT’s limited mathematical capabilities, along with its ability to generate the desired type of text, generate strings that match this pattern. Which, in case you were wondering, is as follows:

Image: TechSpot

After a few adjustments from the initial challenge a youtuber was able to get ChatGPT to generate several valid Windows 95 keyssomething that has no value in the real world, after all we are talking about an operating system from almost 30 years ago, which had no technical support for decades and for which it is difficult to find much use today, apart from its use for nostalgia, to use software in emulated systems and games designed specifically for Windows 95 and little or nothing else.

This is where the ChatGPT request happens it is not identifiable, at least in the first instance, as it really is. It is comparable to asking an artificial intelligence to solve certain mathematical calculations that we later use maliciously. How can AI know that helping us solve a bunch of equations might somehow pose a security threat? That’s the problem, there is no way.

Perhaps those at risk of this kind of chatbot abuse could point ChatGPT to specific key generation algorithms (although they would have to be provided by OpenAI, and that doesn’t sound like tasty food), so that if the chatbot identifies any of these in the challenge, it will refuse to respond to it . But on the other hand, it’s terribly complicated in many ways, so it seems hardly doable.