This new episode of the NVIDIA AI Decoded series focuses on the role of artificial intelligence get more realistic NPCsan advance that, in my opinion, is undoubtedly one of the most important that AI applied to video games will give us in the short to medium term.

Think for a moment about the behavior of NPCs in and in games how little these have evolved over the years. On the other hand, the improvements that the games have received in terms of graphic quality are huge and thanks to ray tracing we can enjoy highly realistic lighting effects, shadows, reflections and caustics.

This evolutionary disparity has led us to the situation where we have games like Cyberpunk 2077, where we can find scenes that border on photorealism and NPCs that generally they’re still the same dehumanized “robots” with one or two pre-recorded catchphrases. This completely destroys the immersion in games, especially the sandbox ones where we have the freedom to explore and interact.

NVIDIA ACE will level up NPCs

And finally, it breaks the huge contrast that exists between the graphical quality of current games and the low quality of NPCs in general. To achieve this, the green giant offers technology to developers NVIDIA ACE according to the microservices model. This technology will allow them to integrate generative artificial intelligence models to create digital avatars in games and apps and will be able to communicate and converse dynamically with players and users in real time.

No more pre-designed and repetitive interactions that also didn’t make sense in the most basic NPCs and less related to the main story of the game. Thanks to NVIDIA ACE, the humanization of NPCs is something real and makes a difference one of the biggest and most important innovations recent years in the world of video games. Finally, we are about to “settle” a large pending bill.

NPC creation process It’s more complicated than it seems, and these end up being deeply tied to both the game’s story and setting, as well as the twists and turns that occur based on the player’s decisions, if the game allows it. The narrative must be perfectly pre-designed and the dialogue must contain relevant contextual elements.

Artificial intelligence to create NPCs capable of expressing emotions

Thanks to NVIDIA ACE, developers have a set of sub-components that work together build better interactivity and responsiveness. In total, we have four AI models that will allow NPCs to listen, process, generate dialogs, and react to players.

The player’s voice comes through NVIDIA Riva, a technology capable of creating real-time and fully customizable AI conversation channels that can turn chatbots (conversational bots) into more realistic and expressive assistants, also using GPU-accelerated multilingual voice and translation microservices.

With function Automatic speech recognition (automatic speech recognition) from NVIDIA Riva is done processing what we say and achieving accurate real-time transcription into different languages. This transcription goes through LLM and uses Riva’s Neural Machine Translation feature to generate a natural language text response. Audio feedback can be generated using Riva’s text-to-speech feature.

NVIDIA Audio2Face is another key technology because it is responsible for create realistic facial expressions which can also be perfectly synchronized with dialogues in different languages. In this way, it is possible to create avatars and NPCs with a much more realistic and dynamic expression, both in real time and in pre-designed scenes.

A network powered by artificial intelligence it automatically performs the process of animating the face, eyes, mouth, tongue and head NPCs so that everything fits not only the conversation they are having, but also the emotion of the moment in which the conversation is taking place. All the steps we have described happen in real-time to ensure complete fluidity of dialogues between the player and the NPC.

Big names in the sector are already starting to work with this technology, so it’s only a matter of time before we see it implemented in commercial games. Ubisoft was one of the first to lend its support and we recently got to see it in action in the Covert Protocol tech demo. We will also tell you all the keys to it in this article that we shared with you a few days ago.