NVIDIA’s relationship with artificial intelligence is more than evident, although one of its various aspects is less well known than the others. First of all, of course, we have to talk about DLSS, its intelligent upscaling technology based on Deep Learning, thanks to which the same system can give us better image quality without having to update the PC components. DLSS 2 available for all GeForce RTX graphics cards and DLSS 3 exclusive to the RTX 40 represent a before and after of what artificial intelligence can do with the graphics section of games.

On the other hand, NVIDIA graphics cards are the preferred choice for assembling systems, workstations, and servers that will be used for any AI-related task. And it is artificial intelligence that requires a large computing capacity with a floating point, i.e. something in which graphics cards offer processors an infinitely higher performance. This has always been the case, but with the proliferation of artificial intelligence, manufacturers like NVIDIA have picked up the ball and subsequently optimized their most professional solutions to perform even higher in this sense.

These two are, as you might imagine, the most well-known aspects of NVIDIA’s relationship with artificial intelligence. However, there is a third one that, while not unknown (in fact, we’ve told you about it here several times), is not so well known to the general public, and that’s development of solutions for automatic content creationmore specifically, graphic content (images and video), which is of course supported on its hardware.

The most famous example is GauGAN 2, AI built into your NVIDIA Canvas application. In its first generation, GauGAN offers us a selector of “materials” (clouds, sea, land, grass, sand, mountains, etc. So just fill the areas of the canvas with the type of element we want to include and, automatically, Canvas will generate the landscape based on our design. And with its second version, in addition to continuing to have this design mode, we can also simply enter a text description of what we want, and it will process the AI, which will automatically return an image that matches our description.

Another example solution AI designed by NVIDIA, we have it in Instant NeRFa development introduced earlier this year that, based on a few normal photographs, can generate all the volumetric information needed to transform them into a three-dimensional scene.

And today we know about a new project in this sense, Magic3D, a rather important step by NVIDIA, because in this case already we find general purpose 3D text to image AIthat is, able to generate images of any type, as well as some solutions of this type that have become very popular in the last year, but of course with the peculiarity of three dimensions compared to the 2D images that we see in the most common models today.

As you can see in the video, Magic3D respond to any type of image request and to create it, it uses a two-stage process that uses a raw model generated at low resolution, which is later rescaled and optimized, which we already know very well thanks to DLSS. In addition, Magic3D also provides cue-based editing of 3D meshes. So we can start from a low resolution 3D model and the basic message and edit the text to change the resulting model. And the key point is that it is possible to maintain the same style for successive generations of images (a concept often called consistency) and apply the style of a 2D image (such as a painting) to a 3D model.

NVIDIA hasn’t released, at least for now, the code for Magic3D, but since it says its intention is to democratize the creation of three-dimensional images, just as other AIs do with two-dimensional ones, we can expect it to start reaching users sooner rather than later.