OpenAI has released an AI model, Shap-E, that allows you to convert text into 3D images

Following book reviews via ChatGPT, now thanks to Shap-E you can also turn these texts into 3D images. This is the new AI model from OpenAI. You can even convert the results to an STL file and send it to a 3D printer.

Work

Shap-E is available for free on GitHub. Once you have downloaded all the files and mockups, the internet is no longer needed to use the mockup. It also doesn’t require an API key from OpenAI, so you won’t be billed for using it.

It takes something to get everything installed and running, OpenAI doesn’t immediately deliver a manual. Either way, start from the Python pip. Once installed, you can access it from Jupyter Notebook.

There are three different notebooks you can start with: text-to-3d (a text prompt), image-to-3d (to convert a 2D image into a 3D object), and encode_model (to convert an existing 3D image). ). with Blender).

Good development

Unlike Point-E – an earlier model by OpenAI for generating 3D images – Shap-E doesn’t generate point clouds based on text prompts, but instead forms the outlines of objects directly via a feature called “Neural Radiance Fields”. simply NeRFs. This way the images are much clearer.

NeRFs are also used in virtual or augmented reality to create the most faithful looking and photorealistic environment possible. Shap-E uses this technology to create the outline and texture of the objects suggested by the text input.

Shap-E is also much faster than its predecessor; everything runs on a single NVIDIA V100 GPU. The developers admit that large projects require significantly more power than Point-E. In addition, the AI ​​still has problems with more complex objects for the time being. But overall, this model is a big step forward.