Nvidia announced Nvidia NIM at GTDC 2024, enabling developers to easily deploy AI models in their enterprise environment.

At GTC 2024, Nvidia introduced Nvidia NIM, a new software platform that makes it easier to implement generative AI models in development environments. It aims to bridge the gap between the complex world of AI developments on the one hand and the operational requirements of business environments on the other.

AI containers

Nvidia describes NIM as “a set of optimized cloud-native microservices designed to accelerate time to market and simplify the deployment of generative AI models anywhere, in the cloud, in the data center, and on GPU-accelerated workstations.”

NIM encompasses the software work Nvidia has done around model inference and optimization. This capacity is then made accessible by combining a specific model with an optimized model Inference engine. It then packages everything into a container and makes it accessible as a microservice.

Typically, developers would take weeks to months to set up similar containers, ideally if the company has AI talent. With NIM, Nvidia offers ready-to-use AI containers that use the hardware as a base layer and add the microservices as a core software layer.

Support NIM

NIM supports various AI models such as community models, Nvidia AI Foundation models, and custom models provided by Nvidia partners for use cases across multiple domains. These include large language models, visual language models, and models for speech, images, video, 3D, and much more.

Developers who want to test the latest generative AI models can do so using Nvidia-managed cloud APIs from the Nvidia API catalog. Another option is to host the models yourself by downloading NIM and deploying them using Kubernetes.