NVIDIA NIM 1.4 Ready to Deploy with 2.4x Faster Inference

The demand for ready-to-deploy high-performance inference is growing as generative AI reshapes industries. NVIDIA NIM provides production-ready microservice containers for AI model inference, constantly improving enterprise-grade generative AI performance. With the upcoming NIM version 1.4 scheduled for release in early December, request performance is improved by up to 2.4x out-of-the-box with the same single-command deployment experience.  

At the core of NIM are multiple LLM inference engines, including NVIDIA TensorRT-LLM, which enables it to achieve speed-of-light inference performance. With each release, NIM incorporates the latest advancements in kernel optimizations, memory management, and scheduling from these engines to improve performance. 

In NIM 1.4, significant improvements in kernel efficiency, runtime heuristics, and memory allocation were added, translating into up to 2.4x faster inferencing, compared to NIM 1.2. These advancements are crucial for businesses that rely on quick responses and high throughput for generative AI applications.

NIM also benefits from continuous updates to full-stack accelerated computing, which enhances performance and efficiency at every level of the computing stack. This includes support for the latest NVIDIA TensorRT and NVIDIA CUDA versions, further boosting inference performance. NIM users benefit from these continuous improvements without manually updating software.

NIM brings together a full suite of preconfigured software to deliver high-performance AI inferencing with minimal setup, enabling developers to quickly get started with high-performance inference. 

A continuous innovation loop means that every improvement in TensorRT-LLM, CUDA, and other core accelerated computing technologies immediately benefits NIM users. Updates are seamlessly integrated and delivered through updates to NIM microservice containers, eliminating the need for manual configuration and reducing the engineering overhead typically associated with maintaining high-performance inference solutions.

Get started today