NVIDIA has announced the release of a groundbreaking language model, Llama 3.1-Nemotron-51B, which promises to deliver unprecedented accuracy and efficiency in AI performance. Derived from Meta’s Llama-3.1-70B, the new model employs a novel Neural Architecture Search (NAS) approach, significantly enhancing both its accuracy and efficiency. According to the NVIDIA Technical Blog, this model can fit on a single NVIDIA H100 GPU even under high workloads, making it more accessible and cost-effective.

Superior Throughput and Workload Efficiency

The Llama 3.1-Nemotron-51B model outperforms its predecessors with 2.2 times faster inference speeds while maintaining nearly the same level of accuracy. This efficiency allows for 4 times larger workloads on a single GPU during inference, thanks to its reduced memory footprint and optimized architecture.

Optimized Accuracy Per Dollar

One of the significant challenges in adopting large language models (LLMs) is their inference cost. The Llama 3.1-Nemotron-51B model addresses this by offering a balanced tradeoff between accuracy and efficiency, making it a cost-effective solution for various applications, ranging from edge systems to cloud data centers. This capability is particularly advantageous for deploying multiple models via Kubernetes and NIM blueprints.

Simplifying Inference with NVIDIA NIM

The Nemotron model is optimized with TensorRT-LLM engines for higher inference performance and is packaged as an NVIDIA NIM inference microservice. This setup simplifies and accelerates the deployment of generative AI models across NVIDIA’s accelerated infrastructure, including cloud, data centers, and workstations.

Under the Hood – Building the Model with NAS

The Llama 3.1-Nemotron-51B-Instruct model was developed using efficient NAS technology and training methods, allowing for the creation of non-standard transformer models optimized for specific GPUs. This approach includes a block-distillation framework to train various block variants in parallel, ensuring efficient and accurate inference.

Tailoring LLMs for Diverse Needs

NVIDIA’s NAS approach allows users to select their optimal balance between accuracy and efficiency. For instance, the Llama-3.1-Nemotron-40B-Instruct variant was created to prioritize speed and cost, achieving a 3.2 times speed increase compared to the parent model with a moderate decrease in accuracy.

Detailed Results

The Llama 3.1-Nemotron-51B-Instruct model has been benchmarked against several industry standards, demonstrating its superior performance in various scenarios. It doubles the throughput of the reference model, making it cost-effective across multiple use cases.

The Llama 3.1-Nemotron-51B-Instruct model provides a new set of opportunities for users and companies aiming to utilize highly accurate foundation models cost-effectively. Its balance between accuracy and efficiency makes it an attractive option for builders and showcases the effectiveness of the NAS approach, which NVIDIA plans to extend to other models.

Image source: Shutterstock