NVIDIA has unveiled its latest AI model, DeepSeek-R1, which boasts an impressive 671 billion parameters. This cutting-edge model is now available as a preview through the NVIDIA NIM microservice, according to a recent NVIDIA blog post. DeepSeek-R1 is designed to help developers create specialized AI agents with state-of-the-art reasoning capabilities.

DeepSeek-R1’s Unique Capabilities

DeepSeek-R1 is an open model that leverages advanced reasoning techniques to deliver accurate responses. Unlike traditional models, it performs multiple inference passes over queries, utilizing methods like chain-of-thought and consensus to arrive at the best possible answers. This process, known as test-time scaling, demonstrates the importance of accelerated computing for agentic AI inference.

The model’s design allows it to iteratively ‘think’ through problems, generating more output tokens and longer generation cycles. This scalability is crucial for achieving high-quality responses and necessitates substantial test-time computing resources.

NIM Microservice Enhancements

The DeepSeek-R1 model is now accessible as a microservice on NVIDIA’s build platform, offering developers the opportunity to experiment with its capabilities. The microservice can process up to 3,872 tokens per second on a single NVIDIA HGX H200 system, showcasing its high inference efficiency and accuracy, particularly for tasks requiring logical inference, reasoning, and language understanding.

To facilitate deployment, the NIM microservice supports industry-standard APIs, allowing enterprises to maximize security and data privacy by running it on their preferred infrastructure. Additionally, NVIDIA AI Foundry and NVIDIA NeMo software enable enterprises to create customized DeepSeek-R1 NIM microservices for specialized AI applications.

Technical Specifications and Performance

DeepSeek-R1 is a mixture-of-experts (MoE) model, featuring 256 experts per layer, with each token being routed to eight separate experts in parallel for evaluation. The model’s real-time performance requires a high number of GPUs with substantial compute capabilities, connected through high-bandwidth, low-latency communication systems to effectively route prompt tokens.

The NVIDIA Hopper architecture’s FP8 Transformer Engine and NVLink bandwidth play a critical role in achieving the model’s high throughput. This setup allows a single server with eight H200 GPUs to run the full model efficiently, delivering significant computational performance.

Future Prospects

The upcoming NVIDIA Blackwell architecture is set to enhance test-time scaling for reasoning models like DeepSeek-R1. It promises to bring substantial improvements in performance with its fifth-generation Tensor Cores, capable of delivering up to 20 petaflops of peak FP4 compute performance, further optimizing inference tasks.

Developers interested in exploring the capabilities of the DeepSeek-R1 NIM microservice can do so on NVIDIA’s build platform, paving the way for innovative AI solutions in various sectors.

Image source: Shutterstock