Big Tech FOSS

NVIDIA Vera Rubin NVL72 shows high agentic inference performance in MLPerf Endpoints v1.0

Key Takeaway

Provisional MLPerf results indicate the NVIDIA Vera Rubin NVL72 platform achieves up to 10x higher throughput per kilowatt compared to the GB300 NVL72 for agentic AI workloads.

Summary

NVIDIA has released provisional performance results for its Vera Rubin NVL72 platform using the MLPerf Endpoints v1.0 benchmark. The testing focuses on agentic inference, which involves AI agents managing long conversation histories, tool calls, and complex reasoning tasks. The benchmarks compare the Vera Rubin architecture against the previous GB300 NVL72 generation across several large language models.

According to the data, the Vera Rubin NVL72 demonstrates significant efficiency gains, specifically in throughput per kilowatt of power consumed. The platform was tested using Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.6-35B-A3B models. The results suggest that the new hardware architecture is specifically optimized to handle the increasing context lengths and computational demands of multi-turn AI agent workflows.

Technical optimizations for agentic workloads

  • Hardware Architecture: The Vera Rubin GPUs utilize HBM4 memory, sixth-generation NVLink, and updated Tensor Cores to manage high-concurrency inference.
  • Context Management: Performance is bolstered by context-aware serving techniques, including prefix caching, state reuse, and NVIDIA Dynamo routing to minimize redundant computations.
  • Model-Specific Kernels: Optimizations include the DeepSeek MegaMoE kernel and Qwen multi-token prediction with Tensor Core verification to reduce data movement and sequential decoding steps.
  • Benchmark Metrics: The MLPerf Endpoints v1.0 evaluation measures the balance between serving capacity (throughput) and responsiveness (end-to-end interactivity) for coding and enterprise API workflows.

Why it matters

As AI agents become more prevalent in enterprise and developer workflows, these hardware advancements dictate the cost and latency of running large-scale, multi-turn LLM applications. For the open-source community, the efficiency of these proprietary accelerators influences how open models like DeepSeek and Qwen are deployed in production environments.

Original reference: developer.nvidia.com →