Why Does NVIDIA Claim 30x More Agentic Work Per Watt With Vera Rubin NVL72?

Agentic workloads eat tokens differently

A simple chat request uses a few thousand tokens. An agent researching an investment decision queries databases, searches filings, spawns sub-agents for peer comparisons, runs valuation models, and synthesizes a recommendation. Every step adds tokens to the context. The next step consumes that accumulated context as input. According to OpenRouter data, agentic workloads consume 15x more tokens than a simple chat request. Context can reach hundreds of thousands of input tokens. Input and output lengths vary widely across requests. Measuring a single inference request no longer captures the real workload.

The measurement that produced the 30x number

NVIDIA measured inference throughput using the SemiAnalysis AgentX workload. This workload consists of recorded real-world agentic coding sessions. Actual context growth, tool calls, and sub-agent spawning are preserved. On the DeepSeek V4 Pro model, GB300 NVL72 already delivered up to 15x better throughput per megawatt than the Hopper architecture. Vera Rubin NVL72 extends that advantage to 30x higher throughput per megawatt than GB300 NVL72. The results are early and currently pending SemiAnalysis review. They do not yet reflect Vera CPU performance for tool calling.

How the hardware and software produce the gain

The NVL72 scale-up domain connects 72 GPUs with high-bandwidth, low-latency communication. The domain enables this. Sixth-generation NVLink interconnect and NVLink Switches deliver 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet alternatives. This domain enables techniques that only work at this scale. Disaggregated serving separates context processing from response generation so each scales independently. Rate matching synchronizes prefill and decode GPU speeds. Large-scale expert parallelism distributes mixture-of-experts sub-networks across the domain. Distributed KV-caching extends memory across GPUs. KV-cache offloading tiers less-active context to host and storage. KV-aware routing directs requests to GPUs already holding relevant cached context. Fused CUDA kernels like MegaMoE combine computation and communication into single execution passes. Enhanced fifth-generation Tensor Cores and the third-generation Transformer Engine accelerate both prefill and decode. NVFP4 quantization compresses model weights to 4-bit precision. NVIDIA DSX MaxLPS technologies manage power across GPU, rack, and workload levels to provision up to 40% more GPUs within the same megawatt budget.

The seven-chip platform and what comes next

The full Vera Rubin platform is a seven-chip architecture. It includes the Vera CPU, Groq 3 LPU, NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX, and ConnectX-9 SuperNIC. All are purpose-built for AI factories deploying agents at scale. Vera Rubin is in full production and scaling across the ecosystem. Continuous software optimizations will improve performance on both Vera Rubin NVL72 and GB300 NVL72. For power-constrained AI factories, throughput per megawatt determines revenue. Cost per million tokens determines the profit margin on that revenue.