# NVIDIA TensorRT-LLM 1.4.0rc2 adds Rubin GPU support, FP4/FP8 inference, and breaking API changes

> **Key takeaway:** TensorRT-LLM 1.4.0rc2 introduces NVFP4 and FP8 inference for NVIDIA's upcoming Rubin architecture, expands model support for GLM-5, MiniMax-M3, and Qwen3 families, and makes four breaking API changes including Python-native LlmRequest fields and KV cache manager V2 migration.

**Published:** 2026-10-28T15:51:34+00:00 
**Source:** NVIDIA TensorRT-LLM Releases 
**Category:** ai-local-edge 
**Canonical URL:** https://fosswire.org/news/nvidia-tensorrt-llm-140rc2-adds-rubin-gpu-support-fp4fp8-inf.html 

## Summary
NVIDIA has released TensorRT-LLM v1.4.0rc2, a pre-release update to its open-source inference engine for large language models, with Python wheels published as 1.4.0rc2.post1 to resolve PyPI publication blockers. The release significantly expands hardware and model coverage, adding NVFP4 low-latency decode support for the upcoming Rubin architecture (SM103) across GLM-5.2, MiniMax-M3, GPT-OSS-120B, and DeepSeek-R1-0528 models, alongside FP8 and static NVFP4 paths for Cosmos3 and FLUX.1-dev visual generation workloads.

The release contains four documented breaking API changes: LlmRequest fields migrate from C++ to Python, KV cache manager V1 support is removed from transceiver V2, obsolete executor backend compatibility paths are deleted, and zero-copy mode is enabled for Mooncake Store. Additional features include native KV cache retention and cross-turn block reuse, FP4 MLA in disaggregated serving, MXFP8 refitting with CUDA graphs, and optimizations for DeepSeek-V4, Kimi K3, and Qwen3 vision-language encoders. A known issue remains with tensor-parallel RPC orchestration hanging during model initialization when reusing MPI worker pools across threads.

## Architecture and inference changes in TensorRT-LLM 1.4.0rc2
- **Rubin/SM103 architecture support:** Adds NVFP4 low-latency decode defaults for GLM-5.2, GPT-OSS-120B, and DeepSeek-R1-0528, plus static NVFP4 with per-tensor FP8 attention for FLUX.1-dev, positioning the codebase for NVIDIA's next-generation datacenter GPUs.
- **KV cache system overhaul:** Native retention and cross-turn generation-block reuse are added to KV cache V2, with sliding-window prefix commits at fork points and first-N/last-N mixed-precision caching; FP4 MLA is now supported in disaggregated serving deployments.
- **Breaking API migrations:** LlmRequest fields move from C++ to Python structures, KV cache manager V1 is dropped from transceiver V2, obsolete executor backend paths are removed, and Mooncake Store gains zero-copy mode—changes that will require downstream integration updates.
- **Model-specific optimizations:** Fused RMSNorm/RoPE kernels for DeepSeek-V4 and DSpark on Rubin, shared CUDA graph setup for Qwen3-family vision-language encoders, prefill-chunk scheduling for multimodal encoders, and asynchronous Ulysses execution for MiniMax-H3.
- **Precision and kernel improvements:** MXFP8 refitting with piecewise CUDA graphs and Ray collectives, BF16 tensor-core router GEMMs for MiniMax-M3, SwiGLU fusion into BF16 gate/up GEMM epilogues on SM100, and default DSA paged-MQA indexing migrated to CuTe DSL.

## Why it matters
For AI infrastructure engineers and cloud providers running local LLM inference, this release advances NVIDIA's open-source inference stack toward Rubin-era hardware while deepening the vendor-specific CUDA/NVFP4 dependency chain. The breaking API changes signal maturation of the KV cache V2 and Python-native serving paths, but will force synchronization between TensorRT-LLM deployments and downstream frameworks like vLLM or custom serving layers. The expanded Qwen3, GLM, and MiniMax model support reflects NVIDIA's continued prioritization of Chinese-origin model families in its optimization targets, relevant for global distribution strategies and competitive positioning against AMD ROCm and Intel OpenVINO ecosystems.