NVIDIA releases Nemotron 3.5 Transcribe open-weight speech model with Granary v2 pipeline

NVIDIA has released Nemotron 3.5 Transcribe, an open-weight multilingual speech model that combines transcription, speaker diarization, translation, and audio question answering in a single checkpoint. The 30B-parameter model uses a hybrid Mamba-Transformer mixture-of-experts design with approximately 4B active parameters per token, achieving 5.33% average WER on the Hugging Face Open ASR Leaderboard for English and 4.62% mean WER across seven languages in the multilingual track. It supports 40 languages for transcription, 31 for English translation, and 13 for speaker attribution with up to eight speakers and one-hour recordings.

Alongside the model, NVIDIA published Granary v2, an open multilingual data curation pipeline and dataset on Hugging Face and ModelScope, plus NeMo Speech fine-tuning recipes. The release includes quantized versions for local deployment on RTX Spark, DGX Spark, and Jetson AGX Thor edge devices, as well as data-center scaling on Blackwell GPUs. A medical case study demonstrated 45.8% relative WER reduction through fine-tuning versus prompting alone.

Architecture and open-weight deployment

• Hybrid Mamba-Transformer MoE architecture: The 30B-parameter model activates only ~4B parameters per token through a mixture-of-experts design, placing it on the accuracy-active-parameter Pareto frontier and enabling efficient inference relative to dense 24B competitors.
• Unified multimodal checkpoint: A single model handles transcription (40 languages), translation to/from English (31 languages), speaker diarization (13 languages, up to 8 speakers), and audio Q&A with natural-language prompt control rather than separate pipelines.
• Sortformer speaker integration: Speaker cues reach the language model alongside acoustic features during generation, enabling real-time speaker attribution during decoding rather than post-hoc speaker-word alignment.
• Granary v2 data pipeline: The open curation pipeline and multilingual dataset allow developers to define custom written forms and adapt the model to domain vocabulary, accents, and speaker distributions beyond prompt-level control.
• Edge-to-datacenter quantization: Quantized variants support local deployment on NVIDIA RTX Spark, DGX Spark, and Jetson AGX Thor, with full-precision scaling on Blackwell data-center GPUs, though ROCm, Vulkan, and CPU/NPU backends are not mentioned.

Why it matters

For the open-source AI ecosystem, Nemotron 3.5 Transcribe offers a rare combination of state-of-the-art open-weight performance with reproducible fine-tuning infrastructure through Granary v2 and NeMo Speech, reducing dependence on proprietary speech APIs from cloud providers. The model's Apache-2.0 licensing of weights and training artifacts enables independent developers to deploy on self-managed NVIDIA hardware or study the data curation pipeline, though the CUDA-centric optimization limits hardware choice compared to fully vendor-neutral alternatives.