# llama.cpp v0.9.0 adds OCP FP8 quantization and multi-node tensor parallelism

> **Key takeaway:** The release introduces OCP FP8 E4M3 support, multi-node tensor parallelism via NCCL, and significant multi-modal expansions for audio and video.

**Published:** 2026-11-04T13:09:55+00:00 
**Source:** Llama.cpp Tensor Acceleration 
**Category:** ai-local-edge 
**Canonical URL:** https://fosswire.org/news/llamacpp-v090-adds-ocp-fp8-quantization-and-multi-node-tenso.html 

## Summary
The llama.cpp project has released version 0.9.0, a major update that significantly expands its quantization capabilities and hardware acceleration. A primary technical addition is the support for the OCP FP8 E4M3 tensor type, which allows for more efficient low-precision inference. The update also introduces a new flash_attn_ext_rows operation to improve unified KV cache efficiency during multi-sequence inference tasks.

Beyond quantization, the release enhances multi-modal capabilities by adding Chatterbox TTS support and Nemotron 3.5 transcription for 40 languages. The underlying ggml library has been updated to v0.29.0, bringing NCCL-based multi-node tensor parallelism and performance optimizations across a wide array of backends, including CUDA, Metal, Vulkan, and Hexagon. New architecture support has also been added for models including Mistral Large 4 and Metis-1.6.

## Technical breakdown of llama.cpp v0.9.0
- **Quantization and Precision:** Implementation of the OCP FP8 E4M3 tensor type with saturating mode and conversion helpers, alongside NVFP4 GGUF support for LFM3 models.
- **Compute Backend Optimizations:** ggml v0.29.0 introduces SVE kernels for CPU, DPAS flash attention for SYCL, fused MoE MMQ for CUDA, and split-K tuning for Vulkan.
- **Distributed Inference:** Addition of a multi-process communicator interface enabling multi-node tensor parallelism through NCCL.
- **KV Cache and Attention:** New flash_attn_ext_rows operator for efficient multi-sequence unified KV caches and the retention of Hadamard rotation constants on the GPU.
- **Multi-modal Expansion:** Integration of Chatterbox TTS, Nemotron 3.5 offline transcription with speaker attribution, and improved Qwen3-VL video prompt template support.

## Why it matters
This release lowers the barrier for high-performance local AI deployment by providing standardized FP8 support and multi-node scaling. It benefits developers building edge AI applications through improved audio/video processing and offers system administrators more efficient ways to run large-scale models across distributed hardware.