# Red Hat releases llm-d 1.0, Kubernetes-native LLM serving orchestrator

> **Key takeaway:** Red Hat and community partners released llm-d 1.0, a CNCF-governed open source orchestration layer for distributed LLM inference on Kubernetes, claiming production stability with up to 98% prefix-cache hit rates and 6.6 million output tokens per minute in benchmarks.

**Published:** 2026-10-20T00:00:00+00:00 
**Source:** Red Hat Blog 
**Category:** distro-news 
**Canonical URL:** https://fosswire.org/news/red-hat-releases-llm-d-10-kubernetes-native-llm-serving-orch.html 

## Summary
llm-d 1.0 graduates from experimental to production-ready infrastructure for serving large open weight models at cluster scale. The project, now under CNCF governance with 500+ contributors across 25 organizations, adds priority-aware flow control, router high availability with hot standby, and inference-aware scheduling that preserves context locality. Red Hat is productizing the stack as Red Hat AI Inference for OpenShift AI and third-party Kubernetes environments.

## What llm-d 1.0 adds for production inference
- **Priority-aware flow control**: latency-sensitive interactive queries are protected from starvation by bursty background agent traffic
- **Router HA with hot standby**: coordinated rolling updates allow model/runtime upgrades without dropping live traffic
- **Inference-aware scheduling**: preserves context locality, disaggregates prefill from decode phases, balances across heterogeneous accelerator pools
- **Production metrics**: IBM Research deployment achieved 98% prefix-cache hit rate; 5-10x cost reduction vs. proprietary alternatives claimed
- **Benchmark results**: 753B-parameter GLM-5.2 model served on NVIDIA GPUs at 6.6 million output tokens/minute peak; 3,000 concurrent coding agents with zero preemptions

## Why it matters
llm-d's CNCF governance and PyTorch Foundation alignment positions it as a vendor-neutral control plane bridging model runtime and cloud-native operations. The 500+ contributor base across 25 organizations suggests broad industry investment in avoiding proprietary serving lock-in.