Ollama v0.40.16-rc0 adds CUDA 13.5 support for Windows ARM64 and MLX optimizations

Ollama has released version 0.40.16-rc0, a pre-release update focused on expanding hardware compatibility and refining API behaviors. The update introduces significant support for Windows ARM64 users by adding CUDA 13.5 and SM 103 support, facilitating better local AI execution on ARM-based Windows hardware.

Additionally, the release includes optimizations for the MLX runner, specifically targeting memory efficiency for the Qwen3.8 Flash Next model. The update also refines the OpenAI-compatible API by implementing truncation options in response requests and ensuring the /v1/models endpoint returns an empty array when no models are present.

Technical improvements in Ollama v0.40.16-rc0

• Windows ARM64 Support: Integration of CUDA 13.5 and SM 103 support to improve GPU acceleration on ARM-based Windows systems.
• MLX Runner Optimizations: Implementation of memory-efficient execution for Qwen3.8 Flash Next and the addition of ripple preview support.
• OpenAI API Refinements: Support for forwarding truncation options in response requests and standardized empty array returns for the /v1/models endpoint when no models are installed.
• Core Dependency Update: A version bump of the underlying llama.cpp backend to version b12177.

Why it matters

These updates improve the viability of local LLM deployment on diverse hardware, particularly for developers using Windows ARM64 devices. By refining OpenAI-compatible API behaviors and optimizing MLX for specific models, Ollama continues to lower the barrier for integrating local AI into existing software workflows without vendor lock-in.