Ollama v0.40.17-rc0 improves Vulkan GPU support and MLX concurrent generation

Ollama has released the v0.40.17-rc0 pre-release, which introduces several performance and compatibility optimizations for local large language model (LLM) execution. The update focuses on improving hardware utilization across different compute backends, specifically targeting Vulkan-based integrated GPUs and Apple's MLX framework.

In addition to backend improvements, the release includes updates to the ChatGPT CLI functionality and internal dependency bumps for llama.cpp and MLX. These changes aim to refine the user experience for developers running local AI workloads on diverse hardware configurations.

Technical improvements in Ollama v0.40.17-rc0

• Vulkan Backend: Implements mmap for integrated GPUs using the Vulkan API, which can optimize memory management during model loading.
• MLX Framework: Adds support for concurrent generation within the MLX backend, allowing for more efficient processing on Apple Silicon hardware.
• Dependency Updates: Includes version bumps for the core llama.cpp (b12257) and both MLX and MLX-C libraries to ensure compatibility with recent model optimizations.
• CLI Enhancements: Improves the ChatGPT CLI with better restore capabilities and graceful restart functionality.

Why it matters

By optimizing Vulkan memory mapping and MLX concurrency, Ollama continues to lower the barrier for running high-performance local AI on consumer-grade integrated graphics and Apple Silicon. These updates are critical for developers building privacy-focused, local-first AI applications that must run efficiently without dedicated high-end discrete GPUs.