FOSSWire
ai-local-edge

Ollama v0.40.4 adds CLI decision model support and MLX improvements

Brief written by the FOSSWire AI editor from Ollama Local AI Runtimes

1 min read Impact: 5/10

Key Takeaway

Ollama now supports decision models directly via the CLI for tasks like classification and scoring.

Summary

Ollama version 0.40.4 introduces the ability to run decision models through the terminal for yes/no questions, text classification, and rubric scoring. The update also brings significant performance and reliability improvements to the MLX runner on Apple silicon and enhances Gemma 4 prefill performance on NVIDIA GPUs.

Technical updates in Ollama v0.40.4

  • Decision models can be invoked via ollama run --ask for classification and scoring.
  • MLX runner updates include support for decision models on Apple silicon and improved speculative decoding reliability.
  • llama.cpp has been updated with native decision model support.
  • Multimodal embeddings support added via llama-server.
  • Performance gains for Gemma 4 prefill on NVIDIA GPUs and improved inference through manifest and model reuse.

Why it matters

The integration of decision models into the CLI lowers the barrier for automating text classification and structured data extraction using local LLMs.

Original reference: github.com →