Local AI & Edge
Running AI models locally and on edge devices.
11 stories • RSS feed
Most important in the last 30 days
- vLLM 0.31.0 improves DeepSeek-V4.1-Flash performance and large-scale serving • vLLM High-Throughput Serving Engine • Impact 7/10
vLLM 0.31.0 introduces significant optimizations for DeepSeek models, enhanced speculative decoding, and new large-scale serving backends.
- LocalAI 4.11.0 adds audio diarization and model failover chains • LocalAI Open Source Self-Hosted AI • Impact 6/10
LocalAI 4.11.0 introduces advanced audio scene processing, structured decision models, and resilient model serving via failover chains.
Latest stories
- Ollama v0.40.6 adds GGUF q2_0 tensor support • Ollama Local AI Runtimes • Impact 4/10
Ollama v0.40.6 introduces support for q2_0 tensor and file types in the GGUF format.
- vLLM v0.32.0 fixes Mamba2 NaN errors on sm90 hardware • vLLM High-Throughput Serving Engine • Impact 4/10
The vLLM update resolves an issue where the Mamba2 chunk scan kernel produced NaN outputs on sm90 architecture.
- Ollama v0.40.6-rc1 improves proxy support for registry redirects • Ollama Local AI Runtimes • Impact 4/10
Ollama adds support for proxies when handling registry redirect targets and fixes a crash in blob downloads.
- Ollama v0.40.5 adds NVIDIA Nemotron 3.5 support and cloud decision models • Ollama Local AI Runtimes • Impact 5/10
Ollama v0.40.5 introduces support for NVIDIA Nemotron 3.5 models and allows running decision models via the ollama.com cloud.
- Ollama v0.40.5-rc1 adds Nemotron 3.5 support and MLX improvements • Ollama Local AI Runtimes • Impact 4/10
Ollama's latest release candidate introduces Nemotron 3.5 support and significant bug fixes for the MLX backend.
- Ollama v0.40.4 adds CLI decision model support and MLX improvements • Ollama Local AI Runtimes • Impact 5/10
Ollama now supports decision models directly via the CLI for tasks like classification and scoring.
- Ollama v0.40.4-rc1 adds Kev decision scoring for MLX • Ollama Local AI Runtimes • Impact 4/10
Ollama's new release candidate introduces Kev decision scoring support for the MLX framework.
- Jan 0.8.6 hotfix addresses Linux AppImage and NVIDIA GPU issues • Jan Open Source Desktop AI Assistant • Impact 5/10
Jan 0.8.6 fixes critical crashes in the Linux AppImage and restores CUDA support for older NVIDIA Turing GPUs.
- Jan Open Source Desktop AI Assistant Releases Version 0.8.5 • Jan Open Source Desktop AI Assistant • Impact 5/10
Jan Open Source Desktop AI Assistant has released version 0.8.5, which includes fixes and improvements to the llama.cpp engine and other features.