# Colibri engine runs 744B parameter LLM on 16 GB RAM

> **Key takeaway:** The Colibri inference engine enables massive Mixture-of-Experts models to run on modest hardware by swapping weights from an SSD to RAM.

**Published:** 2026-10-23T14:47:16+00:00 
**Source:** Hackster.io News 
**Category:** open-hardware 
**Canonical URL:** https://fosswire.org/news/colibri-engine-runs-744b-parameter-llm-on-16-gb-ram.html 

## Summary
Jim Bruges has demonstrated a method to run extremely large language models, including a 744-billion-parameter GLM-5.3, on an Arduino VENTUNO Q single-board computer with only 16 GB of RAM. Using the open source Colibri inference engine, the system avoids the need to load entire models into memory by utilizing the Mixture-of-Experts architecture. While the process is slow due to SSD-to-RAM data transfers, it proves that massive models can be accessed on hardware without high-end GPUs or massive RAM capacities.

## How Colibri manages memory
- Uses **Mixture-of-Experts (MoE)** architecture to activate only a subset of parameters per token.
- Stores model weights on an **NVMe SSD** and loads necessary experts into RAM on demand.
- Implements a **96-slot expert cache** to reduce frequent storage access.
- Tested on **Ubuntu 24.04** using four **Cortex-A78C CPU** cores.
- Performance for Qwen3.6-35B-A3B reached ~0.77 tokens per second, while GLM-5.3 reached ~0.03 tokens per second.

## Why it matters
This approach provides a viable path for experimenting with massive AI models on low-cost, accessible hardware where latency is not the primary concern.