Liquid AI releases LFM3: 2B-parameter multimodal models for edge deployment

Liquid AI has released LFM3, a new generation of Liquid Foundation Models designed as a shared backbone for specialized multimodal intelligence. The family centers on LFM3-1.7B-Base, a 24-layer model with 1.7 billion parameters that branches into six variants: text, vision-language, audio, omni, and two decision models. All models target the 2B parameter range and are built for edge deployment, with the text model fitting in approximately 1 GB at 4-bit precision and supporting a 256K token context window.

The architecture introduces several changes from LFM2: a tri-hybrid layer mix of gated short convolutions, sliding-window attention, and global attention; head-wise gated attention to suppress activation outliers; partial RoPE for global layers; and sandwich norm for stable activations. The base model was pre-trained on 22 trillion tokens, with the final 3.5 trillion focusing on agentic traces and tool calls to extend context capability. New in-house vision and audio encoders reduce compute requirements compared to previous approaches, with the vision encoder using 3x less compute than SigLIP2 and the audio system achieving sub-50ms time-to-first-token with streaming-capable speech synthesis via LiquidCodec.

Architecture and edge-deployment changes in LFM3

• Tri-hybrid layer composition: LFM3-1.7B-Base uses 13 gated short convolutions (kernel size 7, up from 3 in LFM2), 5 sliding-window attention layers with 256-token windows, and 6 global attention layers, replacing LFM2's simpler convolution-attention hybrid.
• Attention refinements: Head-wise gated attention applies a sigmoid gate per head to scale or suppress outputs, which helps control activation outliers that degrade quantization; global attention layers use partial RoPE (only highest-frequency half rotated) while SWA layers retain full RoPE.
• Training scale and curriculum: Pre-training consumed 22T tokens with a 128K vocabulary shared with LFM2.5-8B-A1B, with the final 3.5T tokens emphasizing agentic traces and tool calls to extend context window to 256K tokens.
• In-house vision encoder: A 164M-parameter convolution-plus-attention encoder matches Qwen3.5-2B's vision encoder accuracy while using 2.2x fewer FLOPs and 3x less compute than prior SigLIP2 encoders; video processing embeds frame pairs into single temporal slices, halving temporal sequence length.
• Audio pipeline and codec: The causal audio encoder enables prefill during speech with time-to-first-token as low as 50 ms; LiquidCodec synthesizes speech at approximately 87x real-time on a single Apple M4 Max CPU thread with streaming decode support.

Why it matters

LFM3's single-GPU fine-tuning requirement and sub-1GB 4-bit deployment target lower barriers for independent developers and small teams building specialized multimodal applications without cloud dependency. The explicit hardware-in-the-loop search covering Raspberry Pi 5, Qualcomm Hexagon NPU, and diverse CPU/GPU backends signals practical commitment to open edge deployment rather than vendor-locked API services, though the open-source licensing status of model weights and training code remains to be clarified in the model cards.