Open AI Models

Thinking Machines Lab releases Inkling 2 Mini multimodal looped transformer model

Key Takeaway

Inkling 2 Mini uses a looped transformer architecture to achieve the performance of a much larger model while remaining small enough for local deployment.

Summary

Thinking Machines Lab has announced the release of Inkling 2 Mini, a multimodal Mixture-of-Experts (MoE) model designed specifically for local AI environments. The model utilizes a unique looped transformer architecture where 12 of its 24 physical decoder layers are executed three times, effectively simulating a 48-layer model. This approach allows the model to maintain high performance with a smaller physical parameter count.

Inkling 2 Mini features approximately 26 billion total parameters, with 3 billion active parameters per token. It supports a massive 1-million-token context window and can process both text and image inputs. The release includes support for major inference frameworks including Hugging Face Transformers, llama.cpp (Llama App), MLX, and SGLang, ensuring compatibility across various hardware backends.

Technical architecture of the looped transformer

  • Looped Layer Execution: The model executes layers 0–5 once, repeats layers 6–17 three times, and executes layers 18–23 once, using cross-pass messages to allow information to flow between iterations.
  • Mixture-of-Experts (MoE) Configuration: The architecture employs 128 experts per layer, with the router selecting 8 SwiGLU experts for each token processed.
  • Attention Mechanism: The model utilizes a combination of five sliding-window layers (with a 1,024-token local window) and one full-attention layer per six-layer block, using 24 query heads and eight KV heads.
  • Mixed-Precision Weights: The model is shipped using NVFP4 and MXFP8 formats for expert weights to optimize memory and compute efficiency.
  • Variable Loop Research Variant: A separate research artifact allows users to adjust the number of loop passes, enabling a trade-off between generation speed and output quality.

Why it matters

By providing day-zero support for llama.cpp and MLX, Inkling 2 Mini is immediately accessible for local deployment on consumer hardware, including Mac (via MLX) and standard Linux/Windows setups. Its ability to mimic larger models through weight reuse offers a path for developers to run high-capability multimodal agents on limited local compute resources.

What to do

  • Users can deploy Inkling 2 Mini using Hugging Face Transformers, llama.cpp (Llama App), MLX, or SGLang.
  • To use the model in Transformers, use the AutoModelForMultimodalLM class with the model ID 'thinkingmachines/Inkling-2-Mini'.

Original reference: huggingface.co →