Hugging Face Transformers v5.20.0 Adds New Models

The latest release of Hugging Face Transformers, version 5.20.0, brings several new models to the table. These include Muse Spark Open, a multimodal mixture-of-experts model; Ling 3.0 Flash, a hybrid Mixture-of-Experts language model; Kolibri1, a mixture-of-experts reasoning model; MiniCPM-V 4.7, an efficient multimodal large language model; UnlimitedOcr, a single 3B-parameter OCR model; UltraBERT, a family of bidirectional text encoders; and MOSS-Transcribe-Diarize, an end-to-end audio understanding model. These additions enhance the capabilities of the Transformers library, providing users with more options for various tasks such as text generation, language understanding, and audio processing.

What's New in Transformers v5.20.0

• Muse Spark Open: A multimodal mixture-of-experts model that takes text, images, audio, and video as input and generates text, with 816B total parameters and a context window of up to 1,048,576 tokens.
• Ling 3.0 Flash: A hybrid Mixture-of-Experts language model with a decoder that groups five Kimi Delta Attention layers with one Multi-head Latent Attention layer.
• Kolibri1: A mixture-of-experts reasoning model focused on German and English, with 50 layers that route every token to 6 of 384 experts using sigmoid routing.
• MiniCPM-V 4.7: An efficient multimodal large language model that pairs a SigLIP vision encoder with a Qwen3.5 language model backbone, supporting both 4x and 16x visual downsampling modes.
• UnlimitedOcr: A single 3B-parameter OCR model that builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention.

Why it matters

The introduction of these new models in Hugging Face Transformers v5.20.0 expands the library's capabilities, offering users a wider range of tools for natural language processing, computer vision, and audio processing tasks. This can lead to advancements in various applications such as text generation, language translation, image recognition, and speech recognition, ultimately benefiting the development of AI and machine learning technologies.