# Hugging Face Transformers v5.20.2 fixes multimodal input type loss in generation pipeline

> **Key takeaway:** Transformers v5.20.2 patches a bug where inputs lost their original type in prepare_inputs_for_generation, breaking multimodal model inference, and also fixes bfloat16 gradient zeroing with expert parallelism.

**Published:** 2026-10-19T17:45:37+00:00 
**Source:** Hugging Face Transformers 
**Category:** ai-open-models 
**Canonical URL:** https://fosswire.org/news/hugging-face-transformers-v5202-fixes-multimodal-input-type-.html 

## Summary
Hugging Face has released Transformers v5.20.2, a patch addressing four specific bugs in the widely-used machine learning library. The most significant fix resolves a type-casting issue in the generate() method's prepare_inputs_for_generation function, where input dictionaries could lose their original data types and cause failures in vision-language and other multimodal models. The release also corrects a training bug where expert gradients became zero in bfloat16 precision when expert parallelism size was smaller than FSDP shard size, adds proper input modality declarations for VLMs that previously reported text-only interfaces, and eliminates an unnecessary host-to-device memory copy in the eager_mask operation for modest performance improvement.

## What v5.20.2 patches in the generation and training stack
- **Input type preservation in generate():** The patch fixes dict casting in prepare_inputs_for_generation (PR #49731), where inputs could silently lose their original Python/PyTorch types during the generation pipeline, specifically affecting multimodal models that rely on heterogeneous input structures mixing tensors, lists, and metadata.
- **bfloat16 expert gradient recovery:** PR #49688 resolves a distributed training edge case where gradients for Mixture-of-Experts layers became zero in bfloat16 when expert parallelism (ep_size) was configured smaller than FSDP shard size, a configuration relevant to large-scale MoE training on clusters with limited expert replication.
- **VLM modality declaration fix:** PR #49677 corrects the input modality metadata for vision-language models that were incorrectly reporting text-only interfaces, which could break downstream pipeline routing and processor selection in the Transformers auto-mapping system.
- **Host-to-device copy elimination:** PR #49684 removes an unnecessary CPU-to-GPU memory transfer in eager_mask, a core attention masking utility, reducing synchronization overhead during model forward passes.

## Why it matters
This patch directly affects production inference pipelines running multimodal models like LLaVA or Qwen-VL through Transformers, where the input type bug could cause silent failures or incorrect outputs in vision-language tasks. The bfloat16 gradient fix is particularly important for researchers and organizations training large MoE models with limited GPU budgets, as it removes a hidden barrier to using cheaper bfloat16 precision with expert parallelism configurations that do not fully replicate experts across all FSDP ranks.