# ViTacFormer tactile model hits 40% success on bimanual cup task, up from 0% vision-only

> **Key takeaway:** A two-stage architecture that predicts future tactile states rather than passively consuming touch data improved bimanual manipulation success from 0% to 40% on a contact-rich deformable object task.

**Published:** 2026-10-18T23:05:40+00:00 
**Source:** ROS Robot Operating System News 
**Category:** robotics-science 
**Canonical URL:** https://fosswire.org/news/vitacformer-tactile-model-hits-40-success-on-bimanual-cup-ta.html 

## Summary
ROBOTIS researchers tested three imitation learning approaches on a bimanual paper cup unstacking and stacking task using their AI Worker platform with dual HX5-D20 hands (54-DoF total). A baseline ACT policy using only RGB cameras and joint states failed completely. Adding raw tactile pressure arrays as MLP-projected tokens improved grasp stability but still only achieved 10% success. The ViTacFormer architecture, which first predicts future tactile states from multimodal history before conditioning actions on those predictions, reached 40% success. The work includes published code, video, and documentation.

## How ViTacFormer structures tactile prediction
- **Task**: Bimanual unstacking and stacking of nested paper cups — a contact-rich manipulation problem where visual observation alone cannot distinguish grasping one cup versus two
- **Hardware**: AI Worker with dual 20-DoF HX5-D20 hands; each fingertip carries a tactile pressure array; full action space is 54-DoF
- **Baseline ACT**: RGB + joint states only → 0% success; policy could not regulate contact force, either knocking cups over or failing to separate them
- **Tactile ACT**: Raw fingertip pressure arrays projected through MLP, added as input tokens to ACT → 10% success; improved grasp stability but still tended to lift both cups together
- **ViTacFormer**: Two-stage architecture — (1) predicts future tactile states from visual, proprioceptive, and tactile history, (2) conditions action decoder on predicted touch tokens → 40% success
- **Key design insight**: Treating touch as a predictive signal rather than passive sensor input enables anticipatory force regulation for sliding separation

## Why it matters
The result suggests that simply adding tactile sensors to robot hardware without corresponding architectural innovation yields limited gains for deformable object manipulation. The open publication of code and documentation may help ROS 2 and physical AI researchers integrate predictive tactile reasoning into their own imitation learning pipelines.