TMLR · 2026
Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning
Key takeaway
Visionary-R1 reduces shortcut learning in visual reinforcement learning by requiring image interpretation before reasoning through a caption–reason–answer output format. arXiv abstract · v3
Abstract
Learning general-purpose reasoning capabilities has long been a challenging problem in AI. Recent research in large language models (LLMs), such as DeepSeek-R1, has shown that reinforcement learning techniques like GRPO can enable pre-trained LLMs to develop reasoning capabilities using simple question-answer pairs. In this paper, we aim to train visual language models (VLMs) to perform reasoning on image data through reinforcement learning and visual question-answer pairs, without any explicit chain-of-thought (CoT) supervision. Our findings indicate that simply applying reinforcement learning to a VLM -- by prompting the model to produce a reasoning chain before providing an answer -- can lead the model to develop shortcuts from easy questions, thereby reducing its ability to generalize across unseen data distributions. We argue that the key to mitigating shortcut learning is to encourage the model to interpret images prior to reasoning. Therefore, we train the model to adhere to a caption-reason-answer output format: initially generating a detailed caption for an image, followed by constructing an extensive reasoning chain. When trained on 273K CoT-free visual question-answer pairs and using only reinforcement learning, our model, named Visionary-R1, outperforms strong multimodal models, such as GPT-4o, Claude3.5-Sonnet, and Gemini-1.5-Pro, on multiple visual reasoning benchmarks.
Author abstract · arXiv abstract · v3
Publication
Transactions on Machine Learning Research (TMLR), 2026
Paper and resources
Research topics
Visionary-R1 · Visual reasoning · Shortcut learning · Reinforcement learning · GRPO · Caption–reason–answer · CoT-free supervision · Out-of-distribution generalization
Research problem and approach
Applying reinforcement learning directly to visual question answering can encourage shortcuts learned from easy questions. Visionary-R1 explicitly encourages image interpretation before constructing a reasoning chain. arXiv abstract · v3
Main contributions
- Analyzes shortcut learning and failures to generalize to unseen data distributions. arXiv abstract · v3
- Trains with reinforcement learning on question-answer pairs without explicit chain-of-thought supervision. arXiv abstract · v3
Method comparison
| Approach | Key difference |
|---|---|
| Direct visual-question RL | Can exploit shortcuts from easier questions instead of learning robust visual reasoning. |
| Visionary-R1 | Uses a caption–reason–answer format to encourage image interpretation before reasoning, without CoT labels. |
Selected results
- On the 3B backbone, Visionary-R1 scores 69.4 on MathVista, 24.7 on MathVision and 84.1 on MMBench, compared with 61.8, 20.3 and 78.6 for the GRPO baseline in the same table. Table 1 · 3B backbone comparison · arXiv v3
- In the ChartQA-training ablation, adding captions to GRPO changes MathVista/MathVision from 59.0/18.2 to 62.6/20.9; the caption-reward configuration reaches 64.6/22.7. Adding only a caption-length reward gives 62.0/20.3, so longer captions alone do not explain the improvement. Table 2 · perception-reward ablation · arXiv v3
Cite this paper
Jiaer Xia, Yuhang Zang, Peng Gao, Sharon Li, Kaiyang Zhou. Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning. Transactions on Machine Learning Research, 2026.
@article{arxiv250514677,
title = {{Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning}},
author = {Jiaer Xia and Yuhang Zang and Peng Gao and Sharon Li and Kaiyang Zhou},
journal = {Transactions on Machine Learning Research},
year = {2026},
url = {https://jmlr.org/tmlr/papers/bib/JWkZXBgh5a.bib}
}