ICLR · 2026
CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
Key takeaway
CapRL trains dense captioners with verifiable question-answering rewards: a useful caption should let a vision-free language model answer questions about the image. arXiv abstract · v1
Abstract
Image captioning is a fundamental task that bridges the visual and linguistic domains, playing a critical role in pre-training Large Vision-Language Models (LVLMs). Current state-of-the-art captioning models are typically trained with Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable data annotated by humans or proprietary models. This approach often leads to models that memorize specific ground-truth answers, limiting their generality and ability to generate diverse, creative descriptions. To overcome the limitation of SFT, we propose applying the Reinforcement Learning with Verifiable Rewards (RLVR) paradigm to the open-ended task of image captioning. A primary challenge, however, is designing an objective reward function for the inherently subjective nature of what constitutes a "good" caption. We introduce Captioning Reinforcement Learning (CapRL), a novel training framework that redefines caption quality through its utility: a high-quality caption should enable a non-visual language model to accurately answer questions about the corresponding image. CapRL employs a decoupled two-stage pipeline where an LVLM generates a caption, and the objective reward is derived from the accuracy of a separate, vision-free LLM answering Multiple-Choice Questions based solely on that caption. As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings. Pretraining on the CapRL-5M caption dataset annotated by CapRL-3B results in substantial gains across 12 benchmarks. Moreover, within the Prism Framework for caption quality evaluation, CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%. Code is available here: https://github.com/InternLM/CapRL.
Author abstract · arXiv abstract · v1
Publication
International Conference on Learning Representations (ICLR), 2026
Paper and resources
Research topics
CapRL · Dense captioning · Verifiable rewards · Reinforcement learning · Caption utility · Vision-free evaluation · Multimodal pretraining · CapRL-5M
Research problem and approach
Open-ended caption quality is hard to score objectively, while supervised captioning depends on costly annotations. CapRL evaluates captions through the accuracy of a separate text-only model answering image-related multiple-choice questions. arXiv abstract · v1
Main contributions
- Decouples visual caption generation from vision-free reward evaluation. arXiv abstract · v1
- Releases a captioning model and the CapRL-5M corpus for multimodal pretraining. arXiv abstract · v1
Method comparison
| Approach | Key difference |
|---|---|
| Supervised dense captioning | Depends on reference captions whose collection and quality assessment can be expensive. |
| CapRL | Scores captions by whether a separate text-only model can answer image-related questions from their content. |
Selected results
- For Qwen2.5-3B with Qwen2.5-ViT, CapRL-1M reaches a 12-benchmark average of 59.7 versus 56.7 for ShareGPT4V-1M and 57.1 for DenseFusion-1M. Scaling CapRL to 5M captions raises the average to 62.0. Table 1 · caption pretraining at 1M and 5M scale · arXiv v1
- Recaptioning the same ShareGPT4V-1M images with CapRL raises the 3B model’s average from 56.7 to 58.7; recaptioning DenseFusion-1M raises it from 57.1 to 59.9. This controls image selection when measuring the benefit of CapRL captions. Table 2 · controlled image-source ablation · arXiv v1
Cite this paper
Long Xing, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jianze Liang, Qidong Huang, Jiaqi Wang, Feng Wu, Dahua Lin. CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning. International Conference on Learning Representations, 2026, 2026, pp. 13066–13093.
@inproceedings{arxiv250922647,
title = {{CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning}},
author = {Long Xing and Xiaoyi Dong and Yuhang Zang and Yuhang Cao and Jianze Liang and Qidong Huang and Jiaqi Wang and Feng Wu and Dahua Lin},
booktitle = {International Conference on Learning Representations},
year = {2026},
volume = {2026},
pages = {13066--13093},
url = {https://proceedings.iclr.cc/paper_files/paper/2026/hash/15f1dbc086bfd94d8c32557b573cbe18-Abstract-Conference.html}
}