ICCV · 2025
Visual-RFT: Visual Reinforcement Fine-Tuning
Key takeaway
Visual-RFT shows that GRPO with task-specific verifiable rewards improves classification, detection, and grounding in large vision-language models trained with limited labeled data. Sections 3.2 and 4; Tables 2–5
Abstract
Reinforcement Fine-Tuning (RFT) in Large Reasoning Models like OpenAI o1 learns from feedback on its answers, which is especially useful in applications when fine-tuning data is scarce. Recent open-source work like DeepSeek-R1 demonstrates that reinforcement learning with verifiable reward is possibly one key direction in reproducing o1. While the R1-style model has demonstrated success in language models, its application in multi-modal domains remains under-explored. This work introduces Visual Reinforcement Fine-Tuning (Visual-RFT), which further extends the application areas of RFT on visual tasks. Specifically, Visual-RFT first uses Large Vision-Language Models (LVLMs) to generate multiple responses containing reasoning tokens and final answers for each input, and then uses our proposed visual perception verifiable reward functions to update the model via the policy optimization algorithm such as Group Relative Policy Optimization (GRPO). We design different verifiable reward functions for different perception tasks, such as the Intersection over Union (IoU) reward for object detection. Experimental results on fine-grained image classification, few-shot object detection, reasoning grounding, as well as open-vocabulary object detection benchmarks show the competitive performance and advanced generalization ability of Visual-RFT compared with Supervised Fine-tuning (SFT). For example, Visual-RFT improves accuracy by 24.3% over the baseline in one-shot fine-grained image classification with around 100 samples. In few-shot object detection, Visual-RFT also exceeds the baseline by 21.0 on COCO’s 4-shot setting and 15.4 on LVIS. Our Visual-RFT represents a paradigm shift in fine-tuning LVLMs, offering a data-efficient, reward-driven approach that enhances reasoning and adaptability for domain-specific tasks.
Author abstract · ICCV 2025 proceedings record
Publication
IEEE International Conference on Computer Vision (ICCV), 2025
Paper and resources
Research topics
Visual Reinforcement Fine-Tuning (Visual-RFT) · Reinforcement Learning with Verifiable Rewards (RLVR) · Group Relative Policy Optimization (GRPO) · Large Vision-Language Models (LVLMs) · Multimodal reinforcement learning · Few-shot learning · Fine-grained image classification · Few-shot object detection · Open-vocabulary object detection · Reasoning grounding
Research problem and approach
Visual-RFT studies how to adapt large vision-language models to visual perception tasks with limited labeled examples. It trains with GRPO and task-specific, rule-based rewards, using class correctness for classification and box overlap, confidence, and output format for detection. The experiments compare reinforcement fine-tuning with supervised fine-tuning on classification, detection, and grounding. Sections 1 and 3
Main contributions
- Applies reinforcement learning with verifiable rewards to visual perception, including fine-grained classification, few-shot detection, reasoning grounding, and open-vocabulary detection. Section 1
- Defines task-specific rewards from labels and bounding boxes, allowing the model to learn from multiple sampled responses without a separately trained preference reward model. Section 3.2
- Evaluates learning from limited examples and transfer to novel categories, comparing Visual-RFT with the base model and supervised fine-tuning. Sections 4.2–4.5
Method comparison
| Approach | Learning signal | Optimization | Training data |
|---|---|---|---|
| Supervised fine-tuning (SFT) | Target outputs supplied with each training example. | Increases the likelihood of the demonstrated answers. | Labeled image–instruction–answer examples. |
| RLHF with a learned reward model | Scores predicted by a reward model fitted to human preferences. | Updates the policy to increase the learned reward. | Preference comparisons for reward-model training, plus inputs for policy optimization. |
| Visual-RFT (RLVR + GRPO) | Rules evaluate class correctness, box overlap, confidence, and output format. | Samples multiple responses and uses their relative rewards for GRPO updates. | Task examples with ground-truth labels or boxes; evaluated with limited data. |
Selected results
| Setting / metric | Base model | SFT | Visual-RFT | Gain | Source |
|---|---|---|---|---|---|
| Fine-grained classification · 1-shot per class; average over Flower102, Pets37, Aircraft, and Cars196 Accuracy (%) | 56.0 | 51.7 | 80.3 | +24.3 | Table 2, p. 2039 |
| Few-shot detection · COCO, 8 selected categories, 4-shot per category mAP | 19.6 | 25.2 | 40.6 | +21.0 | Table 3, p. 2039 |
| Few-shot detection · LVIS, 6 selected rare categories, approximately 10-shot (1–10 images per category) mAP | 4.0 | 10.0 | 19.4 | +15.4 | Table 4, p. 2039; Section 4.3 |
Accuracy gains are percentage points; mAP gains are mAP points.
Related work and discussion
Selected references and public discussion. Dates indicate first arXiv submission for papers and posting date for commentary.
Cite this paper
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, Jiaqi Wang. Visual-RFT: Visual Reinforcement Fine-Tuning. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025, pp. 2034–2044.
@inproceedings{Liu_2025_ICCV,
title = {{Visual-RFT: Visual Reinforcement Fine-Tuning}},
author = {Ziyu Liu and Zeyi Sun and Yuhang Zang and Xiaoyi Dong and Yuhang Cao and Haodong Duan and Dahua Lin and Jiaqi Wang},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2025},
pages = {2034--2044},
url = {https://openaccess.thecvf.com/content/ICCV2025/html/Liu_Visual-RFT_Visual_Reinforcement_Fine-Tuning_ICCV_2025_paper.html}
}