NeurIPS · 2026
Visual-ERM: Reward Modeling for Visual Equivalence
Key takeaway
Visual-ERM scores vision-to-code outputs by comparing rendered images with the reference, and using it as the RL reward raises Qwen3-VL-8B-Instruct from 69.6 to 78.0 on ChartMimic, above the 76.1 reached with a DINO-based reward. Table 1 · arXiv v2
Abstract
Vision-to-code tasks require models to reconstruct structured visual inputs, such as charts, tables, and SVGs, into executable or structured representations with high visual fidelity. While recent Large Vision Language Models (LVLMs) achieve strong results via supervised fine-tuning, reinforcement learning remains challenging due to misaligned reward signals. Existing rewards either rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking. We propose Visual Equivalence Reward Model (Visual-ERM), a multimodal generative reward model that provides fine-grained, interpretable, and task-agnostic feedback to evaluate vision-to-code quality directly in the rendered visual space. Integrated into RL, Visual-ERM improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code and yields consistent gains on table and SVG parsing (+2.7, +4.1 on average), and further strengthens test-time scaling via reflection and revision. We also introduce VisualCritic-RewardBench (VC-RewardBench), a benchmark for judging fine-grained image-to-image discrepancies on structured visual data, where Visual-ERM at 8B decisively outperforms Qwen3-VL-235B-Instruct and approaches leading closed-source models. Our results suggest that fine-grained visual reward supervision is both necessary and sufficient for vision-to-code RL, regardless of task specificity.
Author abstract · arXiv abstract · v2
Publication
Neural Information Processing Systems (NeurIPS), 2026
Paper and resources
Research topics
Visual-ERM · VC-RewardBench · Multimodal reward models · Generative reward models · Vision-to-code · Chart-to-code · Table parsing · SVG generation · GRPO · Test-time scaling
Research problem and approach
Rewards for vision-to-code RL either compare text with rules or compare coarse visual embeddings, missing small visual errors and inviting reward hacking. Visual-ERM is an 8B generative reward model that lists localized discrepancies with severities between the reference and the rendered output; the severities become a scalar reward for GRPO, and the descriptions serve as feedback for test-time revision. Section 3 · arXiv v2
Main contributions
- Builds discrepancy-annotated image pairs from targeted edits and natural model predictions, then fine-tunes Qwen3-VL-8B-Instruct into one reward model for charts, tables and SVGs. Section 3.1 · arXiv v2
- Converts summed discrepancy severities into a bounded reward, combined with a render-success term, for GRPO training. Section 3.2 · arXiv v2
- Reuses the discrepancy descriptions as feedback for iterative reflection and revision at inference time. Section 3.3 · arXiv v2
- Introduces VC-RewardBench, 1,335 expert-consolidated instances for judging fine-grained image-to-image discrepancies in structured visuals. Section 3.4 · arXiv v2
Method comparison
| Reward | What it compares | Observed behavior |
|---|---|---|
| Rule-based (TEDS) | Predicted and reference table structure as text | Training reward rises while target metrics barely improve, consistent with reward shortcutting |
| DINO feature similarity | Global visual embeddings of rendered images | Under-penalizes small deviations and rendered-text errors; degrades table parsing |
| Visual-ERM | Localized visual and textual discrepancies in the rendered output, with severities | Improves chart, table and SVG policies and gives feedback usable for revision |
Selected results
| Setting / metric | Before | With Visual-ERM | Gain | Source |
|---|---|---|---|---|
| Chart-to-code · Qwen3-VL-8B-Instruct + RL ChartMimic average | 69.6 | 78.0 | +8.4 | Table 1 |
| Chart-to-code · VinciCoder-8B-SFT + RL ChartMimic average | 67.4 | 77.5 | +10.1 | Table 1 |
| Table-to-Markdown · Qwen3-VL-8B-Instruct + RL Average over OmniDocBench and olmOCRBench | 76.8 | 79.5 | +2.7 | Table 2 |
| SVG-to-code · VinciCoder-8B-SFT + RL UniSVG Score | 87.9 | 91.6 | +3.7 | Table 3 |
| Reward judging · VC-RewardBench Average F1 (strict) | 5.3 | 42.1 | +36.8 | Table 5 |
ChartMimic and table averages are the paper's aggregate scores; the UniSVG value is the aggregate Score. VC-RewardBench reports average strict-match F1 over chart, table and SVG, comparing Visual-ERM with the Qwen3-VL-8B-Instruct model it was trained from. Gains are absolute points.
Cite this paper
Ziyu Liu, Shengyuan Ding, Xinyu Fang, Xuanlang Dai, Penghui Yang, Jiaqi Wang, Kai Chen, Dahua Lin, Yuhang Zang. Visual-ERM: Reward Modeling for Visual Equivalence. Advances in Neural Information Processing Systems, 2026.
@inproceedings{arxiv260313224,
title = {{Visual-ERM: Reward Modeling for Visual Equivalence}},
author = {Ziyu Liu and Shengyuan Ding and Xinyu Fang and Xuanlang Dai and Penghui Yang and Jiaqi Wang and Kai Chen and Dahua Lin and Yuhang Zang},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
url = {https://arxiv.org/abs/2603.13224}
}