ICLR · 2026
Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing
Key takeaway
ChartVSR refines pixel-level data-point locations through visual feedback before decoding chart values, giving chart parsing an explicit visual self-correction process. arXiv abstract · v1
Abstract
While Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities for reasoning and self-correction at the textual level, these strengths provide minimal benefits for complex tasks centered on visual perception, such as Chart Parsing. Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination. Inspired by the human strategy of using a finger as a ``visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR). The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors. We instantiate the VSR paradigm in the domain of Chart Parsing by proposing ChartVSR. This model decomposes the parsing process into two stages: a Refine Stage, where it iteratively uses visual feedback to ensure the accuracy of all data points' Pixel-level Localizations, and a Decode Stage, where it uses these verified localizations as precise visual anchors to parse the final structured data. To address the limitations of existing benchmarks, we also construct ChartP-Bench, a new and highly challenging benchmark for chart parsing. Our work also highlights VSR as a general-purpose visual feedback mechanism, offering a promising new direction for enhancing accuracy on a wide range of vision-centric tasks.
Author abstract · arXiv abstract · v1
Publication
International Conference on Learning Representations (ICLR), 2026
Paper and resources
Research topics
ChartVSR · Visual Self-Refine · Chart parsing · Pixel-level localization · Visual feedback · Perceptual self-correction · ChartP-Bench · Structured data extraction
Research problem and approach
Textual self-correction is insufficient for dense charts with omitted or misaligned data points. Visual Self-Refine renders localization predictions back to the model so that it can inspect and correct perceptual errors. arXiv abstract · v1
Main contributions
- Separates iterative visual localization refinement from final structured-data decoding. arXiv abstract · v1
- Introduces ChartP-Bench to evaluate difficult chart-parsing cases. arXiv abstract · v1
Method comparison
| Approach | Key difference |
|---|---|
| Textual self-correction for chart parsing | Revises text predictions without explicitly checking their pixel-level alignment to chart marks. |
| ChartVSR | Renders point-localization predictions for visual inspection and refinement before decoding structured values. |
Selected results
- ChartVSR-3B achieves an overall average SCRM AP of 38.41, compared with 34.07 for Gemini-2.5-Pro. The score averages the benchmark’s Strict, Slight and High matching criteria. Table 3 · ChartP-Bench evaluation · arXiv v1
- Removing visual self-refinement lowers overall average SCRM AP from 38.41 to 36.54. On the Hard subset, which contains charts with more than 18 data points, the score falls from 37.66 to 35.14. Table 4 · visual-refinement ablation · arXiv v1
Cite this paper
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin. Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing. International Conference on Learning Representations, 2026, 2026, pp. 85499–85522.
@inproceedings{arxiv260216455,
title = {{Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing}},
author = {Jinsong Li and Xiaoyi Dong and Yuhang Zang and Yuhang Cao and Jiaqi Wang and Dahua Lin},
booktitle = {International Conference on Learning Representations},
year = {2026},
volume = {2026},
pages = {85499--85522},
url = {https://proceedings.iclr.cc/paper_files/paper/2026/hash/89b0e466b46292ce0bfe53618aadd3de-Abstract-Conference.html}
}