Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings

Abstract

Despite the strong performance of ColPali/ColQwen2 in Visualized Document Retrieval (VDR), it encodes each page into multiple patch-level embeddings and leads to excessive memory usage. This empirical study investigates methods to reduce patch embeddings per page at minimum performance degradation. We evaluate two token-reduction strategies: token pruning and token merging. Regarding token pruning, we surprisingly observe that a simple random strategy outperforms other sophisticated pruning methods, though still far from satisfactory. Further analysis reveals that pruning is inherently unsuitable for VDR as it requires removing certain page embeddings without query-specific information. Turning to token merging (more suitable for VDR), we search for the optimal combinations of merging strategy across three dimensions and develop Light-ColPali/ColQwen2. It maintains 98.2% of retrieval performance with only 11.8% of original memory usage, and preserves 94.6% effectiveness at 2.8% memory footprint. We expect our empirical findings and resulting Light-ColPali/ColQwen2 offer valuable insights and establish a competitive baseline for future research towards efficient VDR.

Author abstract · arXiv abstract · v1

Publication

Findings of the Association for Computational Linguistics (ACL), 2025

Paper and resources

• Paper · arXiv v1 (PDF)
• arXiv:2506.04997
• Google Scholar

Research topics

Light-ColPali · Light-ColQwen2 · Visual document retrieval · Multi-vector retrieval · Token merging · Patch embeddings · Storage efficiency · Token pruning

Research problem and approach

Multi-vector document retrieval stores many patch embeddings per page. This empirical study compares pruning and merging, finding that query-independent pruning can discard essential page information and that carefully configured merging offers a better storage trade-off. arXiv abstract · v1

Main contributions

• Analyzes token pruning and token merging for visual document retrieval, including the surprising strength of random pruning among pruning methods. arXiv abstract · v1
• Develops Light-ColPali/ColQwen2 by examining merging choices across three dimensions. arXiv abstract · v1

Method comparison

arXiv abstract · v1

Selected results

• At merging factor 25, Light-ColQwen2 retains 96.3% of ColQwen2’s average NDCG@5 (78.4 versus 81.4). Its relative memory cost is 3.0 versus 64.4, with both normalized to the same DSE-Qwen2 baseline. Table 2 · document retrieval, Qwen2-VL-2B · arXiv v1
• At merging factor 9, post-projector merging achieves an average retrieval score of 81.4, versus 81.0 after the LLM, 65.4 after the vision encoder, and 59.1 before it. Delaying merging preserves retrieval information more effectively in this comparison. Table 1 · merging-location ablation · arXiv v1

Cite this paper

Yubo Ma, Jinsong Li, Yuhang Zang, Xiaobao Wu, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Jiaqi Wang, Yixin Cao, Aixin Sun. Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings. Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 19568–19580.

@inproceedings{arxiv250604997,
  title     = {{Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings}},
  author    = {Yubo Ma and Jinsong Li and Yuhang Zang and Xiaobao Wu and Xiaoyi Dong and Pan Zhang and Yuhang Cao and Haodong Duan and Jiaqi Wang and Yixin Cao and Aixin Sun},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  month     = {July},
  year      = {2025},
  publisher = {Association for Computational Linguistics},
  pages     = {19568--19580},
  doi       = {10.18653/v1/2025.findings-acl.1003},
  url       = {https://aclanthology.org/2025.findings-acl.1003/}
}