SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree

Abstract

The Segment Anything Model 2 (SAM 2) has emerged as a powerful foundation model for object segmentation in both images and videos, paving the way for various downstream video applications. The crucial design of SAM 2 for video segmentation is its memory module, which prompts object-aware memories from previous frames for current frame prediction. However, its greedy-selection memory design suffers from the "error accumulation" problem, where an errored or missed mask will cascade and influence the segmentation of the subsequent frames, which limits the performance of SAM 2 toward complex long-term videos. To this end, we introduce SAM2Long, an improved training-free video object segmentation strategy, which considers the segmentation uncertainty within each frame and chooses the video-level optimal results from multiple segmentation pathways in a constrained tree search manner. In practice, we maintain a fixed number of segmentation pathways throughout the video. For each frame, multiple masks are proposed based on the existing pathways, creating various candidate branches. We then select the same fixed number of branches with higher cumulative scores as the new pathways for the next frame. After processing the final frame, the pathway with the highest cumulative score is chosen as the final segmentation result. Benefiting from its heuristic search design, SAM2Long is robust toward occlusions and object reappearances, and can effectively segment and track objects for complex long-term videos. Notably, SAM2Long achieves an average improvement of 3.0 points across all 24 head-to-head comparisons, with gains of up to 5.3 points in J&F on long-term video object segmentation benchmarks such as SA-V and LVOS. The code is released at https://github.com/Mark12Ding/SAM2Long.

Author abstract · arXiv abstract · v3

Publication

IEEE International Conference on Computer Vision (ICCV), 2025

Paper and resources

• Paper · arXiv v3 (PDF)
• arXiv:2410.16268
• Google Scholar
• Code · Mark12Ding/SAM2Long

Research topics

SAM2Long · SAM 2 · Long video segmentation · Memory tree · Constrained tree search · Error accumulation · Occlusion handling · Training-free segmentation

Research problem and approach

A mistaken mask in greedy memory selection can corrupt later frames. SAM2Long maintains a fixed number of candidate paths, branches them at each frame and keeps paths with stronger cumulative scores. arXiv abstract · v3

Main contributions

• Introduces uncertainty-aware pathway selection without retraining SAM 2. arXiv abstract · v3
• Handles occlusion and object reappearance through video-level rather than purely greedy decisions. arXiv abstract · v3

Method comparison

arXiv abstract · v3

Selected results

• On SA-V test, SAM2Long-L improves J&F from the reproduced SAM2-L baseline’s 75.5 to 80.8. In the same evaluation, SA-V validation rises from 76.3 to 80.8. Table 1 · matched SAM2-L comparison · arXiv v3
• Using the official LVOS evaluation code, SAM2.1Long improves LVOS-v1 validation J&F from 80.2 to 83.4 and LVOS-v2 from 84.1 to 85.9. These scores use a different evaluation implementation from Table 1 and should be cited with that distinction. Table 3 · official LVOS evaluation protocol · arXiv v3

Cite this paper

Shuangrui Ding, Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Yuwei Guo, Dahua Lin, Jiaqi Wang. SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025, pp. 13614–13624.

@inproceedings{arxiv241016268,
  title     = {{SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree}},
  author    = {Shuangrui Ding and Rui Qian and Xiaoyi Dong and Pan Zhang and Yuhang Zang and Yuhang Cao and Yuwei Guo and Dahua Lin and Jiaqi Wang},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month     = {October},
  year      = {2025},
  pages     = {13614--13624},
  url       = {https://openaccess.thecvf.com/content/ICCV2025/html/Ding_SAM2Long_Enhancing_SAM_2_for_Long_Video_Segmentation_with_a_ICCV_2025_paper.html}
}