CVPR · 2021
Seesaw Loss for Long-Tailed Instance Segmentation
Key takeaway
Seesaw Loss balances rare-class learning by reducing overwhelming negative gradients while restoring penalties for misclassified examples to control false positives. arXiv abstract · v4
Abstract
Instance segmentation has witnessed a remarkable progress on class-balanced benchmarks. However, they fail to perform as accurately in real-world scenarios, where the category distribution of objects naturally comes with a long tail. Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories. The overwhelming gradients of negative samples on tail classes lead to a biased learning process for classifiers. Consequently, objects of tail categories are more likely to be misclassified as backgrounds or head categories. To tackle this problem, we propose Seesaw Loss to dynamically re-balance gradients of positive and negative samples for each category, with two complementary factors, i.e., mitigation factor and compensation factor. The mitigation factor reduces punishments to tail categories w.r.t. the ratio of cumulative training instances between different categories. Meanwhile, the compensation factor increases the penalty of misclassified instances to avoid false positives of tail categories. We conduct extensive experiments on Seesaw Loss with mainstream frameworks and different data sampling strategies. With a simple end-to-end training pipeline, Seesaw Loss obtains significant gains over Cross-Entropy Loss, and achieves state-of-the-art performance on LVIS dataset without bells and whistles. Code is available at https://github.com/open-mmlab/mmdetection.
Author abstract · arXiv abstract · v4
Publication
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Paper and resources
Research topics
Seesaw Loss · Long-tailed instance segmentation · Gradient rebalancing · Class imbalance · Rare-category detection · Mitigation factor · Compensation factor · Loss function design
Research problem and approach
Frequent categories dominate negative training signals and bias long-tailed classifiers against rare objects. Seesaw Loss dynamically adjusts positive-negative balance with a mitigation factor based on cumulative class counts and a complementary compensation factor. arXiv abstract · v4
Main contributions
- Introduces complementary mitigation and compensation terms for class-dependent gradient rebalancing. arXiv abstract · v4
- Integrates with end-to-end instance segmentation training across mainstream frameworks and sampling strategies. arXiv abstract · v4
Method comparison
| Approach | Key difference |
|---|---|
| Cross-entropy on long-tailed instances | Frequent-category negatives can overwhelm the learning signal for rare classes. |
| Seesaw Loss | Reduces excessive rare-class penalties while compensating for misclassified examples to limit false positives. |
Selected results
- On LVIS-v1 validation with a 2× schedule and repeat-factor sampling, Seesaw Loss raises mask AP from cross-entropy’s 25.5 to 27.6 and rare-class AP from 16.6 to 20.6. Adding normalized mask prediction reaches 28.1 overall mask AP. Table 1 · Mask R-CNN R101-FPN with RFS · arXiv v4
- With Mask R-CNN and RFS, mitigation alone gives 25.1 mask AP, compensation alone 24.1, and both 25.7, versus 23.7 for the baseline. Adding normalized linear activation reaches 26.4 AP and 19.6 rare-class AP. Table 2 · R50-FPN loss-component ablation · arXiv v4
Cite this paper
Jiaqi Wang, Wenwei Zhang, Yuhang Zang, Yuhang Cao, Jiangmiao Pang, Tao Gong, Kai Chen, Ziwei Liu, Chen Change Loy, Dahua Lin. Seesaw Loss for Long-Tailed Instance Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 9695–9704.
@inproceedings{arxiv200810032,
title = {{Seesaw Loss for Long-Tailed Instance Segmentation}},
author = {Jiaqi Wang and Wenwei Zhang and Yuhang Zang and Yuhang Cao and Jiangmiao Pang and Tao Gong and Kai Chen and Ziwei Liu and Chen Change Loy and Dahua Lin},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2021},
pages = {9695--9704},
url = {https://openaccess.thecvf.com/content/CVPR2021/html/Wang_Seesaw_Loss_for_Long-Tailed_Instance_Segmentation_CVPR_2021_paper.html}
}