ICCV · 2021
FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation
Key takeaway
FASA addresses rare-class data scarcity by generating virtual features from observed class statistics and adapting their sampling to the model's training losses. arXiv abstract · v2
Abstract
Recent methods for long-tailed instance segmentation still struggle on rare object classes with few training data. We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes. Both the Feature Augmentation (FA) and feature sampling components are adaptive to the actual training status -- FA is informed by the feature mean and variance of observed real samples from past iterations, and we sample the generated virtual features in a loss-adapted manner to avoid over-fitting. FASA does not require any elaborate loss design, and removes the need for inter-class transfer learning that often involves large cost and manually-defined head/tail class groups. We show FASA is a fast, generic method that can be easily plugged into standard or long-tailed segmentation frameworks, with consistent performance gains and little added cost. FASA is also applicable to other tasks like long-tailed classification with state-of-the-art performance.
Author abstract · arXiv abstract · v2
Publication
IEEE International Conference on Computer Vision (ICCV), 2021
Paper and resources
Research topics
FASA · Long-tailed instance segmentation · Feature augmentation · Adaptive sampling · Rare-class recognition · Virtual features · Class imbalance · Long-tailed classification
Research problem and approach
Long-tailed instance segmentation lacks enough examples for rare categories. FASA augments feature space using running means and variances, then adjusts virtual-feature sampling to avoid overfitting without manual head/tail partitions or an elaborate new loss. arXiv abstract · v2
Main contributions
- Combines adaptive feature augmentation with loss-adapted sampling of virtual examples. arXiv abstract · v2
- Provides a generic module for standard and long-tailed segmentation frameworks and extends it to classification. arXiv abstract · v2
Method comparison
| Approach | Key difference |
|---|---|
| Inter-class transfer for rare categories | Can require costly transfer procedures and manually defined head/tail class groups. |
| FASA | Generates features from observed class statistics and samples them adaptively using training losses. |
Selected results
- FASA raises Mask R-CNN mask AP from 20.8 to 23.7 and rare-class AP from 8.0 to 17.8 on LVIS v1.0 validation. InstaBoost reaches 21.4 overall AP and 10.3 rare-class AP in the same comparison. Table 2 · LVIS v1.0 augmentation comparison · arXiv v2
- Feature augmentation alone gives 22.3 overall AP and 12.7 rare-class AP; adding adaptive feature sampling raises these to 23.7 and 17.8. The additional gain is concentrated in rare classes. Table 1 · adaptive-sampling ablation · arXiv v2
Cite this paper
Yuhang Zang, Chen Huang, Chen Change Loy. FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 3457–3466.
@inproceedings{arxiv210212867,
title = {{FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation}},
author = {Yuhang Zang and Chen Huang and Chen Change Loy},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {3457--3466},
url = {https://openaccess.thecvf.com/content/ICCV2021/html/Zang_FASA_Feature_Augmentation_and_Sampling_Adaptation_for_Long-Tailed_Instance_Segmentation_ICCV_2021_paper.html}
}