ICCV · 2019
Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network
Key takeaway
Pixel Aggregation Network detects arbitrary-shaped scene text efficiently by combining a lightweight segmentation head with learned pixel grouping. arXiv abstract · v2
Abstract
Scene text detection, an important step of scene text reading systems, has witnessed rapid development with convolutional neural networks. Nonetheless, two main challenges still exist and hamper its deployment to real-world applications. The first problem is the trade-off between speed and accuracy. The second one is to model the arbitrary-shaped text instance. Recently, some methods have been proposed to tackle arbitrary-shaped text detection, but they rarely take the speed of the entire pipeline into consideration, which may fall short in practical applications. In this paper, we propose an efficient and accurate arbitrary-shaped text detector, termed Pixel Aggregation Network (PAN), which is equipped with a low computational-cost segmentation head and a learnable post-processing. More specifically, the segmentation head is made up of Feature Pyramid Enhancement Module (FPEM) and Feature Fusion Module (FFM). FPEM is a cascadable U-shaped module, which can introduce multi-level information to guide the better segmentation. FFM can gather the features given by the FPEMs of different depths into a final feature for segmentation. The learnable post-processing is implemented by Pixel Aggregation (PA), which can precisely aggregate text pixels by predicted similarity vectors. Experiments on several standard benchmarks validate the superiority of the proposed PAN. It is worth noting that our method can achieve a competitive F-measure of 79.9% at 84.2 FPS on CTW1500.
Author abstract · arXiv abstract · v2
Publication
IEEE International Conference on Computer Vision (ICCV), 2019
Paper and resources
Research topics
Pixel Aggregation Network · PAN · Arbitrary-shaped text detection · Scene text detection · Pixel aggregation · Feature pyramid enhancement · Real-time OCR · Text instance segmentation
Research problem and approach
Scene text systems must recover irregular text shapes while keeping the full pipeline fast. PAN strengthens and fuses pyramid features, then groups text pixels through predicted similarity vectors rather than relying only on a costly segmentation backbone. arXiv abstract · v2
Main contributions
- Introduces Feature Pyramid Enhancement and Feature Fusion modules for efficient text segmentation. arXiv abstract · v2
- Uses learnable Pixel Aggregation to assemble text instances of arbitrary shape. arXiv abstract · v2
Method comparison
| Approach | Key difference |
|---|---|
| Arbitrary-shape text detection without pipeline speed emphasis | Can recover text contours while leaving the end-to-end speed-accuracy trade-off unresolved. |
| PAN | Combines efficient pyramid feature processing with learned pixel aggregation to group arbitrary-shaped text. |
Selected results
- With external training data, PAN-320 achieves 79.9% F-measure at 84.2 fps, while PAN-640 reaches 83.7% at 39.8 fps. Without external data, these F-measures are 77.1% and 81.0%, respectively. Table 4 · CTW1500, single-scale detection · arXiv v2
- With ResNet18 and feature fusion fixed, enabling pixel aggregation raises CTW1500 F-measure from 79.8% to 81.0%, with throughput changing from 39.9 to 39.8 fps. On ICDAR2015, F-measure rises from 79.3% to 80.3%. Table 3 · pixel-aggregation ablation · arXiv v2
Cite this paper
Wenhai Wang, Enze Xie, Xiaoge Song, Yuhang Zang, Wenjia Wang, Tong Lu, Gang Yu, Chunhua Shen. Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 8440–8449.
@inproceedings{arxiv190805900,
title = {{Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network}},
author = {Wenhai Wang and Enze Xie and Xiaoge Song and Yuhang Zang and Wenjia Wang and Tong Lu and Gang Yu and Chunhua Shen},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019},
pages = {8440--8449},
url = {https://openaccess.thecvf.com/content_ICCV_2019/html/Wang_Efficient_and_Accurate_Arbitrary-Shaped_Text_Detection_With_Pixel_Aggregation_Network_ICCV_2019_paper.html}
}