结合数据融合与特征选择的遥感影像尺度多样目标检测
Multiscale object detection in remote sensing image by combining data fusion and feature selection
- 2022年26卷第8期 页码：1662-1673
纸质出版日期： 2022-08-07
DOI： 10.11834/jrs.20221249
扫 描 看 全 文
浏览全部资源
扫码关注微信
摘要
基于深度神经网络模型的遥感影像地物检测取得了巨大成功，很大程度上得益于大规模数据集的支撑。但是，从现有遥感影像数据集本身来看，不同类别地物的数量分布不一致，同类地物对象以不同尺寸大小呈现，是导致地物样本的尺度不均衡问题的直接因素。对此，本文采用数据集内影像加权融合与地物多尺度特征选择的策略来缓解该问题。首先，将数据集内两张影像的像素值进行加权并得到融合后的影像，从而使不同类别地物样本更加均衡且具有较高的背景多样性；其次，通过选择合适尺度的特征图预测相应尺度的目标类别，且允许同一尺度目标在相邻特征图上进行预测，这样使模型能根据目标尺度进行训练；最后，基于目标中心区域的特征图预测目标边界框，预测的边界框更符合目标本身的尺度。通过在两个遥感数据集上分别进行实验，表明训练的模型在对复杂背景下的类别不均衡目标的识别更加准确，能够适应遥感影像下不同尺度目标的识别。
Abstract
Remote sensing image object detection based on depth neural network model has achieved great success largely due to the support of large-scale data sets. However
from the perspective of the existing remote sensing image datasets
the number distribution of different types of ground objects is inconsistent
and the same type of ground objects is presented in different sizes. This factor leads to the scale imbalance of ground object samples.
To alleviate this problem
the strategy of weighted image fusion and multiscale feature selection is adopted. First
the pixel values of the two images in the data set are weighed to obtain the fused image. Therefore
the different types of feature samples are more balanced and have higher background diversity. Second
the target category of the corresponding scale is predicted by selecting the appropriate scale feature map
and the same scale target can be predicted on the adjacent feature map. Thus
the model can be trained according to the target scale. Finally
the bounding box of the target is predicted based on the feature map of the target center area
and the result is more consistent with the scale of the target itself.
The image fusion and multiscale feature selection network is experimentally compared in the paper on two datasets
RSOD and NWPUVHR10. The results show that the trained model is more accurate in the recognition of imbalanced objects in complex background and can adapt to the recognition of different scale objects in remote sensing images. In addition
the proposed method enhances the diversity of sample scenes and can be adapted to different scales of targets. Qualitative analysis shows that the proposed method has better robustness and performance advantages when applied to remote sensing images.
The proposed method can cope with remote sensing images with complex backgrounds
mitigate the effects of category imbalances and better fit the scenarios in which remote sensing images are used. Mitigation of category imbalances and Scale Selection enhance the performance of remote sensing image object detection.
关键词
图像融合增强多尺度选择与表达高分辨率遥感影像目标检测卷积神经网络
Keywords
image fusion enhancementmulti scale selection and expressionhigh resolution remote sensing imageobject detectionconvolutional neural network
references
Cai Z W and Vasconcelos N. 2018. Cascade R-CNN: Delving into high quality object detection//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 6154-6162 [DOI: 10.1109/CVPR.2018.00644http://dx.doi.org/10.1109/CVPR.2018.00644]
Cao Y H, Chen K, Loy C C and Lin D. 2020. Prime sample attention in object detection//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA: IEEE: 11580-11588 [DOI: 10.1109/CVPR42600.2020.01160http://dx.doi.org/10.1109/CVPR42600.2020.01160]
Chen J, Wan L, Zhu J R, Xu G and Deng M. 2020. Multi-scale spatial and channel-wise attention for improving object detection in remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 17(4): 681-685 [DOI: 10.1109/LGRS.2019.2930462http://dx.doi.org/10.1109/LGRS.2019.2930462]
Cheng G and Han J W. 2016a. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11-28 [DOI: 10.1016/j.isprsjprs.2016.03.014http://dx.doi.org/10.1016/j.isprsjprs.2016.03.014]
Cheng G, Han J W, Zhou P C and Guo L. 2014. Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing, 98: 119-132 [DOI: 10.1016/j.isprsjprs.2014.10.002http://dx.doi.org/10.1016/j.isprsjprs.2014.10.002]
Cheng G, Zhou P C and Han J W. 2016b. Learning Rotation-Invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 54(12): 7405-7415 [DOI: 10.1109/TGRS.2016.2601622http://dx.doi.org/10.1109/TGRS.2016.2601622]
Durieux L, Lagabrielle E and Nelson A. 2008. A method for monitoring building construction in urban sprawl areas using object-based analysis of Spot 5 images and existing GIS data. ISPRS Journal of Photogrammetry and Remote Sensing, 63(4): 399-408 [DOI: 10.1016/j.isprsjprs.2008.01.005http://dx.doi.org/10.1016/j.isprsjprs.2008.01.005]
Feng X, Qiu K, Cui W H, Chen Y X and Li X H. 2014. Multiscale description and recognition of target shape in high-resolution remote sensing images. Journal of Remote Sensing, 18(1): 90-104
冯霞, 秦昆, 崔卫红, 陈一祥, 李向辉. 2014. 高分辨率遥感影像目标形状特征多尺度描述与识别. 遥感学报, 18(1): 90-104 [DOI: 10.11834/jrs.20133056http://dx.doi.org/10.11834/jrs.20133056]
Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH: IEEE: 580-587 [DOI: 10.1109/CVPR.2014.81http://dx.doi.org/10.1109/CVPR.2014.81]
Girshick R. 2015. Fast R-CNN//2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE: 1440-1448 [DOI: 10.1109/ICCV.2015.169http://dx.doi.org/10.1109/ICCV.2015.169]
Jiang Y F, Yu H Y, Li C L, Liu P and Zhang H Y. 2020. Oil storage tank detection and information extraction from high-resolution remote sensing imagery based on improved Faster R-CNN. Remote Sensing Information, 35(4): 89-96
江一帆, 于海洋, 李朝亮, 刘鹏, 张慧勇. 2020. 基于改进faster R-CNN的高分遥感影像储油罐检测与信息提取. 遥感信息, 35(4): 89-96 [DOI: 10.3969/j.issn.1000-3177.2020.04.013http://dx.doi.org/10.3969/j.issn.1000-3177.2020.04.013]
Kong T, Sun F C, Liu H P, Jiang Y N, Li L and Shi J B. 2020. FoveaBox: Beyound anchor-based object detection. IEEE Transactions on Image Processing, 29: 7389-7398 [DOI: 10.1109/TIP.2020.3002345http://dx.doi.org/10.1109/TIP.2020.3002345]
Li B Y, Liu Y and Wang X G. 2019. Gradient harmonized single-stage detector. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1): 8577-8584 [DOI: 10.1609/aaai.v33i01.33018577http://dx.doi.org/10.1609/aaai.v33i01.33018577]
Li C Y, Liu J, Hong H L, Mao W J, Wang C J, Hu C D, Su X and Luo B. 2020. Object detection based on OcSaFPN in aerial images with noise. arXiv:2012.09859
Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017b. Feature pyramid networks for object detection//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE: 936-944 [DOI: 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106]
Lin T Y, Goyal P, Girshick R, He K M and Dollár P. 2017a. Focal loss for dense object detection//2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE: 2999-3007 [DOI: 10.1109/ICCV.2017.324http://dx.doi.org/10.1109/ICCV.2017.324]
Liu S, Qi L, Qin H F, Shi J P and Jia J Y. 2018. Path aggregation network for instance segmentation//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 8759-8768 [DOI: 10.1109/CVPR.2018.00913http://dx.doi.org/10.1109/CVPR.2018.00913]
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//14th European Conference on Computer Vision. Amsterdam: Springer: 21-37 [DOI: 10.1007/978-3-319-46448-0_2http://dx.doi.org/10.1007/978-3-319-46448-0_2]
Long Y, Gong Y P, Xiao Z F and Liu Q. 2017. Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5): 2486-2498 [DOI: 10.1109/TGRS.2016.2645610http://dx.doi.org/10.1109/TGRS.2016.2645610]
Pang J M, Chen K, Shi J P, Feng H J, Ouyang W L and Lin D H. 2019. Libra R-CNN: towards balanced learning for object detection//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE: 821-830 [DOI: 10.1109/CVPR.2019.00091http://dx.doi.org/10.1109/CVPR.2019.00091]
Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE: 779-788 [DOI: 10.1109/CVPR.2016.91http://dx.doi.org/10.1109/CVPR.2016.91]
Redmon J and Farhadi A. 2017. YOLO9000: better, faster, stronger//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE: 6517-6525 [DOI: 10.1109/CVPR.2017.690http://dx.doi.org/10.1109/CVPR.2017.690]
Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement. arXiv:1804.02767
Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149 [DOI: 10.1109/TPAMI.2016.2577031http://dx.doi.org/10.1109/TPAMI.2016.2577031]
Shrivastava A, Gupta A and Girshick R. 2016. Training region-based object detectors with online hard example mining//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE: 761-769 [DOI: 10.1109/CVPR.2016.89http://dx.doi.org/10.1109/CVPR.2016.89]
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for Large-Scale image recognition. arXiv:1409.1556
Singh B and Davis L S. 2018. An analysis of scale invariance in object detection - SNIP//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 3578-3587 [DOI: 10.1109/CVPR.2018.00377http://dx.doi.org/10.1109/CVPR.2018.00377]
Wang B, Zhou Y, Zhang H N and Zhao K. 2021. Aircraft detection method based on SSD framework for remote sensing images. Fire Control and Command Control, 46(1): 14-19
王冰, 周焰, 张怀念, 赵凯. 2021. 基于改进SSD框架的遥感影像飞机目标检测方法. 火力与指挥控制, 46(1): 14-19 [DOI: 10.3969/j.issn.1002-0640.2021.01.003http://dx.doi.org/10.3969/j.issn.1002-0640.2021.01.003]
Xiao Z F, Liu Q, Tang G F and Zhai X F. 2015. Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images. International Journal of Remote Sensing, 36(2): 618-644 [DOI: 10.1080/01431161.2014.999881http://dx.doi.org/10.1080/01431161.2014.999881]
Yang Y, Li W L, Gao S Y, Bai H and Jiang W C. 2021. Objects detection from high-resolution remote sensing imagery using training-optimized YOLOv3 network. Laser and Optoelectronics Progress, 58(16): 147-153
杨耘, 李龙威, 高思岩, 柏晗, 江万成. 2021. 基于YOLOv3网络训练优化的高分辨率遥感影像目标检测. 激光与光电子学进展, 58(16): 147-153 [DOI: 10.3788/lop202158.1601002http://dx.doi.org/10.3788/lop202158.1601002]
Yao H G, Wang C, Yu J, Bai X J and Li W. 2020. Recognition of small-target ships in complex satellite images. Journal of Remote Sensing, 24(2): 116-125
姚红革, 王诚, 喻钧, 白小军, 李蔚. 2020. 复杂卫星图像中的小目标船舶识别. 遥感学报, 24(2): 116-125 [DOI: 10.11834/jrs.20208238http://dx.doi.org/10.11834/jrs.20208238]
Yu Y, Ai H, He X J, Yu S H, Zhong X and Zhu R F. 2020. Attention-based feature pyramid networks for ship detection of optical remote sensing image. Journal of Remote Sensing, 24(2): 107-115
于野, 艾华, 贺小军, 于树海, 钟兴, 朱瑞飞. 2020. A-FPN算法及其在遥感图像船舶检测中的应用. 遥感学报, 24(2): 107-115 [DOI: 10.11834/jrs.20208264http://dx.doi.org/10.11834/jrs.20208264]
Zhong P and Wang R S. 2007. A multiple conditional random fields ensemble model for urban area detection in remote sensing optical images. IEEE Transactions on Geoscience and Remote Sensing, 45(12): 3978-3988 [DOI: 10.1109/TGRS.2007.907109http://dx.doi.org/10.1109/TGRS.2007.907109]
文章被引用时，请邮件提醒。
提交
相关作者
暂无数据
相关机构
国家林业和草原局林业智能信息处理工程技术研究中心
北京林业大学 信息学院
武汉大学 测绘遥感信息工程国家重点实验室
北京师范大学 遥感科学国家重点实验室
南京信息工程大学 遥感与测绘工程学院
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部