基于多方向自适应感知网络的高光谱遥感图像分类
Hyperspectral remote sensing image classification based on multidirectional adaptive aware network
- 2024年28卷第1期 页码：168-186
纸质出版日期： 2024-01-07
DOI： 10.11834/jrs.20243292
移动端阅览
浏览全部资源
扫码关注微信
摘要
高光谱遥感能够同步获取目标场景的光谱数据和空间图像，满足对目标物体成分组成和形貌特征的探测需求，已被广泛应用于精准农业、地质勘察、环境监测和生物医学等领域。近几年，得益于优秀的空谱信息表征能力，基于卷积神经网络的高光谱图像分类方法取得了优于传统方法的分类精度。然而，基于一定规则和固定方形窗口的卷积运算无法满足不同对象、不同分布的空谱特征提取需求，当像素位于类别边缘时该模式无法避免其他类别的无关信息，导致类别间的信息扩散以及跨类别像素的误分和错分。针对上述问题，本文提出了一种基于多方向自适应感知的高光谱图像空谱联合分类方法，并通过联合不同邻域范围的空谱上下文信息来改善模型的局部空谱建模能力。具体研究过程如下：首先，将常规卷积网络的全窗口滤波器拆分为不同方向的半窗滤波器，从而设计出侧窗卷积以捕捉具有方向性的空谱特征；然后，在此基础上进一步将不同方向的侧窗滤波核整合到统一的卷积架构中构造空谱分离多方向卷积，并设计方向自适应感知模块，以密集连接构建基于高光谱图像的多方向自适应感知分类网络，赋予模型自适应学习多种空谱结构特征的能力，弥补常规卷积和侧窗卷积只能建模单一方向空谱关系的不足，提升模型对复杂空谱结构的刻画能力。在3个公开数据集上的实验结果表明，本文提出的多方向自适应感知分类方法较常规卷积方法具有更优的分类性能，能增强边缘样本的表征准确性，改善分类中的边缘混淆现象。
Abstract
Hyperspectral remote sensing can realize the simultaneous acquisition of spectral data and spatial images for the observation scene to satisfy the detection requirements for the composition and morphology of the target object
which is widely used in the fields of precision agriculture
geological survey
environmental monitoring
biology
and medicine. In recent years
benefiting from the powerful representation capabilities of spectral-spatial information
the HyperSpectral Image (HSI) classification methods based on convolutional neural networks have demonstrated superior classification performances to traditional methods. However
the convolutional operation
performed within fixed square windows following certain rules
encounters difficulties in adapting spectral-spatial feature extraction for different objects and spatial distributions and introduces irrelevant information from other categories
leading to information diffusion between different categories and misclassification of edge pixels. This paper proposes a hyperspectral image classification method based on multidirectional adaptive awareness to address this problem. This method integrates the spatial contextual information of different neighborhood ranges to improve the local spectral-spatial modeling capability of the model. First
the full-window filter of the regular convolution is split into side-window filters with different orientations to design the Side-Window Convolution (SWC) acquiring the directional spectral-spatial features. Therefore
multiple side-window filter kernels of different directions are integrated into a unified convolution architecture to construct the spectral-spatial separable multidirectional convolution (S3MDC). The direction-adaptive aware module (DAAM) is designed to assist S3MDC and build a multidirectional adaptive aware network (MDAAN) with dense connections for HSI classification. MDAAN can adaptively learn multidirectional spatial-spectral features
improve the representation of complex spectral-spatial structures
and compensate for the deficiency that SWC only captures spectral-spatial relationships in a single direction. Experiments were conducted on three public datasets
including Indian Pines
University of Pavia
and Kennedy Space Center. MDAAN achieves the overall highest classification accuracy of 97.67% (Indian pines
5%/class)
99.40% (University of Pavia
1%/class)
and 99.64% (Kennedy Space Center
5%/class)
which is superior to other deep learning methods
verifying the effectiveness of the proposed model. First
compared with other deep learning methods
MDAAN can provide better classification performance. Second
MDAAN generalizes better with the small number of training samples to prove the stability of its classification performance. Finally
the ablation analysis of different convolutional models and DAAM demonstrates the effectiveness and necessity of S3MDC and the adaptive aware mechanisms for multidirectional features.
关键词
遥感高光谱图像深度学习多方向自适应感知空谱结构建模空谱联合分类
Keywords
remote sensinghyperspectral imagedeep learningmulti-directional adaptive awarenessspectral-spatial structure modelingspectral-spatial classification
references
Benediktsson J A, Palmason J A and Sveinsson J R. 2005. Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 480-491 [DOI: 10.1109/TGRS.2004.842478http://dx.doi.org/10.1109/TGRS.2004.842478]
Camps-Valls G, Gomez-Chova L, Munoz-Mari J, Vila-Frances J and Calpe-Maravilla J. 2006. Composite kernels for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 3(1): 93-97 [DOI: 10.1109/LGRS.2005.857031http://dx.doi.org/10.1109/LGRS.2005.857031]
Chen C, Zhang J J, Zheng C H, Yan Q and Xun L N. 2018. Classification of hyperspectral data using a multi-channel convolutional neural network//Proceedings of the 14th International Conference on Intelligent Computing Methodologies. Wuhan: Springer: 81-92 [DOI: 10.1007/978-3-319-95957-3_10http://dx.doi.org/10.1007/978-3-319-95957-3_10]
Chen Y, Nasrabadi N M and Tran T D. 2011. Hyperspectral image classification using dictionary-based sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 3973-3985 [DOI: 10.1109/TGRS.2011.2129595http://dx.doi.org/10.1109/TGRS.2011.2129595]
Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6232-6251 [DOI: 10.1109/TGRS.2016.2584107http://dx.doi.org/10.1109/TGRS.2016.2584107]
Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107 [DOI: 10.1109/JSTARS.2014.2329330http://dx.doi.org/10.1109/JSTARS.2014.2329330]
Chen Y S, Zhao X and Jia X P. 2015. Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2381-2392 [DOI: 10.1109/JSTARS.2015.2388577http://dx.doi.org/10.1109/JSTARS.2015.2388577]
Du P J, Xia J S, Xue Z H, Tan K, Su H J and Bao R. 2016. Review of hyperspectral remote sensing image classification. Journal of Remote Sensing, 20(2): 236-256
杜培军, 夏俊士, 薛朝辉, 谭琨, 苏红军, 鲍蕊. 2016. 高光谱遥感影像分类研究进展. 遥感学报, 20(2): 236-256 [DOI: 10.11834/jrs.20165022http://dx.doi.org/10.11834/jrs.20165022]
Gao H M, Yang Y, Li C M, Gao L R and Zhang B. 2021. Multiscale residual network with mixed depthwise convolution for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(4): 3396-3408 [DOI: 10.1109/TGRS.2020.3008286http://dx.doi.org/10.1109/TGRS.2020.3008286]
Gao L R, Du Q, Zhang B, Yang W and Wu Y F. 2013. A comparative study on linear regression-based noise estimation for hyperspectral imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2): 488-498 [DOI: 10.1109/JSTARS.2012.2227245http://dx.doi.org/10.1109/JSTARS.2012.2227245]
Ghamisi P, Yokoya N, Li J, Liao W Z, Liu S C, Plaza J, Rasti B and Plaza A. 2017. Advances in hyperspectral image and signal processing: a comprehensive overview of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 5(4): 37-78 [DOI: 10.1109/MGRS.2017.2762087http://dx.doi.org/10.1109/MGRS.2017.2762087]
Gong Z Q, Zhong P, Yu Y, Hu W D and Li S T. 2019. A CNN with multiscale convolution and diversified metric for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(6): 3599-3618 [DOI: 10.1109/TGRS.2018.2886022http://dx.doi.org/10.1109/TGRS.2018.2886022]
He K M, Zhang X Y, Ren S Q and Jian S. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE: 2261-2269 [DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]
Hughes G. 1968. On the mean accuracy of statistical pattern recognizers. IEEE Transactions on Information Theory, 14(1): 55-63 [DOI: 10.1109/TIT.1968.1054102http://dx.doi.org/10.1109/TIT.1968.1054102]
Huo L Z and Tang P. 2011. Spectral and spatial classification of hyperspectral data using svms and gabor textures// Proceedings of 2011 IEEE International Geoscience and Remote Sensing Symposium. Vancouver: IEEE: 1708-1711 [DOI: 10.1109/IGARSS.2011.6049564http://dx.doi.org/10.1109/IGARSS.2011.6049564]
Jia S, Hu J, Zhu J S, Jia X P and Li Q Q. 2017. Three-dimensional local binary patterns for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 55(4): 2399-2413 [DOI: 10.1109/TGRS.2016.2642951http://dx.doi.org/10.1109/TGRS.2016.2642951]
Konstantinos K, Christina K, Zacharias K and Georgia A. 2018. HyRANK Hyperspectral Satellite Dataset I (Version v001) [Data set][DB/OL]. (2018-04-20. https://zenodo.org/records/1222202https://zenodo.org/records/1222202 [DOI: 10.5281/zenodo.1222202http://dx.doi.org/10.5281/zenodo.1222202]
Lee H and Kwon H. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 26(10): 4843-4855 [DOI: 10.1109/TIP.2017.2725580http://dx.doi.org/10.1109/TIP.2017.2725580]
Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): 582 [DOI: 10.3390/rs12030582http://dx.doi.org/10.3390/rs12030582]
Li W, Chen C, Su H J and Du Q. 2015. Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3681-3693 [DOI: 10.1109/TGRS.2014.2381602http://dx.doi.org/10.1109/TGRS.2014.2381602]
Li W and Du Q. 2014. Gabor-filtering-based nearest regularized subspace for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(4): 1012-1022 [DOI: 10.1109/JSTARS.2013.2295313http://dx.doi.org/10.1109/JSTARS.2013.2295313]
Liu B, Yu X C, Yu A Z and Wan G. 2018. Deep convolutional recurrent neural network with transfer learning for hyperspectral image classification. Journal of Applied Remote Sensing, 12(2): 026028 [DOI: 10.1117/1.JRS.12.026028http://dx.doi.org/10.1117/1.JRS.12.026028]
Liu J J, Wu Z B, Wei Z H, Xiao L and Sun L. 2013. Spatial-spectral kernel sparse representation for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(6): 2462-2471 [DOI: 10.1109/JSTARS.2013.2252150http://dx.doi.org/10.1109/JSTARS.2013.2252150]
Liu Q, Wu Z B, Xu Y and Wei Z H. 2023. A unified attention paradigm for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 61: 1-16 [DOI: 10.1109/TGRS.2023.3257321http://dx.doi.org/10.1109/TGRS.2023.3257321]
Ma W P, Yang Q F, Wu Y, Zhao W and Zhang X R. 2019. Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sensing, 11(11): 1307 [DOI: 10.3390/rs11111307http://dx.doi.org/10.3390/rs11111307]
Mura M D, Benediktsson J A, Waske B and Bruzzone L. 2010a. Morphological attribute profiles for the analysis of very high resolution images. IEEE Transactions on Geoscience and Remote Sensing, 48(10): 3747-3762 [DOI: 10.1109/TGRS.2010.2048116http://dx.doi.org/10.1109/TGRS.2010.2048116]
Mura M D, Benediktsson J A, Waske B and Bruzzone L. 2010b. Extended profiles with morphological attribute filters for the analysis of hyperspectral data. International Journal of Remote Sensing, 31(22): 5975-5991 [DOI: 10.1080/01431161.2010.512425http://dx.doi.org/10.1080/01431161.2010.512425]
Ojala T, Pietikainen M and Maenpaa T. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7): 971-987 [DOI: 10.1109/TPAMI.2002.1017623http://dx.doi.org/10.1109/TPAMI.2002.1017623]
Paoletti M E, Haut J M, Fernandez-Beltran R, Plaza J, Plaza A J and Pla F. 2019. Deep pyramidal residual networks for spectral-spatial hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 740-754 [DOI: 10.1109/TGRS.2018.2860125http://dx.doi.org/10.1109/TGRS.2018.2860125]
Pesaresi M and Benediktsson J A. 2001. A new approach for the morphological segmentation of high-resolution satellite imagery. IEEE Transactions on Geoscience and Remote Sensing, 39(2): 309-320 [DOI: 10.1109/36.905239http://dx.doi.org/10.1109/36.905239]
Roy S K, Krishna G, Dubey S R and Chaudhuri B B. 2020. HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2): 277-281 [DOI: 10.1109/LGRS.2019.2918719http://dx.doi.org/10.1109/LGRS.2019.2918719]
Sun X X, Qu Q, Nasrabadi N M and Tran T D. 2014. Structured priors for sparse-representation-based hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 11(7): 1235-1239 [DOI: 10.1109/LGRS.2013.2290531http://dx.doi.org/10.1109/LGRS.2013.2290531]
Tarabalka Y, Benediktsson J A and Chanussot J. 2009. Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques. IEEE Transactions on Geoscience and Remote Sensing, 47(8): 2973-2987 [DOI: 10.1109/TGRS.2009.2016214http://dx.doi.org/10.1109/TGRS.2009.2016214]
Tarabalka Y, Fauvel M, Chanussot J and Benediktsson J A. 2010. SVM- and MRF-based method for accurate classification of hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 7(4): 736-740 [DOI: 10.1109/LGRS.2010.2047711http://dx.doi.org/10.1109/LGRS.2010.2047711]
Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707
童庆禧, 张兵, 张立福. 2016. 中国高光谱遥感的前沿进展. 遥感学报, 20(5): 689-707 [DOI: 10.11834/jrs.20166264http://dx.doi.org/10.11834/jrs.20166264]
Wang W J, Dou S G, Jiang Z M and Sun L J. 2018. A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sensing, 10(7): 1068 [DOI: 10.3390/rs10071068http://dx.doi.org/10.3390/rs10071068]
Xi B B, Li J J, Li Y S, Song R, Xiao Y C, Shi Y Z and Du Q. 2022. Multi-direction networks with attentional spectral prior for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-15 [DOI: 10.1109/TGRS.2020.3047682http://dx.doi.org/10.1109/TGRS.2020.3047682]
Ye Z, Bai L and He M Y. 2021. Review of spatial-spectral feature extraction for hyperspectral image. Journal of Image and Graphics, 26(8): 1737-1763
叶珍, 白璘, 何明一. 2021. 高光谱图像空谱特征提取综述. 中国图象图形学报, 26(8): 1737-1763 [DOI: 10.11834/jig.210198http://dx.doi.org/10.11834/jig.210198]
Yin H, Gong Y H and Qiu G P. 2019. Side window filtering//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE: 8750-8758 [DOI: 10.1109/CVPR.2019.00896http://dx.doi.org/10.1109/CVPR.2019.00896]
Zhang B. 2016. Advancement of hyperspectral image processing and information extraction. Journal of Remote Sensing, 20(5): 1062-1090
张兵. 2016. 高光谱图像处理与信息提取前沿. 遥感学报, 20(5): 1062-1090 [DOI: 10.11834/jrs.20166179http://dx.doi.org/10.11834/jrs.20166179]
Zhang B, Li S S, Jia X P, Gao L R and Peng M. 2011. Adaptive Markov random field approach for classification of hyperspectral imagery. IEEE Geoscience and Remote Sensing Letters, 8(5): 973-977 [DOI: 10.1109/LGRS.2011.2145353http://dx.doi.org/10.1109/LGRS.2011.2145353]
Zhang H K, Li Y, Zhang Y Z and Shen Q. 2017. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letter, 8(5): 438-447 [DOI: 10.1080/2150704X.2017.1280200http://dx.doi.org/10.1080/2150704X.2017.1280200]
Zhang L P, Zhang L F and Du B. 2016. Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2): 22-40 [DOI: 10.1109/MGRS.2016.2540798http://dx.doi.org/10.1109/MGRS.2016.2540798]
Zhang M M, Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 27(6): 2623-2634 [DOI: 10.1109/TIP.2018.2809606http://dx.doi.org/10.1109/TIP.2018.2809606]
Zheng Z, Zhong Y F, Ma A L and Zhang L P. 2020. FPGA: fast patch-free global learning framework for fully end-to-end hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(8): 5612-5626 [DOI: 10.1109/TGRS.2020.2967821http://dx.doi.org/10.1109/TGRS.2020.2967821]
Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858 [DOI: 10.1109/TGRS.2017.2755542http://dx.doi.org/10.1109/TGRS.2017.2755542]
文章被引用时，请邮件提醒。
提交
相关作者
靳淇文 武汉大学 电子信息学院
马泳 武汉大学 电子信息学院;武汉大学 宇航科学与技术研究院
樊凡 武汉大学 电子信息学院;武汉大学 宇航科学与技术研究院
黄珺 武汉大学 电子信息学院;武汉大学 宇航科学与技术研究院
李皞 武汉轻工大学 数学与计算机科学学院
梅晓光 武汉大学 电子信息学院;武汉大学 宇航科学与技术研究院
魏祥坡 信息工程大学
余旭初 信息工程大学
相关机构
武汉大学 电子信息学院
武汉大学 宇航科学与技术研究院
武汉轻工大学 数学与计算机科学学院
信息工程大学
复旦大学 信息科学与工程学院 电磁波信息科学教育部重点实验室
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部