高光谱影像三分支分组空谱注意力深度分类网络

苏涵; 陈娜; 彭江涛; 孙伟伟

doi:10.11834/jrs.20232492

高光谱地物识别 | 浏览量 : 0 下载量: 389 CSCD: 0

R-PDF
PDF
导出
分享
收藏
专辑

高光谱影像三分支分组空谱注意力深度分类网络
Hyperspectral image classification based on three branch network with grouped spatial-spectral attention
2024年28卷第1期页码：247-265
纸质出版日期： 2024-01-07 ，
DOI： 10.11834/jrs.20232492
稿件说明：

移动端阅览

苏涵，陈娜，彭江涛，孙伟伟.2024.高光谱影像三分支分组空谱注意力深度分类网络.遥感学报，28（1）： 247-265

Su H，Chen N，Peng J T and Sun W W. 2024. Hyperspectral image classification based on three branch network with grouped spatial-spectral attention. National Remote Sensing Bulletin， 28（1）：247-265
苏涵，陈娜，彭江涛，孙伟伟.2024.高光谱影像三分支分组空谱注意力深度分类网络.遥感学报，28（1）： 247-265 DOI： 10.11834/jrs.20232492.

Su H，Chen N，Peng J T and Sun W W. 2024. Hyperspectral image classification based on three branch network with grouped spatial-spectral attention. National Remote Sensing Bulletin， 28（1）：247-265 DOI： 10.11834/jrs.20232492.

摘要

高光谱影像具有丰富的空间和光谱信息，充分提取和利用这两个维度的信息是高光谱分类算法重点关注的问题。目前深度特征提取网络通常利用单分支串行网络连续提取空谱特征或双分支并行网络分别提取空谱特征。由于空间和光谱维内在差异，单分支串行网络连续提取的两类特征之间会互相干扰。并行双分支网络虽然可以减少两类特征之间的干扰，但同时会忽略空间和光谱特征间的潜在相关性。为解决上述问题，本文提出了一种三分支分组空谱注意力深度网络结构。该网络具有3个分支，分别用于提取空间、光谱和空谱联合特征。针对3个分支的不同特性，设计了不同的注意力机制以加强特征的判别性。该网络既可以提取独立的空间和光谱特征，又保留了空间和光谱之间的相关性。在5个数据集上的实验表明，本文所提出的方法要优于现有的一些先进算法。

Abstract

Hyperspectral images have abundant spatial and spectral information. Numerous hyperspectral classification algorithms focus on the extraction and maximization of spatial and spectral information. Deep feature extraction networks generally extract spectral-spatial features using single-branch serial or double-branch parallel structures. However

single-branch structures may lead to mutual interference between features of spectral and spatial dimensions

and double-branch parallel structures tend to ignore the correlation between spatial and spectral features. This paper proposes a three-branch grouped spatial-spectral attention network (TGSSAN) to consider the differences and correlations between spatial and spectral features. TGSSAN can extract independent spectral-spatial features while preserving their correlation.

This paper proposes the TGSSAN

which has three parallel branches (i.e.

spectral

spatial

and spectral-spatial branches). These branches can separately extract spectral

spatial

and spatial-spectral features. Different attention blocks are designed in three branches to enhance the discriminative capability of features. In particular

a grouped spatial-spectral attention mechanism is proposed in the spectral-spatial branch to obtain spatial and spectral attention simultaneously. Finally

three branch features are fused for classification.

In the experiment

the proposed TGSSAN algorithm is compared with some advanced deep learning algorithms

such as SSRN

FDSSC

DBMA

DBDA

HResNetAM

and A2S2KResNet. The performance of different algorithms is evaluated on five hyperspectral data sets. Experimental results show that the proposed algorithm achieved superior classification performance on IP

and HHK datasets. In particular

the proposed algorithm achieves higher classification accuracy despite limited training samples compared with the existing advanced algorithms.

The TGSSAN method proposed in this paper improves the shortcomings of the single-branch serial and double-branch parallel structures for continuous extraction of spectral-spatial features

which can effectively extract image spectral-spatial feature information. The three attention blocks designed in this paper namely

spectral

grouped spatial-spectral

and spatial attention modules

can effectively enhance the feature discrimination capability and further improve the classification performance.

关键词

高光谱影像分类注意力机制三分支结构深度网络

Keywords

hyperspectral image classificationattention mechanismthree-branch networkdeep network

references

Chen G Y and Qian S E. 2007. Dimensionality reduction of hyperspectral imagery. Journal of Applied Remote Sensing, 1(1): 013509 [DOI: 10.1117/1.2723663http://dx.doi.org/10.1117/1.2723663]

Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251 [DOI: 10.1109/TGRS.2016.2584107http://dx.doi.org/10.1109/TGRS.2016.2584107]

Chen Y S, Zhu L, Ghamisi P, Jia X P, Li G Y and Tang L. 2017. Hyperspectral images classification with Gabor filtering and convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 14(12): 2355-2359 [DOI: 10.1109/LGRS.2017.2764915http://dx.doi.org/10.1109/LGRS.2017.2764915]

Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W and Lu H Q. 2019. Dual attention network for scene segmentation//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 3141-3149 [DOI: 10.1109/CVPR.2019.00326http://dx.doi.org/10.1109/CVPR.2019.00326]

Ghamisi P, Yokoya N, Li J, Liao W Z, Liu S C, Plaza J, Rasti B and Plaza A. 2017. Advances in hyperspectral image and signal processing: a comprehensive overview of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 5(4): 37-78 [DOI: 10.1109/MGRS.2017.2762087http://dx.doi.org/10.1109/MGRS.2017.2762087]

Govender M, Chetty K and Bulcock H. 2007. A review of hyperspectral remote sensing and its application in vegetation and water resource studies. Water SA, 33(2): 145-152 [DOI: 10.4314/wsa.v33i2.49049http://dx.doi.org/10.4314/wsa.v33i2.49049]

Guo M H, Xu T X, Liu J J, Liu Z N, Jiang P T, Mu T J, Zhang S H, Martin R R, Cheng M M and Hu S M. 2022. Attention mechanisms in computer vision: a survey. Computational Visual Media, 8(3): 331-368 [DOI: 10.1007/s41095-022-0271-yhttp://dx.doi.org/10.1007/s41095-022-0271-y]

Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7132-7141[DOI:10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745]

Jia X P, Kuo B C and Crawford M M. 2013. Feature mining for hyperspectral image classification. Proceedings of the IEEE, 101(3): 676-697 [DOI: 10.1109/JPROC.2012.2229082http://dx.doi.org/10.1109/JPROC.2012.2229082]

Kumar B, Dikshit O, Gupta A and Singh M K. 2020. Feature extraction for hyperspectral image classification: a review. International Journal of Remote Sensing, 41(16): 6248-6287 [DOI: 10.1080/01431161.2020.1736732http://dx.doi.org/10.1080/01431161.2020.1736732]

Li J J, Ma Y L, Song R, Xi B B, Hong D F and Du Q. 2022. A triplet semisupervised deep network for fusion classification of hyperspectral and LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 60: 5540513 [DOI: 10.1109/TGRS.2022.3213513http://dx.doi.org/10.1109/TGRS.2022.3213513]

Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): 582 [DOI: 10.3390/rs12030582http://dx.doi.org/10.3390/rs12030582]

Liu H, Li W, Xia X G, Zhang M M, Gao C Z and Tao R. 2023. Central attention network for hyperspectral imagery classification. IEEE Transactions on Neural Networks and Learning Systems, 34(11): 8989-9003 [DOI: 10.1109/TNNLS.2022.3155114http://dx.doi.org/10.1109/TNNLS.2022.3155114]

Liu Y S, Zhou S B, Han W, Liu W X, Qiu Z F and Li C. 2019. Convolutional neural network for hyperspectral data analysis and effective wavelengths selection. Analytica Chimica Acta, 1086: 46-54 [DOI: 10.1016/j.aca.2019.08.026http://dx.doi.org/10.1016/j.aca.2019.08.026]

Ma W P, Yang Q F, Wu Y, Zhao W and Zhang X R. 2019. Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sensing, 11(11): 1307 [DOI: 10.3390/rs11111307http://dx.doi.org/10.3390/rs11111307]

Mei S H, Ji J Y, Bi Q Q, Hou J H, Du Q and Li W. 2016. Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification//2016 IEEE International Geoscience and Remote Sensing Symposium. Beijing: IEEE: 5067-5070 [DOI: 10.1109/IGARSS.2016.7730321http://dx.doi.org/10.1109/IGARSS.2016.7730321]

Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790 [DOI: 10.1109/TGRS.2004.831865http://dx.doi.org/10.1109/TGRS.2004.831865]

Pal M. 2012. Multinomial logistic regression-based feature selection for hyperspectral data. International Journal of Applied Earth Observation and Geoinformation, 14(1): 214-220 [DOI: 10.1016/j.jag.2011.09.014http://dx.doi.org/10.1016/j.jag.2011.09.014]

Peng J T, Sun W W, Wei T H and Fan W Q. 2020. A modified correlation alignment algorithm for the domain adaptation of GF-5 hyperspectral image. Journal of Remote Sensing, 24(4): 417-426

彭江涛, 孙伟伟, 魏天慧, 范文琦. 2020. 高分五号高光谱影像的关联对齐域适应与分类. 遥感学报, 24(4): 417-426 [DOI: 10.11834/jrs.20209212http://dx.doi.org/10.11834/jrs.20209212]

Qian Y T, Ye M C and Zhou J. 2013. Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features. IEEE Transactions on Geoscience and Remote Sensing, 51(4): 2276-2291 [DOI: 10.1109/TGRS.2012.2209657http://dx.doi.org/10.1109/TGRS.2012.2209657]

Roy S K, Manna S, Song T C and Bruzzone L. 2021. Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7831-7843 [DOI: 10.1109/TGRS.2020.3043267http://dx.doi.org/10.1109/TGRS.2020.3043267]

Sisodia P S, Tiwari V and Kumar A. 2014. Analysis of supervised maximum likelihood classification for remote sensing image//International Conference on Recent Advances and Innovations in Engineering. Jaipur: IEEE: 1-4 [DOI: 10.1109/ICRAIE.2014.6909319http://dx.doi.org/10.1109/ICRAIE.2014.6909319]

Sun W W, Zhang D F, Yang G and Li W Y. 2018. Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis. Journal of Remote Sensing, 22(1): 110-118

孙伟伟, 张殿发, 杨刚, 李巍岳. 2018. 加权概率原型分析的高光谱影像波段选择. 遥感学报, 22(1): 110-118 [DOI: 10.11834/jrs.20186446http://dx.doi.org/10.11834/jrs.20186446]

Tang J L, Alelyani S and Liu H. 2014. Feature selection for classification: a review//Aggarwal C C. Data Classification: Algorithms and Applications. New York: Chapman and Hall/CRC: 37-64

Tang X, Meng F B, Zhang X R, Cheung Y M, Ma J J, Liu F and Jiao L C. 2021. Hyperspectral image classification based on 3-D octave convolution with spatial–spectral attention network. IEEE Transactions on Geoscience and Remote Sensing, 59(3): 2430-2447 [DOI: 10.1109/TGRS.2020.3005431http://dx.doi.org/10.1109/TGRS.2020.3005431]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł and Polosukhin I. 2017. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc.: 6000-6010

Vincent P, Larochelle H, Lajoie I, Bengio Y and Manzagol P A. 2010. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11: 3371-3408

Wan S, Yeh M L and Ma H L. 2021. An innovative intelligent system with integrated CNN and SVM: considering various crops through hyperspectral image data. ISPRS International Journal of Geo-Information, 10(4): 242 [DOI: 10.3390/ijgi10040242http://dx.doi.org/10.3390/ijgi10040242]

Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M and Hu Q H. 2020. ECA-Net: efficient channel attention for deep convolutional neural networks//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 11531-11539 [DOI: 10.1109/CVPR42600.2020.01155http://dx.doi.org/10.1109/CVPR42600.2020.01155]

Wang W J, Dou S G, Jiang Z M and Sun L J. 2018. A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sensing, 10(7): 1068 [DOI: 10.3390/rs10071068http://dx.doi.org/10.3390/rs10071068]

Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//15th European Conference on Computer Vision. Munich: Springer: 3-19 [DOI: 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1]

Xue Z X, Yu X C, Liu B, Tan X and Wei X P. 2021. HResNetAM: hierarchical residual network with attention mechanism for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 3566-3580 [DOI: 10.1109/JSTARS.2021.3065987http://dx.doi.org/10.1109/JSTARS.2021.3065987]

Yang K, Sun H, Zou C B and Lu X Q. 2022. Cross-attention spectral-spatial network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60: 5518714 [DOI: 10.1109/TGRS.2021.3133582http://dx.doi.org/10.1109/TGRS.2021.3133582]

Zhang B, Li S S, Jia X P, Gao L R and Peng M. 2011. Adaptive Markov random field approach for classification of hyperspectral imagery. IEEE Geoscience and Remote Sensing Letters, 8(5): 973-977 [DOI: 10.1109/LGRS.2011.2145353http://dx.doi.org/10.1109/LGRS.2011.2145353]

Zhang M M, Li W, Tao R, Li H C and Du Q. 2022a. Information fusion for classification of hyperspectral and LiDAR Data using IP-CNN. IEEE Transactions on Geoscience and Remote Sensing, 60: 5506812 [DOI: 10.1109/TGRS.2021.3093334http://dx.doi.org/10.1109/TGRS.2021.3093334]

Zhang Q L and Yang Y B. 2021. SA-Net: Shuffle attention for deep convolutional neural networks. IEEE International Conference on Acoustics, Speech and Signal Processing, 2235-2239 [DOI:10.1109/ICASSP39728.2021.9414568http://dx.doi.org/10.1109/ICASSP39728.2021.9414568]

Zhang X, Sun Y L, Shang K, Zhang L F and Wang S D. 2016. Crop classification based on feature band set construction and object-oriented approach using hyperspectral images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(9): 4117-4128 [DOI: 10.1109/JSTARS.2016.2577339http://dx.doi.org/10.1109/JSTARS.2016.2577339]

Zhang X M, Sun G Y, Jia X P, Wu L X, Zhang A Z, Ren J C, Fu H and Yao Y J. 2022b. Spectral-spatial self-attention networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60: 5512115 [DOI: 10.1109/TGRS.2021.3102143http://dx.doi.org/10.1109/TGRS.2021.3102143]

Zhao H S. 2022. Research on Dimensionality Reduction and Classification of Hyperspectral Remote Sensing Images under Weakly Supervised Learning Framework. Changchun: Jilin University (赵海士. 2022. 弱监督学习框架下高光谱遥感影像降维与分类方法研究. 长春: 吉林大学) [DOI: 10.27162/d.cnki.gjlin.2022.000591]

Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858 [DOI: 10.1109/TGRS.2017.2755542http://dx.doi.org/10.1109/TGRS.2017.2755542]

Zhou P C, Han J W, Cheng G and Zhang B C. 2019. Learning compact and discriminative stacked autoencoder for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(7): 4823-4833 [DOI: 10.1109/TGRS.2019.2893180http://dx.doi.org/10.1109/TGRS.2019.2893180]

Zhu C Q and Yang X M. 1998. Study of remote sensing image texture analysis and classification using wavelet. International Journal of Remote Sensing, 19(16): 3197-3203 [DOI: 10.1080/014311698214262http://dx.doi.org/10.1080/014311698214262]

Zhu Z X, Jia S, He S, Sun Y W, Ji Z and Shen L L. 2015. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 298: 274-287 [DOI: 10.1016/j.ins.2014.11.045http://dx.doi.org/10.1016/j.ins.2014.11.045]

文章被引用时，请邮件提醒。

提交

基于多尺度注意力机制的PolSAR深度学习超分辨率模型

从光学到SAR：基于多级跨模态对齐的SAR图像舰船检测算法

光学遥感图像的小样本目标检测

多层级几何—语义融合的图神经网络地表异常检测框架

注意力引导全尺度连接网络的高分辨率影像变化检测