结合协方差池化与跨尺度特征提取的高光谱分类

汪乐; 彭江涛; 陈娜; 孙伟伟

doi:10.11834/jrs.20242326

高光谱地物识别 | 浏览量 : 0 下载量: 502 CSCD: 0

R-PDF
PDF
导出
分享
收藏
专辑

结合协方差池化与跨尺度特征提取的高光谱分类
Hyperspectral classification algorithm based on covariance pooling and cross scale feature extraction
2024年28卷第1期页码：203-218
纸质出版日期： 2024-01-07 ，
DOI： 10.11834/jrs.20242326
稿件说明：

移动端阅览

汪乐，彭江涛，陈娜，孙伟伟.2024.结合协方差池化与跨尺度特征提取的高光谱分类.遥感学报，28（1）： 203-218

Wang L，Peng J T，Chen N and Sun W W. 2024. Hyperspectral classification algorithm based on covariance pooling and cross scale feature extraction. National Remote Sensing Bulletin， 28（1）：203-218
汪乐，彭江涛，陈娜，孙伟伟.2024.结合协方差池化与跨尺度特征提取的高光谱分类.遥感学报，28（1）： 203-218 DOI： 10.11834/jrs.20242326.

Wang L，Peng J T，Chen N and Sun W W. 2024. Hyperspectral classification algorithm based on covariance pooling and cross scale feature extraction. National Remote Sensing Bulletin， 28（1）：203-218 DOI： 10.11834/jrs.20242326.

摘要

深度卷积神经网络在高光谱图像分类任务上取得了优越性能。但是，主流深度学习算法通常采用一阶池化运算，容易忽略光谱之间的相关性，因而难以获取高阶统计判别特征。另外，这类算法往往难以选择最优的窗口大小去捕获不同感受野信息。针对上述问题，本文提出了一种结合协方差池化和跨尺度特征提取的高光谱影像分类方法。该方法设计了跨尺度自适应特征提取模块，能够自动提取多尺度特征，获取不同视野的互补信息，避免了尺度选择问题；进一步利用平均池化和快速协方差池化的联合池化操作，得到一阶统计量和结合空间光谱信息的二阶统计量；最终，将一阶和二阶池化特征进行融合用于分类。在3个公开高光谱数据集Indian Pines、Houston和Pavia University上分别随机选取5%、5%和1%标记样本进行训练，本文算法得到的总体分类精度分别达到97.63%、98.48%和98.21%，分类性能优于主流深度学习方法。

Abstract

The deep convolution neural network algorithm has achieved excellent performance in hyperspectral image classification. However

these deep learning algorithms generally use first-order pooling operation

which ignores the correlation between different spectral bands. Thus

obtaining high-order statistical discriminant features is difficult. In addition

using these algorithms to choose the optimal window size and capture different receptive field information is complicated. This paper proposes a hyperspectral classification method combining covariance pooling and cross-scale feature extraction to solve the aforementioned problems. This method aims to automatically extract the complementary and discriminative information of different scales and exploit the first- and second-order pooling features to improve the classification performance.

A covariance pooling and cross-scale feature extraction method is proposed for hyperspectral image classification. In this method

a cross-scale adaptive feature extraction module is designed. This module can automatically combine multiscale feature information and obtain complementary information of different visual fields

avoiding the scale selection problem. Furthermore

the first- and second-order statistics combined with spatial-spectral information are obtained using the joint pooling operation of average and fast covariance pooling. Finally

the first- and second-order pooled features are fused for classification.

A total of 5%

and 1% labeled samples were randomly selected from three public hyperspectral datasets

namely

Indian pines

Houston University

and Pavia University

respectively. The overall classification accuracy of the proposed algorithm reached 97.63%

98.48%

and 98.21%

and the classification performance was better than the state-of-the-art deep learning methods.

Cross-scale feature extraction considers the complementary spatial-spectral information between different scales to obtain additional adaptive feature information. Combining fast covariance and average pooling

the discriminant features are obtained by pooling feature fusion to obtain superior classification results.

关键词

高光谱图像分类协方差池化多尺度特征融合卷积神经网络

Keywords

hyperspectral image classificationcovariance poolingmultiscalefeature fusionconvolution neural networks

references

Benediktsson J A, Palmason J A and Sveinsson J R. 2005. Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 480-491 [DOI: 10.1109/TGRS.2004.842478http://dx.doi.org/10.1109/TGRS.2004.842478]

Camps-Valls G, Gomez-Chova L, Munoz-Mari J, Vila-Frances J and Calpe-Maravilla J. 2006. Composite kernels for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 3(1): 93-97 [DOI: 10.1109/LGRS.2005.857031http://dx.doi.org/10.1109/LGRS.2005.857031]

Carreira J, Caseiro R, Batista J and Sminchisescu C. 2015. Free-form region description with second-order pooling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(6): 1177-1189 [DOI: 10.1109/TPAMI.2014.2361137http://dx.doi.org/10.1109/TPAMI.2014.2361137]

Chen Y, Nasrabadi N M and Tran T D. 2011. Hyperspectral image classification using dictionary-based sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 3973-3985 [DOI: 10.1109/TGRS.2011.2129595http://dx.doi.org/10.1109/TGRS.2011.2129595]

Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251 [DOI: 10.1109/TGRS.2016.2584107http://dx.doi.org/10.1109/TGRS.2016.2584107]

Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107 [DOI: 10.1109/JSTARS.2014.2329330http://dx.doi.org/10.1109/JSTARS.2014.2329330]

Demarez V. 1999. Seasonal variation of leaf chlorophyll content of a temperate forest. Inversion of the PROSPECT model. International Journal of Remote Sensing, 20(5): 879-894 [DOI: 10.1080/014311699212975http://dx.doi.org/10.1080/014311699212975]

Du P J, Xia J S, Xue Z H, Tan K, Su H J and Bao R. 2016. Review of hyperspectral remote sensing image classification. Journal of Remote Sensing, 20(2): 236-256

杜培军, 夏俊士, 薛朝辉, 谭琨, 苏红军, 鲍蕊. 2016. 高光谱遥感影像分类研究进展. 遥感学报, 20(2): 236-256 [DOI: 10.11834/jrs.20165022http://dx.doi.org/10.11834/jrs.20165022]

Gong Z Q, Zhong P, Yu Y, Hu W D and Li S T. 2019. A CNN with multiscale convolution and diversified metric for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(6): 3599-3618 [DOI: 10.1109/TGRS.2018.2886022http://dx.doi.org/10.1109/TGRS.2018.2886022]

Ham J, Chen Y C, Crawford M M and Ghosh J. 2005. Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 492-501 [DOI: 10.1109/TGRS.2004.842481http://dx.doi.org/10.1109/TGRS.2004.842481]

He N J, Fang L Y, Li S T, Plaza A and Plaza J. 2018. Remote sensing scene classification using multilayer stacked covariance pooling. IEEE Trans actions on Geoscience and Remote Sensing, 56(12): 6899-6910 [DOI: 10.1109/TGRS.2018.2845668http://dx.doi.org/10.1109/TGRS.2018.2845668]

He N J, Paoletti M E, Haut J M, Fang L Y, Li S T, Plaza A and Plaza J. 2019. Feature extraction with multiscale covariance maps for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 755-769 [DOI: 10.1109/TGRS.2018.2860464http://dx.doi.org/10.1109/TGRS.2018.2860464]

Ionescu C, Vantzos O and Sminchisescu C. 2015. Matrix backpropagation for deep networks with structured layers//2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 2965-2973 [DOI: 10.1109/ICCV.2015.339http://dx.doi.org/10.1109/ICCV.2015.339]

Jia S, Shen L L and Li Q Q. 2015. Gabor feature-based collaborative representation for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(2): 1118-1129 [DOI: 10.1109/TGRS.2014.2334608http://dx.doi.org/10.1109/TGRS.2014.2334608]

Lee H and Kwon H. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 26(10): 4843-4855 [DOI: 10.1109/TIP.2017.2725580http://dx.doi.org/10.1109/TIP.2017.2725580]

Li P H, Xie J T, Wang Q L and Gao Z L. 2018. Towards faster training of global covariance pooling networks by iterative matrix square root normalization//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 947-955 [DOI: 10.1109/CVPR.2018.00105http://dx.doi.org/10.1109/CVPR.2018.00105]

Li P H, Xie J T, Wang Q L and Zuo W M. 2017a. Is second-order information helpful for large-scale visual recognition?//2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2089-2097 [DOI: 10.1109/ICCV.2017.228http://dx.doi.org/10.1109/ICCV.2017.228]

Li S M, Zhu X Y and Bao J. 2019. Hierarchical multi-scale convolutional neural networks for hyperspectral image classification. Sensors, 19(7): 1714 [DOI: 10.3390/s19071714http://dx.doi.org/10.3390/s19071714]

Li T, Zhang J P and Zhang Y. 2014. Classification of hyperspectral image based on deep belief networks//2014 IEEE International Conference on Image Processing. Paris: IEEE: 5132-5136 [DOI: 10.1109/ICIP.2014.7026039http://dx.doi.org/10.1109/ICIP.2014.7026039]

Li W, Chen C, Su H J and Du Q. 2015. Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3681-3693 [DOI: 10.1109/TGRS.2014.2381602http://dx.doi.org/10.1109/TGRS.2014.2381602]

Li Y, Zhang H K and Shen Q. 2017b. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1): 67 [DOI: 10.3390/rs9010067http://dx.doi.org/10.3390/rs9010067]

Lin T Y, RoyChowdhury A and Maji S. 2015. Bilinear CNN models for fine-grained visual recognition//2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 1449-1457 [DOI: 10.1109/ICCV.2015.170http://dx.doi.org/10.1109/ICCV.2015.170]

Ma L, Crawford M M and Tian J W. 2010. Local manifold learning based K-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 48(11): 4099-4109 [DOI: 10.1109/TGRS.2010.2055876http://dx.doi.org/10.1109/TGRS.2010.2055876]

Makantasis K, Karantzalos K, Doulamis A and Doulamis N. 2015. Deep supervised learning for hyperspectral data classification through convolutional neural networks. IEEE International Geoscience and Remote Sensing Symposium. Milan: IEEE: 4959-4962 [DOI: 10.1109/IGARSS.2015.7326945http://dx.doi.org/10.1109/IGARSS.2015.7326945]

Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790 [DOI: 10.1109/TGRS.2004.831865http://dx.doi.org/10.1109/TGRS.2004.831865]

Mura M D, Benediktsson J A, Waske B and Bruzzone L. 2010. Extended profiles with morphological attribute filters for the analysis of hyperspectral data. International Journal of Remote Sensing, 31(22): 5975-5991 [DOI: 10.1080/01431161.2010.512425http://dx.doi.org/10.1080/01431161.2010.512425]

Murphy R J, Whelan B, Chlingaryan A and Sukkarieh S. 2019. Quantifying leaf-scale variations in water absorption in lettuce from hyperspectral imagery: a laboratory study with implications for measuring leaf water content in the context of precision agriculture. Precision Agriculture, 20(4): 767-787 [DOI: 10.1007/s11119-018-9610-5http://dx.doi.org/10.1007/s11119-018-9610-5]

Peng J T, Chen H, Zhou Y C and Li L Q. 2017. Ideal regularized composite kernel for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(4): 1563-1574 [DOI: 10.1109/JSTARS.2016.2621416http://dx.doi.org/10.1109/JSTARS.2016.2621416]

Peng J T, Sun W W and Du Q. 2019. Self-paced joint sparse representation for the classification of hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 1183-1194 [DOI: 10.1109/TGRS.2018.2865102http://dx.doi.org/10.1109/TGRS.2018.2865102]

Peng J T, Sun W W, Wei T H and Fan W Q. 2020. A modified correlation alignment algorithm for the domain adaptation of GF-5 hyperspectral image. Journal of Remote Sensing (Chinese), 24(4): 417-426

彭江涛, 孙伟伟, 魏天慧, 范文琦. 2020. 高分五号高光谱影像的关联对齐域适应与分类. 遥感学报, 24(4): 417-426 [DOI: 10.11834/jrs.20209212http://dx.doi.org/10.11834/jrs.20209212]

Roy S K, Krishna G, Dubey S R and Chaudhuri B B. 2020. HybridSN: exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2): 277-281 [DOI: 10.1109/LGRS.2019.2918719http://dx.doi.org/10.1109/LGRS.2019.2918719]

Roy S K, Manna S, Song T C and Bruzzone L. 2021. Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7831-7843 [DOI: 10.1109/TGRS.2020.3043267http://dx.doi.org/10.1109/TGRS.2020.3043267]

Shimoni M, Haelterman R and Perneel C. 2019. Hypersectral imaging for military and security applications: combining myriad processing and sensing techniques. IEEE Geoscience and Remote Sensing Magazine, 7(2): 101-117 [DOI: 10.1109/MGRS.2019.2902525http://dx.doi.org/10.1109/MGRS.2019.2902525]

Song W W, Li S T, Fang L Y and Lu T. 2018. Hyperspectral image classification with deep feature fusion network. IEEE Transactions on Geoscience and Remote Sensing, 56(6): 3173-3184 [DOI: 10.1109/TGRS.2018.2794326http://dx.doi.org/10.1109/TGRS.2018.2794326]

Sun W W, Zhang D F, Yang G and Li W Y. 2018. Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis. Journal of Remote Sensing, 22(1): 110-118

孙伟伟, 张殿发, 杨刚, 李巍岳. 2018. 加权概率原型分析的高光谱影像波段选择. 遥感学报, 22(1): 110-118 [DOI: 10.11834/jrs.20186446http://dx.doi.org/10.11834/jrs.20186446]

Tao C, Pan H B, Li Y S and Zou Z R. 2015. Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geoscience and Remote Sensing Letters, 12(12): 2438-2442 [DOI: 10.1109/LGRS.2015.2482520http://dx.doi.org/10.1109/LGRS.2015.2482520]

Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707

童庆禧, 张兵, 张立福. 2016. 中国高光谱遥感的前沿进展. 遥感学报, 20(5): 689-707 [DOI: 10.11834/jrs.20166264http://dx.doi.org/10.11834/jrs.20166264]

Tuzel O, Porikli F, Meer P. 2006. Region covariance: A fast descriptor for detection and classification. ECCV European Conference on Computer Vision. 3952: 589-600. [DOI: 10.1007/11744047_45http://dx.doi.org/10.1007/11744047_45]

Xie J, He N J, Fang L Y and Ghamisi P. 2021. Multiscale Densely-Connected Fusion Networks for Hyperspectral Images Classification. IEEE Transactions on Circuits and Systems for Video Technology, 31(1): 246-259 [DOI: 10.1109/TCSVT.2020.2975566http://dx.doi.org/10.1109/TCSVT.2020.2975566]

Xue Z H, Zhang M X, Liu Y F and Du P J. 2021. Attention-based second-order pooling network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(11): 9600-9615 [DOI: 10.1109/TGRS.2020.3048128http://dx.doi.org/10.1109/TGRS.2020.3048128]

Zhang C J, Li G D, Du S H, Tan W Z and Gao F. 2019. Three-dimensional densely connected convolutional network for hyperspectral remote sensing image classification. Journal of Applied Remote Sensing, 13(1): 016519 [DOI: 10.1117/1.JRS.13.016519http://dx.doi.org/10.1117/1.JRS.13.016519]

Zhang H K, Li Y, Zhang Y Z and Shen Q. 2017. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letters, 8(5): 438-447 [DOI: 10.1080/2150704X.2017.1280200http://dx.doi.org/10.1080/2150704X.2017.1280200]

Zhao W Z and Du S H. 2016. Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing, 54(8): 4544-4554 [DOI: 10.1109/TGRS.2016.2543748http://dx.doi.org/10.1109/TGRS.2016.2543748]

Zheng J W, Feng Y C, Bai C and Zhang J L. 2021. Hyperspectral image classification using mixed convolutions and covariance pooling. IEEE Transactions on Geoscience and Remote Sensing, 59(1): 522-534 [DOI: 10.1109/TGRS.2020.2995575http://dx.doi.org/10.1109/TGRS.2020.2995575]

Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858 [DOI: 10.1109/TGRS.2017.2755542http://dx.doi.org/10.1109/TGRS.2017.2755542]

文章被引用时，请邮件提醒。

提交

具有分类器机制的高光谱图像特征提取方法

联合超像素降维和后处理优化的高光谱图像分类方法

基于光谱—空间注意力双边网络的高光谱图像分类

新型语义分割D-UNet的建筑物提取

基于深度学习的高分辨率卫星遥感影像条带噪声去除