光学信号Token引导的异源遥感变化检测网络

刘秦森; 孙帮勇

doi:10.11834/jrs.20233067

高光谱信息提取 | 浏览量 : 0 下载量: 301 CSCD: 0

R-PDF
PDF
导出
分享
收藏
专辑

光学信号Token引导的异源遥感变化检测网络
Optical-signal token guided change detection network for heterogeneous remote sensing image
2024年28卷第1期页码：88-104
纸质出版日期： 2024-01-07 ，
DOI： 10.11834/jrs.20233067
稿件说明：

移动端阅览

刘秦森，孙帮勇.2024.光学信号Token引导的异源遥感变化检测网络.遥感学报，28（1）： 88-104

Liu Q S and Sun B Y. 2024. Optical-signal token guided change detection network for heterogeneous remote sensing image. National Remote Sensing Bulletin， 28（1）：88-104
刘秦森，孙帮勇.2024.光学信号Token引导的异源遥感变化检测网络.遥感学报，28（1）： 88-104 DOI： 10.11834/jrs.20233067.

Liu Q S and Sun B Y. 2024. Optical-signal token guided change detection network for heterogeneous remote sensing image. National Remote Sensing Bulletin， 28（1）：88-104 DOI： 10.11834/jrs.20233067.

摘要

基于遥感图像中的光学信号检测出一定时间内特定区域的变化状态的遥感图像变化检测方法，在国防安全、环境监测、城市建设等领域具有重要应用价值。由于多时相异源图像在成像机理、光谱范围、空间分辨率等方面存在差异，现阶段异源遥感图像变化检测仍存在精度不够高、漏检和误检等问题，本文提出一种基于Transformer网络的异源变化检测网络框架，该框架能够利用不同类别的异源遥感图像获得准确的变化检测结果。首先，所提出检测网络为多时相遥感图像自适应生成对应的光学信号Token（光信Token）；然后，以光信Token作为引导与对应图像块Token进行交互计算，从而对双时相序列特征进行变化分析，并且在交互学习过程中构建了差分放大模块以提高网络对特征间差分信息的提取精度；最后，利用多层感知机对输出的差分Token进行预测并分割出变化区域。采用Sardinia、Shuguang和Bastrop等3个不同类别的异源遥感图像数据集和Farmland同源高光谱图像数据集来验证本文提出的方法，结果证明在选取有限训练样本数据情况下，本文方法与现有主流变化检测方法相比，在多个客观指标以及主观视觉上都表现出先进性。

Abstract

Change Detection (CD) is a vital technique for identifying and analyzing changes over time in a specific area using optical signals from remote sensing images. This technique has been extensively utilized in various fields

including national defense security

environmental monitoring

and urban construction. However

some challenges in achieving accurate and reliable CD are still encountered due to inherent disparities in imaging mechanisms

spectral ranges

and spatial resolutions among heterogeneous images. These challenges lead to issues such as inadequate accuracy

missed detections

and false detections. Heterogeneous remote sensing images can be regarded as sequences of different optical signals from the channel perspective. For example

RGB and infrared images can be regarded as sequences of spectral signals from different ranges. Transformers employ a multi-head attention mechanism that can effectively handle and analyze sequence information to achieve accurate heterogeneous CD. Thus

the paper proposes an optical signal token guided CD network for heterogeneous remote sensing images.

This paper presents a novel heterogeneous CD network

primarily comprising the optical-signal token transformer (OT-Former) and the cross-temporal transformer (CT-Former). The proposed method demonstrates the capacity to effectively handle diverse remote sensing images of distinct categories and attain precise CD results. Specifically

OT-Former can encode diverse heterogeneous images in channel-wise for adaptively generating the optical-signal tokens. Meanwhile

CT-Former can use the optical-signal tokens as a guide to interact with the patch token for the learning of change rules. Moreover

a Difference Amplification Module (DAM) is embedded into the network to enhance the extraction of difference information. This module utilizes a 1×2 convolutional kernel to effectively fuse difference information. Finally

the differential token is predicted by multilayer perceptron to output the CD results.

Experiments were conducted on three heterogeneous datasets and one homogeneous dataset to evaluate the performance of the proposed method. Furthermore

the proposed method was compared with six typical CD methods and evaluated the performance using overall accuracy (OA)

Kappa coefficient

and F1-score

among other evaluation metrics

to validate the effectiveness of the proposed network in this study. A limited number of samples were utilized for training during the experiment. Under identical experimental conditions

the proposed method demonstrated exceptional performance in homogeneous and heterogeneous CD. The results show that the proposed approach surpasses existing state-of-the-art methods in terms of qualitative and visual performance. Additionally

ablation experiments and parameter analyses were conducted to validate the effectiveness of the proposed methods

including the OT-Former

CT-Former

and DAM modules

and to assess the impact of various parameters within the network.

Overall

the current study presents a novel heterogeneous CD network based on the transformer framework. Within this network

OT-Former is proposed to achieve the adaptive generation of optical-signal tokens from diverse remote sensing images. Moreover

the CT-Former utilizes these optical-signal tokens as a guide to facilitate interaction with patch tokens for the learning of change rules. Additionally

DAM modules were embedded into the network to effectively extract the difference information. An extremely limited number of samples were utilized only for training in the experiments. Remarkably

the proposed method outperformed the existing state-of-the-art methods

achieving a significantly advanced performance in heterogeneous CD.

关键词

遥感异源图像变化检测多模态分析深度学习Transformer

Keywords

remote sensingheterogeneous imageschange detectionmultimodal analysisdeep learningtransformer

references

Adão T, Hruška J, Pádua L, Bessa J, Peres E, Morais R and Sousa J J. 2017. Hyperspectral imaging: a review on UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sensing, 9(11): 1110 [DOI: 10.3390/rs9111110http://dx.doi.org/10.3390/rs9111110]

Bandara W G C and Patel V M. 2022. A transformer-based Siamese network for change detection//IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur: IEEE: 207-210 [DOI: 10.1109/IGARSS46834.2022.9883686http://dx.doi.org/10.1109/IGARSS46834.2022.9883686]

Bengio Y, Simard P and Frasconi P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2): 157-166 [DOI: 10.1109/72.279181http://dx.doi.org/10.1109/72.279181]

Bovolo F and Bruzzone L. 2007. A theoretical framework for unsupervised change detection based on change vector analysis in the polar domain. IEEE Transactions on Geoscience and Remote Sensing, 45(1): 218-236 [DOI: 10.1109/TGRS.2006.885408http://dx.doi.org/10.1109/TGRS.2006.885408]

Bovolo F, Marchesi S and Bruzzone L. 2012. A framework for automatic and unsupervised detection of multiple changes in multitemporal images. IEEE Transactions on Geoscience and Remote Sensing, 50(6): 2196-2212 [DOI: 10.1109/TGRS.2011.2171493http://dx.doi.org/10.1109/TGRS.2011.2171493]

Canty M J and Nielsen A A. 2008. Automatic radiometric normalization of multitemporal satellite imagery with the iteratively re-weighted MAD transformation. Remote Sensing of Environment, 112(3): 1025-1036 [DOI: 10.1016/j.rse.2007.07.013http://dx.doi.org/10.1016/j.rse.2007.07.013]

Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J and Houlsby N. 2020. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint 2010.11929 [DOI: 10.48550/arXiv.2010.11929http://dx.doi.org/10.48550/arXiv.2010.11929]

Hasanlou M and Seydi S T. 2018. Hyperspectral change detection: an experimental comparative study. International Journal of Remote Sensing, 39(20): 7029-7083 [DOI: 10.1080/01431161.2018.1466079http://dx.doi.org/10.1080/01431161.2018.1466079]

Hong D F, Han Z, Yao J, Gao L R, Zhang B, Plaza A and Chanussot J. 2022. SpectralFormer: rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-15 [DOI: 10.1109/TGRS.2021.3130716http://dx.doi.org/10.1109/TGRS.2021.3130716]

Hou Z F, Li W and Du Q. 2021. A patch tensor-based change detection method for hyperspectral images//2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Brussels: IEEE: 4328-4331 [DOI: 10.1109/IGARSS47720.2021.9554630http://dx.doi.org/10.1109/IGARSS47720.2021.9554630]

Liu S C, Du K C, Zheng Y J, Chen J, Du P J and Tong X H. 2023. Remote sensing change detection technology in the era of artificial intelligence: inheritance, development and challenges. National Remote Sensing Bulletin, 27(9): 1975-1987

柳思聪, 都科丞, 郑永杰, 陈晋, 杜培军, 童小华. 2023. 人工智能时代的遥感变化检测技术: 继承、发展与挑战. 遥感学报, 27(9): 1975-1987 [DOI: 10.11834/jrs.20222199http://dx.doi.org/10.11834/jrs.20222199]

Lyu H B, Lu H and Mou L C. 2016. Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 8(6): 506 [DOI: 10.3390/rs8060506http://dx.doi.org/10.3390/rs8060506]

Ma W P, Yang H, Wu Y, Xiong Y T, Hu T, Jiao L C and Hou B. 2019. Change detection based on multi-grained cascade forest and multi-scale fusion for SAR images. Remote Sensing, 11(2): 142 [DOI: 10.3390/rs11020142http://dx.doi.org/10.3390/rs11020142]

Mandal M and Vipparthi S K. 2022. An empirical review of deep learning frameworks for change detection: model design, experimental frameworks, challenges and research needs. IEEE Transactions on Intelligent Transportation Systems, 23(7): 6101-6122 [DOI: 10.1109/TITS.2021.3077883http://dx.doi.org/10.1109/TITS.2021.3077883]

Pirrone D, Bovolo F and Bruzzone L. 2020. A novel framework based on polarimetric change vectors for unsupervised multiclass change detection in dual-pol intensity SAR images. IEEE Transactions on Geoscience and Remote Sensing, 58(7): 4780-4795 [DOI: 10.1109/TGRS.2020.2966865http://dx.doi.org/10.1109/TGRS.2020.2966865]

Qu J H, Xu Y S, Dong W Q, Li Y S and Du Q. 2022. Dual-branch difference amplification graph convolutional network for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-12 [DOI: 10.1109/TGRS.2021.3135567http://dx.doi.org/10.1109/TGRS.2021.3135567]

Sun W W, Yang G, Peng J T and Du Q. 2020. Lateral-slice sparse tensor robust principal component analysis for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(1): 107-111 [DOI: 10.1109/LGRS.2019.2915315http://dx.doi.org/10.1109/LGRS.2019.2915315]

Sun Y L, Lei L, Guan D D, Kuang G Y and Liu L. 2022a. Graph signal processing for heterogeneous change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-23 [DOI: 10.1109/TGRS.2022.3221489http://dx.doi.org/10.1109/TGRS.2022.3221489]

Sun Y L, Lei L, Guan D D, Li M and Kuang G Y. 2022b. Sparse-constrained adaptive structure consistency-based unsupervised image regression for heterogeneous remote-sensing change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14 [DOI: 10.1109/TGRS.2021.3110998http://dx.doi.org/10.1109/TGRS.2021.3110998]

Wang Y H, Hong D F, Sha J J, Gao L R, Liu L, Zhang Y G and Rong X H. 2022a. Spectral-spatial-temporal transformers for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14 [DOI: 10.1109/TGRS.2022.3203075http://dx.doi.org/10.1109/TGRS.2022.3203075]

Wang Y K, Chen X H, Gao L L, Huang W B, Sun F C and Wang Y H. 2022b. Multimodal token fusion for vision transformers//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12176-12185 [DOI: 10.1109/CVPR52688.2022.01187http://dx.doi.org/10.1109/CVPR52688.2022.01187]

Wu C, Du B and Zhang L P. 2014. Slow feature analysis for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 52(5): 2858-2874 [DOI: 10.1109/TGRS.2013.2266673http://dx.doi.org/10.1109/TGRS.2013.2266673]

Yang B, Mao Y, Chen J, Liu JQ, Chen J and Yan K. 2023. Review of remote sensing change detection in deep learning: bibliometric and analysis. National Remote Sensing Bulletin, 27(9): 1988-2005

杨彬, 毛银, 陈晋, 刘建强, 陈杰, 闫凯. 2023. 深度学习的遥感变化检测综述: 文献计量与分析. 遥感学报, 27(9): 1988-2005 [DOI: 10.11834/jrs.20222156http://dx.doi.org/10.11834/jrs.20222156]

Yang X F, Ye Y M, Li X T, Lau R Y K, Zhang X F and Huang X H. 2018. Hyperspectral image classification with deep learning models. IEEE Transactions on Geoscience and Remote Sensing, 56(9): 5408-5423 [DOI: 10.1109/TGRS.2018.2815613http://dx.doi.org/10.1109/TGRS.2018.2815613]

Zhan Y, Fu K, Yan M L, Sun X, Wang H Q and Qiu X S. 2017. Change detection based on deep Siamese convolutional network for optical aerial images. IEEE Geoscience and Remote Sensing Letters, 14(10): 1845-1849 [DOI: 10.1109/LGRS.2017.2738149http://dx.doi.org/10.1109/LGRS.2017.2738149]

文章被引用时，请邮件提醒。

提交

深度学习的遥感变化检测综述：文献计量与分析

人工智能时代的遥感变化检测技术：继承、发展与挑战

面向高分辨率遥感影像变化检测的混合空间金字塔池化网络