Hybrid-Gird： 遥感图像细粒度分类可解释方法
Hybrid-Gird： An explainable method for fine-grained classification of remote sensing images
- 2024年28卷第7期 页码：1722-1734
收稿：2023-07-11，
纸质出版：2024-07-07
DOI： 10.11834/jrs.20243252
移动端阅览
摘要
基于遥感图像的目标细粒度分类深度神经网络已技术日益成熟，网络决策的可解释性研究是当前细粒度分类深度学习算法进一步提高决策可信度的关键问题。为精确表征对模型决策起决定性作用的本质特征，本文基于博弈竞争理论对遥感图像目标细粒度分类任务进行建模，分析了IG、SmoothGrad、Grad-CAM等可解释性方法在遥感图像目标细粒度分类网络上的适用性，提出了一种尺度自适应的目标细粒度分类本质特征可解释性分析方法Hybrid-Grid，使用像素级与局部特征关系融合算法提高对支撑网络决策的目标本质特征的精确描述能力。结果表明：本文提出的Hybrid-Grid对目标细粒度分类网络的解释效果在ADCC量化评估指标上达到78.87，相较Score-CAM有大幅提升；与SmoothGrad、Grad-CAM的解释结果相对比，本文方法在删除及精度损失实验上表现最好，使EFM-Net的Top-1准确率、Top-5准确率、F1得分分别损失了16.92%、1.61%、17.21%，证明Hybrid-Grid准确解释了对细粒度分类网络决策贡献最大的目标本质特征。本文提出的可解释性分析方法能够更精准地揭示当前目标细粒度分类网络的决策特征依据。
Abstract
At present
the deep neural network for targets fine-grained classification based on remote sensing images has been widely applied in military and civilian fields. The “black box” problem of deep learning network models still makes it difficult for people to understand the decision-making basis of the network in fine-grained classification tasks. This not only limits the possibility of deep neural networks optimizing and improving through feedback guidance
but also makes them unable to be fully trusted by humans and applied in important fields such as military and medical fields. Thus
how to carry out explainability research on its internal decision-making mechanism is the key problem for the current fine-grained algorithm to further improve the credibility of decision-making basis. This article summarized the commonly used explainable methods in existing fine-grained classification networks for remote sensing image targets. On this basis
a mathematical model for the problem of target fine-grained classification is established. It also models the fine-grained classification task of remote sensing image targets based on game competition theory
analyzes the applicability of explainable methods such as IG
SmoothGrad
and Grad CAM on the fine-grained classification network of remote sensing image targets
and proposes a scale adaptive method for analyzing the explainability of essential features of fine-grained classification of targets
that is
Hybrid Grid. The fusion algorithm of pixel level and local feature relationships is used to improve the accurate description ability of essential features of targets that support network decision-making. The typical explaining methods involved in this article were evaluated using indicators such as Average Drop
Coherence
Complexity
ADCC
and Deletion and precision loss
confirming the accuracy of the proposed method. The experimental results show that the Hybrid-Grid proposed in this paper achieves a score of 78.87 on the quantitative evaluation index of ADCC for target fine-grained classification networks
which is significantly improved compared to Score-CAM. Compared with the explanation results of SmoothGrad and Grad-CAM
our method performed the best in deletion and accuracy loss experiments
resulting in a loss of 16.92%
1.61%
and 17.21% in the Top-1 accuracy
Top-5 accuracy
and F1 score of EFM Net
respectively. This proves that the Hybrid Grid accurately explains the essential feature of the target that contributes the most to fine-grained classification network decision-making. Based on several typical fine-grained classification networks
the feasibility and universality of the explaining method proposed in this paper have been confirmed. The explaining method proposed in this article can more accurately reveal the decision feature basis of the current target fine-grained classification network. Future work will further enhance the visualization and interpretation accuracy of the essential features of the target
assisting in improving the classification performance and decision-making credibility of fine-grained classification networks.
关键词
Keywords
references
Bach S , Binder A , Montavon G , Klauschen F , Müller K R and Samek W . 2015 . On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation . PLoS One , 10 ( 7 ): e 0130140 [ DOI: 10.1371/journal.pone.0130140 http://dx.doi.org/10.1371/journal.pone.0130140 ]
Cheng X , Chu C , Zheng Y , Ren J and Zhang Q S . 2021 . A game-theoretic taxonomy of visual concepts in DNNs. arXiv preprint arXiv: 2106 . 10938
Deng H , Zou N , Du M , Chen W , Feng G , Yang Z , Li Z and Zhang Q . 2023 . Understanding and unifying fourteen attribution methods with taylor interactions . arXiv preprint arXiv: 2303.01506 .
Du R , Xie J , Ma Z , Chang D , Song Y Z and Guo J . 2021 . Progressive learning of category-consistent multi-granularity features for fine-grained visual classification . IEEE Transactions on Pattern Analysis and Machine Intelligence , 44 ( 12 ): 9521 - 9535 .
Fu K , Dai W , Zhang Y , Wang Z R , Yan M L and Sun X . 2019 . MultiCAM: multiple class activation mapping for aircraft recognition in remote sensing images . Remote Sensing , 11 ( 5 ): 544 [ DOI: 10.3390/rs11050544 http://dx.doi.org/10.3390/rs11050544 ]
Gong J Y , Huan L X and Zheng X W . 2022 . Deep learning interpretability analysis methods in image interpretation . Acta Geodaetica et Cartographica Sinica , 51 ( 6 ): 873 - 884
龚健雅 , 宦麟茜 , 郑先伟 . 2022 . 影像解译中的深度学习可解释性分析方法 . 测绘学报 , 51 ( 6 ): 873 - 884 [ DOI: 10.11947/j.AGCS.2022.20220106 http://dx.doi.org/10.11947/j.AGCS.2022.20220106 ]
Han Y Q , Yang X Y , Pu T and Peng Z M . 2022 . Fine-grained recognition for oriented ship against complex scenes in optical remote sensing images . IEEE Transactions on Geoscience and Remote Sensing , 60 : 5612318 [ DOI: 10.1109/TGRS.2021.3123666 http://dx.doi.org/10.1109/TGRS.2021.3123666 ]
He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition/ /Proceedings of the IEEE conference on computer vision and pattern recognition: 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Liang W , Li J H , Diao W H , Sun X , Fu K and Wu Y R . 2020 . FGATR-Net: automatic network architecture design for fine-grained aircraft type recognition in remote sensing images . Remote Sensing , 12 ( 24 ): 4187 [ DOI: 10.3390/rs12244187 http://dx.doi.org/10.3390/rs12244187 ]
Lundberg S M and Lee S I . 2017 . A unified approach to interpreting model predictions // Proceedings of the 31st International Conference on Neural Information Processing Systems . Long Beach : Curran Associates, Inc.: 4768 - 4777
Nie Y , Bian C J and Li L G . 2022 . Adap-EMD: adaptive EMD for aircraft fine-grained classification in remote sensing . IEEE Geoscience and Remote Sensing Letters , 19 : 6509005 [ DOI: 10.1109/LGRS.2022.3168581 http://dx.doi.org/10.1109/LGRS.2022.3168581 ]
Petsiuk V , Das A and Saenko K . 2018 . RISE: randomized input sampling for explanation of black-box models // Proceedings of the British Machine Vision Conference 2018 . Newcastle : BMVA Press: 151
Poppi S , Cornia M , Baraldi L and Cucchiara R . 2021 . Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis/ /Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 2299 - 2304
Ribeiro M T , Singh S and Guestrin C . 2016 . Why should I trust you?": explaining the predictions of any classifier // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . San Francisco : ACM: 1135 - 1144 [ DOI: 10.1145/2939672.2939778 http://dx.doi.org/10.1145/2939672.2939778 ]
Chen S , Wang H , Xu F and Jin Y Q . 2016 . Target classification using the deep convolutional networks for sar images . IEEE transactions on geoscience and remote sensing , 54 ( 8 ): 4806 - 4817
Selvaraju R R , Cogswell M , Das A , Vedantam R , Parikh D and Batra D . 2017 . Grad-cam: Visual explanations from deep networks via gradient-based localization/ /Proceedings of the IEEE international conference on computer vision: 618 - 626
Selvaraju R R , Das A , Vedantam R , Cogswell M , Parikh D and Batra D . 2016 . Grad-CAM: Why did you say that?. arXiv preprint arXiv: 1611 . 07450
Simonyan K , Vedaldi A and Zisserman A . 2013 . Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv: 1312 . 6034
Smilkov D , Thorat N , Kim B , Viégas F and Wattenberg M . 2017 . Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: 1706 . 03825
Springenberg J T , Dosovitskiy A , Brox T and Riedmiller M . 2014 . Striving for simplicity: The all convolutional net. arXiv preprint arXiv: 1412 . 6806
Springenberg J T , Dosovitskiy A , Brox T and Riedmiller M . Striving for simplicity: The all convolutional net . arXiv preprint arXiv: 1412.6806 , 2014 .
Sundararajan M , Taly A , Yan Q . Axiomatic attribution for deep networks // International conference on machine learning . PMLR, 2017 : 3319 - 3328 .
Wang H F , Wang Z F , Du M N , Yang F , Zhang Z J , Ding S R , Mardziel P and Hu X . 2020 . Score-CAM: Score-weighted visual explanations for convolutional neural networks // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops . 2020: 24 - 25 [D0I: arXiv: 1910. 01279 v 2 ]
Xiong W , Xiong Z Y and Cui Y Q . 2022 . An explainable attention network for fine-grained ship classification using remote-sensing images . IEEE Transactions on Geoscience and Remote Sensing , 60 : 5620314 [ DOI: 10.1109/TGRS.2022.3162195 http://dx.doi.org/10.1109/TGRS.2022.3162195 ]
Yang P B , Sang J T , Zhang B , Feng Y G and Yu J . 2023 . Survey on interpretability of deep models for image classification . Journal of Software , 34 ( 1 ): 230 - 254
杨朋波 , 桑基韬 , 张彪 , 冯耀功 , 于剑 . 2023 . 面向图像分类的深度模型可解释性研究综述 . 软件学报 , 34 ( 1 ): 230 - 254 [ DOI: 10.13328/j.cnki.jos.006415 http://dx.doi.org/10.13328/j.cnki.jos.006415 ]
Yi Y H , You Y N , Li C and Zhou W L . 2023 . EFM-net: An essential feature mining network for target fine-grained classification in optical remote sensing images . IEEE Transactions on Geoscience and Remote Sensing . [ DOI: 10.1109/TGRS.2023.3265669 http://dx.doi.org/10.1109/TGRS.2023.3265669 ]
Zeiler M D and Fergus R . 2014 . Visualizing and understanding convolutional networks // Computer Vision-ECCV 2014: 13th European Conference , Zurich, Switzerland, September 6 - 12 , 2014, Proceedings, Part I 13 . Springer International Publishing, 2014 : 818 - 833
文章被引用时，请邮件提醒。
提交
相关作者
陈满 陆军工程大学 指挥控制工程学院
黄勇杰 陆军工程大学 指挥控制工程学院
徐磊 陆军工程大学 指挥控制工程学院
潘志松 陆军工程大学 指挥控制工程学院
王一帆 西南石油大学 计算机与软件学院
黄鲜 西南石油大学 计算机与软件学院
王建林 西南石油大学 计算机与软件学院
周彤 西南石油大学 计算机与软件学院
相关机构
陆军工程大学 指挥控制工程学院
西南石油大学 计算机与软件学院
长沙理工大学 计算机与通信工程学院
上海交通大学 电子信息与电气工程学院
武汉大学 电子信息学院
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
批量引用
导出至
BibTeXEndnoteRefManNoteExpressNoteFirst
回到顶部