层级特征交互与增强感受野双分支遥感图像去雾网络
A two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field
- 2023年27卷第12期 页码：2831-2846
纸质出版日期： 2023-12-07
DOI： 10.11834/jrs.20232333
扫 描 看 全 文
浏览全部资源
扫码关注微信
摘要
近年来，深度学习的去雾方法在图像去雾领域取得了显著的成绩。然而，大多数基于U型网络的去雾方法将编码层特征直接传递到对应解码层，缺乏浅层和深层特征之间的信息交互。此外，基于非U型网络的去雾方法存在感受野受限问题，无法有效的利用上下文信息。从而导致这些方法在场景尺度变化较大的遥感图像去雾中无法取得理想效果。为此，本文提出了一种层级特征交互与增强感受野的双分支遥感图像去雾网络，该方法包含层级特征交互子网和多尺度信息提取子网。其中，层级特征交互子网利用层级特征交互融合模块，逐层的在浅层特征中引入语义信息，深层特征中引入空间细节信息，从而增强编码层中不同层级特征之间的信息交互。多尺度信息提取子网利用多尺度残差空洞卷积模块，融合不同感受野的特征，从而获取对于遥感图像去雾至关重要的上下文信息。在两个公开数据集上的实验结果表明，本文提出的去雾方法相比现有的9种优秀的去雾算法，取得了最好的客观评价指标和视觉效果。
Abstract
In recent years
deep learning-based dehazing methods have achieved remarkable results in the field of image dehazing. However
most dehazing methods based on U-shaped networks directly transfer the features of the encoding layer to the corresponding decoding layers
which lacks information interaction between the low- and high-level features. Meanwhile
the network model designed based on the U-shaped structure may destroy the detailed information important for the restored image in the process of downsampling. As a result
the restored clear image lacks detailed texture and structure information. In addition
the dehazing method based on non-U-shaped network has limited receptive field
which hinders its capability to effectively utilize contextual information. As a result
these methods cannot achieve ideal dehazing results in remote sensing images with large scene scale changes. Therefore
this study proposes a two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field. This network includes hierarchical feature interaction sub-net and multi-scale information extraction sub-net. The hierarchical feature interaction sub-net uses the hierarchical feature interaction fusion module to introduce semantic information into low-level features and spatial details into high-level features layer by layer. This way enhances the information interaction between features at different levels in the encoding layer. The multi-scale information extraction sub-net uses the multi-scale residual dilated convolution module to fuse the features of different receptive fields for obtaining contextual information
which is crucial for remote sensing image dehazing. The experiment on two public datasets show that the dehazing method proposed in this study achieves the best evaluation compared with the existing nine excellent dehazing algorithms. Among them
in the three sub-test sets of the public remote sensing dataset Haze1k
the quantitative index PSNR values of this study reach 27.362
28.171
and 25.137 dB. In the two sub-test sets of the public remote sensing dataset RICE
the quantitative index PSNR values of this study reach 37.79 and 35.367 dB. In addition
the method proposed in this study is the closest to ground truth in terms of subjective visual qualities such as color
saturation
and sharpness
while still achieving the dehazing effect. The following conclusions can be drawn: (1) through the proposed hierarchical feature interaction fusion module
the deep semantic information in the coding stage is gradually interactively fused with the shallow detailed texture information
which enhances the expressive ability of the network and restores clear images with higher quality. (2) Through the multi-scale residual dilated convolution module
the dehazing network proposed in this study can increase the receptive field of the network without changing the size of the feature map. The contextual information of different scales can also be fused. (3) In two public remote sensing image dehazing datasets
namely
Haze1k and RICE
the dehazing method proposed in this study outperforms nine recently proposed excellent dehazing algorithms in terms of objective evaluation indexes and subjective visual effects.
关键词
深度学习遥感图像去雾层级特征交互感受野双分支
Keywords
deep learningremote sensing image dehazinghierarchical feature interactionreceptive fieldtwo-branch
references
Cai B L, Xu X M, Jia K, Qing C and Tao D C. 2016. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11): 5187-5198 [DOI: 10.1109/TIP.2016.2598681http://dx.doi.org/10.1109/TIP.2016.2598681]
Cox L J. 1977. Optics of the atmosphere-scattering by molecules and particles. Optica Acta: International Journal of Optics, 24(7): 779-779 [DOI: 10.1080/713819629http://dx.doi.org/10.1080/713819629]
Deng J, Dong W, Socher R, Li L J, Li K and Fei-Fei L. 2009. ImageNet: a large-scale hierarchical image database//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 248-255 [DOI: 10.1109/CVPR.2009.5206848http://dx.doi.org/10.1109/CVPR.2009.5206848]
Dong Y, Liu Y H, Zhang H, Chen S F and Qiao Y. 2020. FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 10729-10736 [DOI: 10.1609/aaai.v34i07.6701http://dx.doi.org/10.1609/aaai.v34i07.6701]
Engin D, Gen A and Ekenel H K. 2018. Cycle-Dehaze: enhanced CycleGAN for single image dehazing//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE: 825-833 [DOI: 10.1109/CVPRW.2018.00127http://dx.doi.org/10.1109/CVPRW.2018.00127]
Gao S H, Cheng M M, Zhao K, Zhang X Y, Yang M H and Torr P. 2021. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(2): 652-662 [DOI: 10.1109/TPAMI.2019.2938758http://dx.doi.org/10.1109/TPAMI.2019.2938758]
He K M, Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353 [DOI: 10.1109/TPAMI.2010.168http://dx.doi.org/10.1109/TPAMI.2010.168]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Hu J, Shen L, Albanie S, Sun G and Wu E H. 2020. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8): 2011-2023 [DOI: 10.1109/TPAMI.2019.2913372http://dx.doi.org/10.1109/TPAMI.2019.2913372]
Huang B H, Li Z, Yang C, Sun F C and Song Y X. 2020. Single satellite optical imagery dehazing using SAR image prior based on conditional generative adversarial networks//2020 IEEE Winter Conference on Applications of Computer Vision. Snowmass: IEEE: 1795-1802 [DOI: 10.1109/WACV45572.2020.9093471http://dx.doi.org/10.1109/WACV45572.2020.9093471]
Huang Y F and Chen X. 2021. Single remote sensing image dehazing using a dual-step cascaded residual dense network//2021 IEEE International Conference on Image Processing. Anchorage: IEEE: 3852-3856 [DOI: 10.1109/ICIP42928.2021.9506603http://dx.doi.org/10.1109/ICIP42928.2021.9506603]
Kim U H, Kim S H and Kim J H. 2022. SimVODIS: simultaneous visual odometry, object detection, and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1): 428-441 [DOI: 10.1109/TPAMI.2020.3007546http://dx.doi.org/10.1109/TPAMI.2020.3007546]
Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z H and Shi W Z. 2017. Photo-realistic single image super-resolution using a generative adversarial network//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 105-114 [DOI: 10.1109/CVPR.2017.19http://dx.doi.org/10.1109/CVPR.2017.19]
Li B, Luo H, Zhang H X, Tan S Q and Ji Z Z. 2017. A multi-branch convolutional neural network for detecting double JPEG compression. arXiv:1710.05477 [DOI: 10.48550/arXiv.1710.05477http://dx.doi.org/10.48550/arXiv.1710.05477]
Li Y F and Chen X. 2021. A coarse-to-fine two-stage attentive network for haze removal of remote sensing images. IEEE Geoscience and Remote Sensing Letters, 18(10): 1751-1755 [DOI: 10.1109/LGRS.2020.3006533http://dx.doi.org/10.1109/LGRS.2020.3006533]
Li Z L, Wang L Y, Jiang S, Wu Y H and Zhang Q J. 2021. On orbit extraction method of ship target in SAR images based on ultra-lightweight network. National Remote Sensing Bulletin, 25(3): 765-775
李宗凌, 汪路元, 蒋帅, 吴雨航, 张庆君. 2021. 超轻量网络的SAR图像舰船目标在轨提取. 遥感学报, 25(3): 765-775 [DOI: 10.11834/jrs.20210160http://dx.doi.org/10.11834/jrs.20210160]
Lin D Y, Xu G L, Wang X K, Wang Y, Sun X and Fu K. 2019. A remote sensing image dataset for cloud removal. arXiv:1901.00600 [DOI: 10.48550/arXiv.1901.00600http://dx.doi.org/10.48550/arXiv.1901.00600]
Mehta A, Sinha H, Mandal M and Narang P. 2021. Domain-aware unsupervised hyperspectral reconstruction for aerial image dehazing//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE: 413-422 [DOI: 10.1109/WACV48630.2021.00046http://dx.doi.org/10.1109/WACV48630.2021.00046]
Mou F. 2021. Research on Thin Cloud Removal and Application of Remote Sensing Image Based on Deep Learning. Chengdu: University of Electronic Science and Technology of China (牟范. 2021. 基于深度学习的遥感影像去薄云及应用研究. 成都: 电子科技大学) [DOI: 10.27005/d.cnki.gdzku.2021.001113]
Qin X, Wang Z L, Bai Y C, Xie X D and Jia H Z. 2020. FFA-Net: feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 11908-11915 [DOI: 10.1609/aaai.v34i07.6865http://dx.doi.org/10.1609/aaai.v34i07.6865]
Qu Y Y, Chen Y Z, Huang J Y and Xie Y. 2019. Enhanced Pix2pix dehazing network//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 8152-8160 [DOI: 10.1109/CVPR.2019.00835http://dx.doi.org/10.1109/CVPR.2019.00835]
Ren W Q, Liu S, Zhang H, Pan J S, Cao X C and Yang M H. 2016. Single image dehazing via multi-scale convolutional neural networks//14th European Conference on Computer Vision. Amsterdam: Springer: 154-169 [DOI: 10.1007/978-3-319-46475-6_10http://dx.doi.org/10.1007/978-3-319-46475-6_10]
Ren W Q, Ma L, Zhang J W, Pan J S, Cao X C, Liu W and Yang M H. 2018. Gated fusion network for single image dehazing//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3253-3261 [DOI: 10.1109/CVPR.2018.00343http://dx.doi.org/10.1109/CVPR.2018.00343]
Shi W Z, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, Rueckert D and Wang Z H. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 1874-1883 [DOI: 10.1109/CVPR.2016.207http://dx.doi.org/10.1109/CVPR.2016.207]
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [DOI: 10.48550/arXiv.1409.1556http://dx.doi.org/10.48550/arXiv.1409.1556]
Song Y F, Li J, Wang X G and Chen X W. 2018. Single image dehazing using ranking convolutional neural network. IEEE Transactions on Multimedia, 20(6): 1548-1560 [DOI: 10.1109/TMM.2017.2771472http://dx.doi.org/10.1109/TMM.2017.2771472]
Sun X and Xu J D. 2021. Remote sensing image dehazing method based on cascaded generative adversarial network. Journal of Computer Applications, 41(8): 2440-2444
孙潇, 徐金东. 2021. 基于级联生成对抗网络的遥感图像去雾方法. 计算机应用, 41(8): 2440-2444 [DOI: 10.11772/j.issn.1001-9081.2020101563http://dx.doi.org/10.11772/j.issn.1001-9081.2020101563]
Tan R T. 2008. Visibility in bad weather from a single image//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage: IEEE: 1-8 [DOI: 10.1109/CVPR.2008.4587643http://dx.doi.org/10.1109/CVPR.2008.4587643]
Tang K T, Yang J C and Wang J. 2014. Investigating haze-relevant features in a learning framework for image dehazing//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 2995-3002 [DOI: 10.1109/CVPR.2014.383http://dx.doi.org/10.1109/CVPR.2014.383]
Wang L W, Li Y, Huang J and Lazebnik S. 2019. Learning two-branch neural networks for image-text matching tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2): 394-407 [DOI: 10.1109/TPAMI.2018.2797921http://dx.doi.org/10.1109/TPAMI.2018.2797921]
Wei X S, Xu Y Y and Yang J. 2022. Review of webly-supervised fine-grained image recognition. Journal of Image and Graphics, 27(7): 2057-2077
魏秀参, 许玉燕, 杨健. 2022. 网络监督数据下的细粒度图像识别综述. 中国图象图形学报, 27(7): 2057-2077 [DOI: 10.11834/jig.210188http://dx.doi.org/10.11834/jig.210188]
Wu H Y, Liu J, Xie Y, Qu Y Y and Ma L Z. 2020. Knowledge transfer dehazing network for NonHomogeneous dehazing//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE: 1975-1983 [DOI: 10.1109/CVPRW50498.2020.00247http://dx.doi.org/10.1109/CVPRW50498.2020.00247]
Wu H Y, Qu Y Y, Lin S H, Zhou J, Qiao R Z, Zhang Z Z, Xie Y and Ma L Z. 2021. Contrastive learning for compact single image dehazing//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE: 10546-10555 [https://doi.org/10.1109/CVPR46437.2021.01041https://doi.org/10.1109/CVPR46437.2021.01041]
Yang S T, Wang P F, Wang J, Lou H Z and Gong T L. 2021. River flow estimation method based on UAV aerial photogrammetry. National Remote Sensing Bulletin, 25(6): 1284-1293
杨胜天, 王鹏飞, 王娟, 娄和震, 巩同梁. 2021. 结合无人机航空摄影测量的河道流量估算. 遥感学报, 25(6): 1284-1293 [DOI: 10.11834/jrs.20209082http://dx.doi.org/10.11834/jrs.20209082]
Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122 [DOI: 10.48550/arXiv.1511.07122http://dx.doi.org/10.48550/arXiv.1511.07122]
Yu Y K, Liu H, Fu M H, Chen J, Wang X Y and Wang K Y. 2021. A two-branch neural network for non-homogeneous dehazing via ensemble learning//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE: 193-202 [DOI: 10.1109/CVPRW53098.2021.00028http://dx.doi.org/10.1109/CVPRW53098.2021.00028]
Yuan F N, Li Z Q, Shi J T, Xia X and Li Y. 2021. Image defogging algorithm using a two-phase feature extraction strategy. Journal of Image and Graphics, 26(3): 568-580
袁非牛, 李志强, 史劲亭, 夏雪, 李雅. 2021. 两阶段特征提取策略的图像去雾. 中国图象图形学报, 26(3): 568-580 [DOI: 10.11834/jig.200057http://dx.doi.org/10.11834/jig.200057]
Zhang H and Patel V M. 2018. Densely connected pyramid dehazing network//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3194-3203 [DOI: 10.1109/CVPR.2018.00337http://dx.doi.org/10.1109/CVPR.2018.00337]
Zhang Z L, Zhang X Y, Peng C, Xue X Y and Sun J. 2018. ExFuse: enhancing feature fusion for semantic segmentation//15th European Conference on Computer Vision. Munich: Springer: 273-288 [DOI: 10.1007/978-3-030-01249-6_17http://dx.doi.org/10.1007/978-3-030-01249-6_17]
Zhao W D, Li S S, Li A, Zhang B and Chen J. 2021. Deep fusion of hyperspectral images and multi-source remote sensing data for classification with convolutional neural network. National Remote Sensing Bulletin, 25(7): 1489-1502
赵伍迪, 李山山, 李安, 张兵, 陈俊. 2021. 结合深度学习的高光谱与多源遥感数据融合分类. 遥感学报, 25(7): 1489-1502 [DOI: 10.11834/jrs.20219117http://dx.doi.org/10.11834/jrs.20219117]
Zhu Q S, Mai J and Shao L. 2014. Single image dehazing using color attenuation prior//Proceedings of the British Machine Vision Conference 2014. Nottingham: BMVA [DOI: 10.5244/C.28.114http://dx.doi.org/10.5244/C.28.114]
文章被引用时，请邮件提醒。
提交
相关作者
暂无数据
相关机构
北京林业大学 信息学院
国家林业和草原局林业智能信息处理工程技术研究中心
南京信息工程大学 遥感与测绘工程学院
北京师范大学 遥感科学国家重点实验室
武汉大学 测绘遥感信息工程国家重点实验室
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部