收稿：2023-07-11，
纸质出版：2024-07-07
移动端阅览
超分辨率重建技术在卫星遥感图像信息智能处理领域中有重要的应用。现有面向遥感图像超分辨率重建的深度学习方法大多只能处理一种比例因子的超分辨率重建任务，在多尺度层面上缺少泛化性，难以满足真实遥感图像多倍率连续放大的超分辨率重建任务需求。为解决遥感图像超分辨率重建过程中的多尺度放大问题，本文采用元学习的方法，在构建单一自适应模型的基础上实现对遥感图像的任意尺度超分辨率重建，提升遥感图像的空间分辨率，利用密集残差网络和通道注意力机制重建遥感图像中地物纹理、目标边缘等丰富细节信息。在真实遥感图像上的定量实验表明，本文所提方法重建结果的峰值信噪比能达到40 dB以上，同时在多种数据上的定量和定性实验结果证明了本文方法的有效性。
Super-resolution reconstruction technology plays an important role in the intelligent processing of satellite remote sensing images. Existing deep learning methods for super-resolution remote sensing image reconstruction can only handle super-resolution tasks with a single scale factor
lacking generalization at the multiscale level and failing to meet the requirements of real super-resolution remote sensing image reconstruction for continuous zooming at multiple magnification levels.
To address the problem of arbitrary-scale super-resolution reconstruction and enhance the quality of reconstructed real remote sensing images
this paper proposes a super-resolution reconstruction method called the Meta-RDCAN
which utilizes meta-learning and residual dense channel attention network.The proposed method employs a meta-upscale module that incorporates three functions: weight prediction
location projection
and feature mapping. The module adaptively adjusts the internal parameters of a model according to different scale factors for arbitrary-scale super-resolution reconstruction. From the perspective of extracting detailed information of local land objects in a remote sensing image
a dense residual network with an attention mechanism is used as a feature extractor
enabling the reconstructed results to possess clear and distinguishable details.Extensive experiments are conducted on various datasets
including DIV2K
AID
UCMerced
WIDS
Set5
and real remote sensing image from Macao Science Popularization Satellite. The influence of variations in spatial resolution on the super-resolution reconstruction results is analyzed
and the effectiveness of the training scheme
which involves pretraining on a general dataset followed by fine-tuning on a remote sensing dataset
is validated using the loss curve.
The test results with different scale factors demonstrate that the proposed model is suitable for arbitrary-scale super-resolution reconstruction tasks in remote sensing images
having scale factors of up to 4.0. Comparative experimental results show that the improved model with added channel attention achieves enhanced performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM)
compared with the baseline model. On real remote sensing data
the reconstruction results of the proposed model achieve a PSNR of over 40 dB and an SSIM of over 0.95. The comparison based on the no-reference quality indicator NIQE confirms that the perceived quality of the super-resolution reconstruction results of the improved model surpasses that of the baseline model.
The proposed method for arbitrary-scale super-resolution reconstruction is effective for remote sensing images by utilizing meta-learning and dense residual channel attention. The main contributions of this paper include two aspects. First
for the arbitrary-scale super-resolution reconstruction of remote sensing image
the meta-learning approach is employed to adaptively adjust the internal parameters of a model. This approach enables continuous integer- and non-integer-scale super-resolution reconstruction of a single remote sensing image with a single model. Second
to address issues
such as missing details and unclear edges of geographic features
in reconstruction results
a channel-attention mechanism is applied to enhance the dense residual network and improve the quality of super-resolution reconstruction results.
Agustsson E and Timofte R . 2017 . NTIRE 2017 challenge on single image super-resolution: dataset and study // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Honolulu : IEEE: 1122 - 1131 [ DOI: 10.1109/cvprw.2017.150 http://dx.doi.org/10.1109/cvprw.2017.150 ]
Bavirisetti D P and Dhuli R . 2016 . Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform . IEEE Sensors Journal , 16 ( 1 ): 203 - 209 [ DOI: 10.1109/jsen.2015.2478655 http://dx.doi.org/10.1109/jsen.2015.2478655 ]
Dosovitskiy A and Brox T . 2016 . Inverting visual representations with convolutional networks // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas : IEEE: 4829 - 4837 [ DOI: 10.1109/cvpr.2016.522 http://dx.doi.org/10.1109/cvpr.2016.522 ]
He Z and He D . 2020 . Deep learning-based super-resolution for GF-4 satellite imagery . Journal of Remote Sensing (in Chinese) , 24 ( 12 ): 1500 - 1510
贺智 , 贺丹 . 2020 . 基于深度学习的高分四号卫星图像超分辨率重建 . 遥感学报 , 24 ( 12 ): 1500 - 1510 [ DOI: 10.11834/jrs.20208397 http://dx.doi.org/10.11834/jrs.20208397 ]
Hu X C , Mu H Y , Zhang X Y , Wang Z L , Tan T N and Sun J . 2019 . Meta-SR: a magnification-arbitrary network for super-resolution // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach : IEEE: 1575 - 1584 [ DOI: 10.1109/CVPR.2019.00167 http://dx.doi.org/10.1109/CVPR.2019.00167 ]
Kulkarni K , Lohit S , Turaga P , Kerviche R and Ashok A . 2016 . ReconNet: non-iterative reconstruction of images from compressively sensed measurements // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas : IEEE: 449 - 458 [ DOI: 10.1109/cvpr.2016.55 http://dx.doi.org/10.1109/cvpr.2016.55 ]
Liu E , Logan J , Lee M , Roy S , Wang S , Valerie V M and Venkatesh V . 2019 . WiDS Datathon 2019 . Kaggle . https://kaggle.com/competitions/widsdatathon2019 https://kaggle.com/competitions/widsdatathon2019
Mittal A , Soundararajan R and Bovik A C . 2013 . Making a “completely blind” image quality analyzer . IEEE Signal Processing Letters , 20 ( 3 ): 209 - 212 [ DOI: 10.1109/lsp.2012.2227726 http://dx.doi.org/10.1109/lsp.2012.2227726 ]
Pratt L and Thrun S . 1997 . Guest editors’ introduction . Machine Learning , 28 ( 1 ): 5 [ DOI: 10.1023/A:1007322005825 http://dx.doi.org/10.1023/A:1007322005825 ]
Ren S B , Meng Q and Wu Z . 2022 . Flow-based super-resolution reconstruction of remote sensing images . Chinese Space Science and Technology , 42 ( 6 ): 99 - 106
任术波 , 孟倩 , 吴钻 . 2022 . 基于流的遥感图像超分辨率重建 . 中国空间科学技术 , 42 ( 6 ): 99 - 106 [ DOI: 10.16708/j.cnki.1000-758X.2022.0088 http://dx.doi.org/10.16708/j.cnki.1000-758X.2022.0088 ]
Thrun S and Pratt L . 1998 . Learning to learn: introduction and overview // Learning to Learn . New York : Springer: 3 - 17 [ DOI: 10.1007/978-1-4615-5529-2_1 http://dx.doi.org/10.1007/978-1-4615-5529-2_1 ]
Vilalta R and Drissi Y . 2002 . A perspective view and survey of meta-learning . Artificial Intelligence Review , 18 ( 2 ): 77 - 95 [ DOI: 10.1023/A:1019956318069 http://dx.doi.org/10.1023/A:1019956318069 ]
Xia G S , Hu J W , Hu F , Shi B G , Bai X , Zhong Y F , Zhang L P and Lu X Q . 2017 . AID: a benchmark data set for performance evaluation of aerial scene classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 7 ): 3965 - 3981 [ DOI: 10.1109/tgrs.2017.2685945 http://dx.doi.org/10.1109/tgrs.2017.2685945 ]
Yang X , Li F , Lu M , Xin L , Lu X T and Zhang N . 2022 . New super-resolution reconstruction method based on mixed sparse representations . National Remote Sensing Bulletin , 26 ( 8 ): 1685 - 1697
杨雪 , 李峰 , 鹿明 , 辛蕾 , 鲁啸天 , 张南 . 2022 . 混合稀疏表示模型的超分辨率重建 . 遥感学报 , 26 ( 8 ): 1685 - 1697 [ DOI: 10.11834/jrs.20219409 http://dx.doi.org/10.11834/jrs.20219409 ]
Yang Y and Newsam S . 2010 . Bag-of-visual-words and spatial extensions for land-use classification // Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems . San Jose : Association for Computing Machinery: 270 - 279 [ DOI: 10.1145/1869790.1869829 http://dx.doi.org/10.1145/1869790.1869829 ]
Zhang Y L , Li K P , Li K , Wang L C , Zhong B N and Fu Y . 2018a . Image super-resolution using very deep residual channel attention networks // 15th European Conference on Computer Vision . Munich : Springer: 294 - 310 [ DOI: 10.1007/978-3-030-01234-2_18 http://dx.doi.org/10.1007/978-3-030-01234-2_18 ]
Zhang Y L , Tian Y P , Kong Y , Zhong B N and Fu Y . 2018b . Residual dense network for image super-resolution // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 2472 - 2481 [ DOI: 10.1109/CVPR.2018.00262 http://dx.doi.org/10.1109/CVPR.2018.00262 ]
Zhao H Y and Li Y H . 2022 . Super-resolution analysis of Sentinel-2 images using generative adversarial networks . Journal of Computer Applications , 42 ( S1 ): 298 - 304
赵慧岩 , 李云鹤 . 2022 . 基于生成对抗网络的Sentinel-2遥感图像超分辨率分析 . 计算机应用 , 42 ( S1 ): 298 - 304 [ DOI: 10.11772/j.issn.1001-9081.2021060953 http://dx.doi.org/10.11772/j.issn.1001-9081.2021060953 ]
相关作者
相关机构
京公网安备11010802024621
