收稿：2019-03-14，
录用：2019-5-30，
纸质出版：2019-07
移动端阅览
近年来，遥感应用的快速发展推动了遥感载荷指标性能的不断提升。但由于遥感传感器的硬件技术瓶颈，遥感数据无法同时具有高空间分辨率、高光谱分辨率、高时间分辨率的指标特性。遥感数据融合是解决该问题的有效方法。为了深入了解目前遥感数据融合技术的研究进展情况，本文对国内外1992年—2018年间在该领域有一定影响力的相关成果进行了调研、分析与归纳总结。首先对遥感数据融合相关论文的年发文量、发文国家与机构、发表刊物以及关键词等进行了统计，梳理其发展历史及趋势；系统性的总结了各类数据融合算法，将其分为面向空间维提升的融合算法、面向光谱维提升的融合算法以及面向时间维提升的融合算法3类，并对各类算法的优势与适用性进行了分析；归纳总结了遥感数据融合的质量评价指标，包括有参考影像的融合评价指标以及无参考影像的融合评价指标；最后对遥感数据融合进展进行了总结与展望。
Remote sensing applications have been promoting the development of satellite sensors and their performances. The satellite sensors are now possessing higher and higher data resolutions
and it in turn greatly facilitating remote sensing applications. The improvement of spatial resolution makes it possible to precisely record textures and spatial features of land covers. The improvement of spectral resolution makes it possible to precisely classify and retrieve parameters. The improvement of temporal resolution makes it easier to record the temporal changes of land covers during different phases. However
due to the limitation of sensors
satellite data can not possess high spatial resolution
high temporal resolution and high spectral resolution at the same time
which greatly hinders further remote sensing applications. Remote sensing data fusion is one of the effective solutions to deal with the problem of limitation in sensors’ resolutions
which is to integrate data from different sources and with different resolutions using algorithm methods to get richer information than one single image data. In recent years
the remote sensing data fusion methods have been greatly developed and related articles boost in growing numbers and importance. Thus
this article aims to systematically introduce the remote sensing data fusion and its current progress. Former reviews of remote sensing data fusion divides fusion methods into different categories. Based on processing levels
the remote sensing data fusion methods can be divided into pixel-level fusion
feature-level fusion and decision-level fusion. And based on the data sources
the remote sensing data fusion methods can be divided into homogeneous data fusion
heterogeneous data fusion
fusion for remote sensing observation and station data
and fusion for remote sensing observation and non-observed data. And this article adopts another category system which makes it more systematic and comprehensive in mathematical principles. In order to systematically indicate the current progress and developed history of remote sensing data fusion methods
this article first did bibliometric analysis on the remote sensing data fusion articles from data sources from Web of Science (WOS) and Chinese National Knowledge Infrastructure (CNKI). The data was preprocessed and has removed duplicates. Analysis including the aspects of yearly publishments
countries and organizations
journals and key words to see the history and trends have been done by Histcite
Bibliometric
Citespace and Endnote. Then the remote sensing fusion methods are systematically introduced. According to different resolution improvements
this article firstly divided the fusion methods into three categories: fusion methods emphasizing improvements of spatial resolution development
spectral development and temporal development
and based on that we further divide them into subcategories according to different mathematical principles. And the basic principles
some of the important methods
the advantages and flaws of these methods are systematically introduced in this section. Also
we did detailed instructions on fusion result assessment methods including methods with reference image and without reference image. Under these two categories we further introduced different assessment metrics and its applicable scenarios. And finally
this article made summaries of these fusion methods and described its features and directions. And finally makes predictions of future trends in remote sensing data fusion methods.
Adams J B, Smith M O and Johnson P E. 1986. Spectral mixture modeling: a new analysis of rock and soil types at the Viking Lander 1 sites. Journal of Geophysical Research: Solid Earth, 91(B8): 8098–8112
Aiazzi B, Baronti S and Selva M. 2007. Improving component substitution pansharpening through multivariate regression of MS+Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3230–3239
Alparone L, Aiazzi B, Baronti S, Garzelli A, Nencini F and Selva M. 2008. Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering and Remote Sensing, 74(2): 193–200
Alparone L, Wald L, Chanussot J, Thomas C, Gamba P and Bruce L M. 2007. Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3012–3021
Amorós-López J, Gómez-Chova L, Alonso L, Guanter L, Zurita-Milla R, Moreno J and Camps-Valls G. 2013. Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring. International Journal of Applied Earth Observation and Geoinformation, 23: 132–141
Azarang A and Ghassemian H. 2017. A new pansharpening method using multi resolution analysis framework and deep neural networks//Proceedings of the 3rd International Conference on Pattern Recognition and Image Analysis. Shahrekord, Iran: IEEE: 1-6 [DOI: 10.1109/PRIA.2017.7983017]
Bendoumi M A, He M Y and Mei S H. 2014. Hyperspectral image resolution enhancement using high-resolution multispectral image based on spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 52(10): 6574–6583
Chang C I. 1999. Spectral information divergence for hyperspectral image analysis//Proceedings of 1999 International Geoscience and Remote Sensing Symposium. Hamburg, Germany: IEEE [DOI: 10.1109/IGARSS.1999.773549]
Chavez P C Jr, Sides S C and Anderson J A. 1991. Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogrammetric Engineering and Remote Sensing, 57(3): 295–303
Chen Z, Pu H Y, Wang B and Jiang G M. 2014. Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods. IEEE Geoscience and Remote Sensing Letters, 11(8): 1418–1422
陈仲新, 任建强, 唐华俊, 史云, 冷佩, 刘佳, 王利民, 吴文斌, 姚艳敏, 哈斯图亚. 2016. 农业遥感研究应用进展与展望. 遥感学报, 20(5): 748–767
Chen Z X, Ren J Q, Tang H J, Shi Y, Leng P, Liu J, Wang L M, Wu W B, Yao Y M and Hasiyuya. 2016. Progress and perspectives on agricultural remote sensing research and applications in China. Journal of Remote Sensing, 20(5): 748–767
Choi M. 2006. A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Transactions on Geoscience and Remote Sensing, 44(6): 1672–1682
Do M N and Vetterli M. 2005. The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12): 2091–2106
Eismann M T and Hardie R C. 2003. Resolution enhancement of hyperspectral imagery using coincident panchromatic imagery and a stochastic mixing model//Proceedings of IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data. Greenbelt, MD, USA: IEEE: 282-289 [DOI: 10.1109/WARSD.2003.1295206]
Eismann M T and Hardie R C. 2005. Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 455–465
Elad M, Figueiredo M A T and Ma Y. 2010. On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 98(6): 972–982
Fu D J, Chen B Z, Wang J, Zhu X L and Hilker T. 2013. An improved image fusion approach based on enhanced spatial and temporal the adaptive reflectance fusion model. Remote Sensing, 5(12): 6346–6360
Gao F, Masek J, Schwaller M and Hall F. 2006. On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance. IEEE Transactions on Geoscience and Remote Sensing, 44(8): 2207–2218
Ge Z R, Wang B and Zhang L M. 2007. Remote sensing image fusion based on Bayesian linear estimation. Science in China Series F: Information Sciences, 50(2): 227–240
Ghassemian H. 2016. A review of remote sensing image fusion methods. Information Fusion, 32: 75–89
Gross H N and Schott J R. 1998. Application of spectral mixture analysis and image fusion techniques for image sharpening. Remote Sensing of Environment, 63(2): 85–94
Han C, Zhang H Y, Gao C X, Jiang C, Sang N and Zhang L P. 2016. A remote sensing image fusion method based on the analysis sparse model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(1): 439–453
Hardie R C, Eismann M T and Wilson G L. 2004. MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing, 13(9): 1174–1184
Hilker T, Wulder M A, Coops N C, Linke J, McDermid G, Masek J G, Gao F and White J C. 2009. A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sensing of Environment, 113(8): 1613–1627
Huang B and Song H H. 2012. Spatiotemporal reflectance fusion via sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 50(10): 3707–3716
Huang B, Song H H, Cui H B, Peng J G and Xu Z B. 2014. Spatial and spectral image fusion using sparse matrix factorization. IEEE Transactions on Geoscience and Remote Sensing, 52(3): 1693–1704
Huang B, Wang J, Song H H, Fu D J and Wong K. 2013. Generating high spatiotemporal resolution land surface temperature for urban heat island monitoring. IEEE Geoscience and Remote Sensing Letters, 10(5): 1011–1015
Huang W, Xiao L, Wei Z H, Liu H Y and Tang S Z. 2015. A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5): 1037–1041
Huang Z, Yu X C, Wang G A and Wang Z N. 2008. Application of several non-negative matrix factorization-based methods in remote sensing image fusion//Proceedings of the Fifth International Conference on Fuzzy Systems and Knowledge Discovery. Shandong, China: IEEE [DOI: 10.1109/FSKD.2008.439]
Huynh-Thu Q and Ghanbari M. 2008. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13): 800–801
Jagalingam P and Hegde A V. 2015. A review of quality metrics for fused image. Aquatic Procedia, 4: 133–142
Li D, Hao M F, Zhang J Q, Hu B and Lu Q Y. 2012. A universal hypercomplex color image quality index//Proceedings of 2012 IEEE International Instrumentation and Measurement Technology Conference. Graz, Austria: IEEE: 985-990 [DOI: 10.1109/I2MTC.2012.6229639]
Li S T and Yang B. 2011. A new pan-sharpening method using a compressed sensing technique. IEEE Transactions on Geoscience and Remote Sensing, 49(2): 738–746
Liao C H, Wang J F, Pritchard I, Liu J G and Shang J L. 2017. A spatio-temporal data fusion model for generating NDVI time series in heterogeneous regions. Remote Sensing, 9(11): 1125
Liao L M, Song J L, Wang J D, Xiao Z Q and Wang J. 2016. Bayesian method for building frequent Landsat-like NDVI datasets by integrating MODIS and Landsat NDVI. Remote Sensing, 8(6): 452
Liu J G. 2000. Smoothing filter-based intensity modulation: a spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18): 3461–3472
Liu X, Deng C W, Wang S G, Huang G B, Zhao B J and Lauren P. 2016. Fast and accurate spatiotemporal fusion based upon extreme learning machine. IEEE Geoscience and Remote Sensing Letters, 13(12): 2039–2043
Lu M, Chen J, Tang H J, Rao Y H, Yang P and Wu W B. 2016. Land cover change detection by integrating object-based data blending model of Landsat and MODIS. Remote Sensing of Environment, 184: 374–386
Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2016. Pansharpening by convolutional neural networks. Remote Sensing, 8(7): 594
Moosavi V, Talebi A, Mokhtari M H, Shamsi S R F and Niazi Y. 2015. A wavelet-artificial intelligence fusion approach (WAIFA) for blending Landsat and MODIS surface temperature. Remote Sensing of Environment, 169: 243–254
Nencini F, Garzelli A, Baronti S and Alparone L. 2007. Remote sensing image fusion using the curvelet transform. Information Fusion, 8(2): 143–156
Nunez J, Otazu X, Fors O, Prades A, Palà V and Arbiol R. 1999. Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37(3): 1204–1211
Palsson F, Sveinsson J R and Ulfarsson M O. 2017. Multispectral and hyperspectral image fusion using a 3-D-convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 14(5): 639–643
Pohl C and Van Genderen J L. 1998. Review article Multisensor image fusion in remote sensing: concepts, methods and applications. International Journal of Remote Sensing, 19(5): 823–854
Schmitt M and Zhu X X. 2016. Data fusion and remote sensing: an ever-growing relationship. IEEE Geoscience and Remote Sensing Magazine, 4(4): 6–23
Shettigara V K. 1992. A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogrammetric Engineering and Remote Sensing, 58(5): 561–567
Song H H, Liu Q S, Wang G J, Hang R L and Huang B. 2018. Spatiotemporal satellite image fusion using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 821–829
Sun X J, Zhang L F, Yang H, Wu T X, Cen Y and Guo Y. 2015. Enhancement of spectral resolution for remotely sensed multispectral image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(5): 2198–2211
Tan Z Y, Yue P, Di L P and Tang J M. 2018. Deriving high spatiotemporal remote sensing images using deep convolutional network. Remote Sensing, 10(7): 1066
唐华俊, 吴文斌, 杨鹏, 周清波, 陈仲新. 2010. 农作物空间格局遥感监测研究进展. 中国农业科学, 43(14): 2879–2888
Tang H J, Wu W B, Yang P, Zhou Q B and Chen Z X. 2010. Recent progresses in monitoring crop spatial patterns by using remote sensing technologies. Scientia Agricultura Sinica, 43(14): 2879–2888
Thomas C and Wald L. 2006. Analysis of changes in quality assessment with scale//Proceedings of the 9th International Conference on Information Fusion. Florence, Italy: IEEE [DOI: 10.1109/ICIF.2006.301595]
Tong Q X, Xue Y Q and Zhang L F. 2014. Progress in hyperspectral remote sensing science and technology in China over the past three decades. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(1): 70–91
Tu T M, Lee Y C, Chang C P and Huang P S. 2005. Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/QuickBird imagery. Optical Engineering, 44(11): 116201
Tu T M, Su S C, Shyu H C and Huang P S. 2001. A new look at IHS-like image fusion methods. Information Fusion, 2(3): 177–186
Wald L, Ranchin T, Mangolini M. 1997. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing, 63(6): 691–699
Wang J and Huang B. 2017. A rigorously-weighted spatiotemporal fusion model with uncertainty analysis. Remote Sensing, 9(10): 990
王建宇, 王跃明, 李春来. 2010. 高光谱成像系统的噪声模型和对辐射灵敏度的影响. 遥感学报, 14(4): 607–620
Wang J Y, Wang Y M and Li C L. 2010. Noise model of hyperspectral imaging system and influence on radiation sensitivity. Journal of Remote Sensing, 14(4): 607–620
Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600–612
Wei J B, Wang L Z, Liu P, Chen X D, Li W and Zomaya A Y. 2017a. Spatiotemporal fusion of MODIS and Landsat-7 reflectance images via compressed sensing. IEEE Transactions on Geoscience and Remote Sensing, 55(12): 7126–7139
Wei J B, Wang L Z, Liu P and Song W J. 2017b. Spatiotemporal fusion of remote sensing images with structural sparsity and semi-coupled dictionary learning. Remote Sensing, 9(1): 21
Wei Q, Bioucas-Dias J, Dobigeon N and Tourneret J Y. 2015. Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3658–3668
Wei Q, Dobigeon N and Tourneret J Y. 2014a. Bayesian fusion of hyperspectral and multispectral images//Proceedings of 2014 IEEE International Conference on Acoustics, Speech and Signal Processing. Florence, Italy: IEEE: 3176-3180 [DOI: 10.1109/ICASSP.2014.6854186]
Wei Q, Dobigeon N and Tourneret J Y. 2014b. Bayesian fusion of multispectral and hyperspectral images with unknown sensor spectral response//Proceedings of 2014 IEEE International Conference on Image Processing. Paris, France: IEEE: 698-702 [DOI: 10.1109/ICIP.2014.7025140]
Wei Y C, Yuan Q Q, Shen H F and Zhang L P. 2017. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geoscience and Remote Sensing Letters, 14(10): 1795–1799
Weng Q H, Fu P and Gao F. 2014. Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data. Remote Sensing of Environment, 145: 55–67
Wu B, Huang B and Zhang L P. 2015. An error-bound-regularized sparse coding for spatiotemporal reflectance fusion. IEEE Transactions on Geoscience and Remote Sensing, 53(12): 6791–6803
Wu M Q, Huang W J, Niu Z and Wang C Y. 2015. Generating daily synthetic Landsat imagery by combining Landsat and MODIS data. Sensors, 15(9): 24002–24025
Wu M Q, Niu Z, Wang C Y, Wu C Y and Wang L. 2012. Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model. Journal of Applied Remote Sensing, 6(1): 063507
Wu P H, Shen H F, Zhang L P and Göttsche F M. 2015. Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature. Remote Sensing of Environment, 156: 169–181
Xue J, Leung Y and Fung T. 2017. A Bayesian data fusion approach to spatio-temporal fusion of remotely sensed images. Remote Sensing, 9(12): 1310
Yokoya N, Yairi T and Iwasaki A. 2012. Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50(2): 528–537
Yuan Q Q, Wei Y C, Meng X C, Shen H F and Zhang L P. 2018. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 978–989
Yusuf Y, Sumantyo J T S and Kuze H. 2013. Spectral information analysis of image fusion data for remote sensing applications. Geocarto International, 28(4): 291–310
张良培, 沈焕锋. 2016. 遥感数据融合的进展与前瞻. 遥感学报, 20(5): 1050–1061
Zhang L P and Shen H F. 2016. Progress and future of remote sensing data fusion. Journal of Remote Sensing, 20(5): 1050–1061
Zhang W, Li A N, Jin H A, Bian J H, Zhang Z J, Lei G B, Qin Z H and Huang C Q. 2013. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data. Remote Sensing, 5(10): 5346–5368
Zhang Y F. 2012. Wavelet-based Bayesian fusion of multispectral and hyperspectral images using Gaussian scale mixture model. International Journal of Image and Data Fusion, 3(1): 23–37
Zhang Y F, De Backer S and Scheunders P. 2009. Noise-resistant wavelet-based Bayesian fusion of multispectral and hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 47(11): 3834–3843
Zhu X L, Cai F Y, Tian J Q and Williams T K A. 2018. Spatiotemporal fusion of multisource remote sensing data: literature survey, taxonomy, principles, applications, and future directions. Remote Sensing, 10(4): 527
Zhu X L, Chen J, Gao F, Chen X H and Masek J G. 2010. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sensing of Environment, 114(11): 2610–2623
Zhukov B, Oertel D, Lanzl F and Reinhackel G. 1999. Unmixing-based multisensor multiresolution image fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3): 1212–1226
Zurita-Milla R, Clevers J G P W and Schaepman M E. 2008. Unmixing-based Landsat TM and MERIS FR data fusion. IEEE Geoscience and Remote Sensing Letters, 5(3): 453–457
相关作者
相关机构
京公网安备11010802024621
