目的 全色图像的空间细节信息增强和多光谱图像的光谱信息保持通常是相互矛盾的,如何能够在这对矛盾中实现最佳融合效果一直以来都是遥感图像融合领域的研究热点与难点。为了有效结合光谱信息与空间细节信息,进一步改善多光谱与全色图像的融合质量,提出了一种基于形态学滤波和改进脉冲耦合神经网络(PCNN)的NSST域多光谱与全色图像融合方法。方法 该方法首先分别对多光谱和全色图像进行非下采样剪切波变换(NSST)；对二者的低频分量采用形态学滤波和高通调制框架HPM进行融合,将全色图像低频子带图像的细节信息注入到多光谱图像低频子带图像中得到融合后的低频子带图像；对高频子带图像采用改进脉冲耦合神经网络的方法进行融合,进一步提高融合图像的空间细节信息；最后通过NSST逆变换得到融合图像。结果 仿真实验表明,本文方法在融合效果上优势明显,各项指标与其他方法相比整体上较优,且多波段指标平均值为各比较方法中最优,相比于五种方法中各指标的最优值,本文方法得到融合图像的清晰度比PCA方法提高2.8%,信息熵和相关系数分别比基于NSST与PCNN的方法提高0.4%和4.8%,空间频率比PCA方法提高0.3%,光谱扭曲度比基于NSST与PCNN的方法降低了11.5%。结论 分析各项评价指标可以看出,本文方法在提高融合结果的空间分辨率的同时,很好的保持了光谱信息。综合来看,本文方法在主观与客观方面的总体效果均要优于基于HIS变换、PCA变换、CNMF变换、基于NSCT与PCNN以及基于NSST与PCNN五种现有的流行方法。
Multispectral and panchromatic image fusion based on morphological filter and improved PCNN in NSST domain
Jiao Jiao,Wu Lingda(Science and Technology on Complex Electronic System Simulation Laboratory,Space Engineering University,Beijing)
Objective Nowadays a variety of remote sensing sensors exist, and the multi-source remote sensing images can be achieved, such as the multispectral (MS) image and the panchromatic (PAN) image. The MS image has rich spectral information and high spectral resolution, but the spatial resolution is low, while the PAN image has more spatial details and higher spatial resolution. The significance of the fusion of the MS and the PAN images is to improve the spatial resolution of the MS image while maintaining the spectral information, combining the shape structure of the features in the PAN image and the spectral information in the MS image to provide stronger interpreting capabilities and more reliable results, and to improve the classification accuracy of objects and the detection accuracy of the target. However, the enhancement of spatial resolution of the PAN image and the maintenance of the spectral information of the MS image is usually contradictory. How to achieve better fusion result in these contradictions has always been a hot and difficult point in the research field of remote sensing image fusion and has an extensive prospect in research and application. In order to combine the spectral information and spatial details to further improve the fusion quality of the MS image and the PAN image effectively, a fusion method based on morphological filter and improved pulse coupled neural network (PCNN) in NSST domain is proposed. Method The proposed method is conducted on the MS and the PAN images that have been accurately registered. Firstly, the PAN and the MS images are decomposed by NSST respectively to obtain the low-frequency and the high-frequency sub-bands coefficients; secondly, the low-frequency sub-bands are the approximate sub-graphs of the original image, which inheriting the overall characteristics, and still have some edges and detailed information, so the fusion rule of the low-frequency coefficients based on morphological filtering and high-pass modulation scheme is proposed. The morphological half-gradient operator is used to extract the details of the low-frequency sub-bands of the PAN image because of its preliminary encouraging fusion results on the remote sensing images. Low-resolution PAN sub-band image can be obtained by a morphological filtering, and the detailed PAN sub-band image is estimated by subtracting the low-resolution PAN sub-band image from the PAN sub-band image which equalized with the MS sub-band image, and then the spatial details are injected into the low-frequency sub-band of the MS image through the high-pass modulation scheme. For the fusion of the high-frequency sub-bands, an improved PCNN is taken to enhance the image details. The existing PCNN model usually adopts a hard-limiting function as output, and the firing output is zero or one, which cannot reflect the amplitude difference of the synchronous pulse excitation well. So a soft-limiting Sigmoid function is adopted to calculate the firing output amplitude during the iterations, and the decision matrix for the selection of the high-frequency coefficients can be achieved by summing up the firing output amplitude in the iterative process; thirdly, The fusion low-frequency coefficients and the fusion high-frequency coefficients are reconstructed with the inverse NSST to obtain the final fusion image. Result A series of simulation experiments is conducted to verify the superiority and validity of the proposed fusion method. Two groups of LANDSET TM and SPOT 4 remote sensing images are utilized to test the proposed method. The performance evaluation of the remote sensing image fusion methods includes both the subjective visual effect and objective standard evaluation. Visual analysis is the most immediate detection method. At the same time, in order to evaluate the fusion results quantitatively and objectively, five objective evaluation indicators are selected, including image clarity, information entropy, correlation coefficient, spatial frequency and spectral distortion. Experimental results show that the proposed method has obvious advantages in the fusion effect. The subjective visual effect of the proposed method is obviously better than the other five methods. Details such as image textures and edges are clear and the spectral information is maintained well, and through being compared with other methods of fusion, the objective evaluation indicators of the proposed method also have great superiority. The average value of five indicators of three bands in the MS image is the best among the comparison methods. Compared with the best indicator of the other five methods, the image clarity of the proposed method is 2.8% higher than that of PCA, and the information entropy and the correlation coefficient are improved by 0.4% and 4.8% respectively compared with the method based on NSST and PCNN, and the spatial frequency is improved by 0.3% compared with PCA. The degree of spectral distortion is 11.5% lower than the method based on NSST and PCNN. Conclusion A fusion method of the MS and the PAN images based on morphological operator and improved PCNN in NSST domain is proposed. Based on the NSST decomposition of the original MS and PAN images, we investigate the fusion rules for different frequency bands, and propose fusion rules based on the morphological half-gradients filtering and the high pass modulation scheme for the low-frequency coefficients, and fusion rules based on the improved PCNN for the high-frequency coefficients. Real satellite dataset is employed for the performance evaluation of the proposed method. Analysing the evaluation indicators can be seen that our method can effectively improve the spatial resolution of the fusion results while maintaining the spectral information well. In general, the proposed method is superior to the traditional HIS and PCA method and some current popular fusion methods from the overall effect of the visual aspects and the objective indicators.