浏览全部资源
扫码关注微信
Published： 16 August 2021 ，
Accepted： 03 November 2020
移动端阅览
Qichao Liu, Liang Xiao, Jinxiang Yang. Parallel implementation of content-guided deep convolutional network for hyperspectral image classification. [J]. Journal of Image and Graphics 26(8):1926-1939(2021)
目的
2
受限于卷积核形状固定，传统卷积神经网络（convolutional neural network，CNN）方法难以精确分类高光谱图像（hyperspectral image，HSI）中的跨类别边缘区域，导致地物边界模糊。内容引导CNN（content-guided CNN，CGCNN）能够根据地物形态自适应调整卷积核形状，具有地物边缘保持分类能力。但由于内容引导卷积属于非固定模板结构，不能直接调用现有深度学习加速库实现并行计算。针对该问题，本文设计了一种内容引导卷积的并行计算方法，并验证其加速及分类性能。
方法
2
本文基于内容引导卷积等价于各向异性核加权和标准卷积的组合结构，通过利用深度学习库中的平铺、堆叠、网格和采样等底层函数构造索引矩阵来定义重采样方式，以将内容引导卷积分解为与空间位置无关的像素级独立计算过程，并在图形处理器（graphics processing unit，GPU）上并行执行。
结果
2
经测试，本文提出的并行化内容引导卷积相比串行运算方式平均提速近700倍。在分类性能测试中，并行化CGCNN在合成数据集上表现出优异的细节保持分类能力，总精度平均高于对比方法7.10%；同时在两组真实数据集上亦取得最优分类结果，分别高于对比方法7.21%、2.70%。
结论
2
通过将内容引导卷积分步拆解，能够将其转化为一系列并行计算过程，且能够在GPU上高效执行；并通过在多组数据集上的分类精度、参数敏感度和小样本学习等综合性能测试进一步表明，并行化CGCNN在具有优良分类性能的同时，亦具有对不同地物的边缘保持分类能力，能够获得更精细的分类结果。
Objective
2
Deep learning-based hyperspectral image (HSI) classification has become an important research point due to its ability to learn adaptive and robust features from training data automatically. However
issues
such as high-dimensionality of spectra
imbalanced and limited labeled samples
and diversity of land cover
persist. Hence
improving the robustness of deep networks under limited samples has become a key research hotpot. Evaluation indices of deep learning methods
such as reproducibility
credibility
efficiency
and accuracy
have also received research attention. Content-guided convolutional neural network (CGCNN) was proposed for HSI classification. Compared with shape-fixed kernels in traditional CNNs
CGCNN can adaptively adjust its kernel shape according to the spatial distribution of land covers; as a result
it can preserve the boundaries and details of land covers in classification maps. In particular
each kernel in content-guided convolution is composed of two kernels
namely
an isotropic kernel used to extract spatial features from HSI and an anisotropic kernel used to control the spatial shape of the isotropic kernel. The isotropic kernel is shared at different sampling regions
similar to regular convolution. However
anisotropic kernels are computed independently according to the local distribution of land covers at different spatial locations. Content-guided convolution cannot be computed in parallel straightforwardly by the deep learning acceleration library (e.g.
Tensorflow and PyTorch) due to the unshared anisotropic kernels. To tackle this issue
we propose a parallel computing method for content-guided convolution to allow the CGCNN to be computed efficiently by a graphics processing unit (GPU) and evaluate its performance in hyperspectral image classification.
Method
2
To make content-guided convolution parallelizable
we break down the content-guided convolution into two steps: weighting the input feature map with the anisotropic kernels
and conducting a standard convolution with the isotropic kernels on the weighted feature map. Although the anisotropic kernels in the content-guided convolution are different at different sampling locations
their sampling grids (receptive fields) are restricted to small windows with a fixed size (e.g.
3×3). If we resample the input feature map into small cubes
in which each cube contains only the needed pixels used to compute the anisotropic kernel weights for its center pixel
then the content-guided convolution can be converted to a pixel-level computation process independent of spatial locations and then be executed on the GPU in parallel. To this end
we propose to construct an index matrix to define the resampling manner and then resample the input feature map to separate the overlapping pixels into non-overlapping pixels by employing the underlying interfaces of the deep learning acceleration library. Finally
a standard convolution is conducted on the weighted feature map to finish the weighting process of the isotropic kernels in the content-guided convolution
which can be easily performed by the deep learning acceleration library. The resampling index matrix is shared in the whole CGCNN and thus only needs to be computed once. The matrix can also be executed efficiently on the GPU because the resampling process is just a data copy operation and does not involve numerical computation.
Result
2
Compared with serial computing
the parallel-accelerated content-guided convolution achieves an average acceleration of nearly 700 times under different conditions. With increasing input scale
the acceleration ratio also increases steadily. As a result
the parallelizable CGCNN can classify the benchmark data sets in seconds. Additionally
in the comparison of classification accuracies
the parallelizable CGCNN shows great ability to preserve details and boundaries compared with other state-of-the-art deep learning-based methods. On the synthetic data set
the classification map acquired by the CGCNN preserves more edges and small objects
while the compared methods lead the classification maps to be smoother. As a result
the overall accuracy of the CGCNN is 7.10% higher than that of the other methods on average. On the two other real data sets (i.e.
Indian Pines and Loukia)
the CGCNN still achieves the best classification results
with classification accuracies that are 7.21% and 2.70% higher on average than the compared methods
respectively. Furthermore
the sensitivity parameter analysis shows the effect and function of the sensitivity value. Smaller sensitivity value can preserve more details and boundaries
but the outliers and noise are also more; in contrast
larger sensitivity value can suppress more outliers and noise
but the classification map may be smoother and lose more details. The automatic learning mechanism of sensitivity value can achieve a trade-off between the details and smoothness and give a suitable sensitivity for different HSIs according to their spatial contents and training samples. In the evaluation of small-sample learning capacity
the performance advantage of the CGCNN increases compared with the other methods gradually with decreasing number of training samples
indicating the great small-sample learning ability of the CGCNN.
Conclusion
2
By breaking down the content-guided convolution into a series of pixel-level computation processes independent of spatial locations
we propose a parallel computing method for the content-guided convolution to run the CGCNN in parallel on the GPU. Diverse experiments such as comparison of classification accuracies
sensitivity parameter analysis
and evaluation of small-sample learning capacity
demonstrate that the CGCNN can greatly preserve the edges and small objects in classification maps and outperforms the compared methods in quantitative and qualitative performance indices.
内容引导卷积深度学习高光谱图像(HSI)分类并行加速边缘保持分类
content-guided convolutiondeep learninghyperspectral image (HSI) classificationparallel accelerationedge-preserving classification
Cao X Y, Zhou F, Xu L, Meng D Y, Xu Z B and Paisley J. 2018. Hyperspectral image classification with Markov random fields and a convolutional neural network. IEEE Transactions on Image Processing, 27(5): 2354-2367[DOI: 10.1109/TIP.2018.2799324]
Chen C, Zhang J J, Zheng C H, Yan Q and Xun L N. 2018. Classification of hyperspectral data using a multi-channel convolutional neural network//Proceedings of International Conference on Intelligent Computing. Wuhan, China: Springer: 81-92[DOI: 10.1007/978-3-319-95957-3_10http://dx.doi.org/10.1007/978-3-319-95957-3_10]
Chen Y, Nasrabadi N M and Tran T D. 2011. Hyperspectral image classification using dictionary-based sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 3973-3985[DOI: 10.1109/TGRS.2011.2129595]
Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107[DOI: 10.1109/JSTARS.2014.2329330]
Chen Z, Wang B and Zhang L M. 2013. Dimensionality reduction and classification based on lower rank tensor analysis for hyperspectral imagery. Journal of Infrared and Millimeter Waves, 32(6): 569-575
陈昭, 王斌, 张立明. 2013. 基于低秩张量分析的高光谱图像降维与分类. 红外与毫米波学报, 32(6): 569-575 [DOI: 10.3724/SP.J.1010.2013.00569]
Fang B, Li Y, Zhang H K and Chan J C W. 2019. Hyperspectral images classification based on dense convolutional networks with spectral-wise attention mechanism. Remote Sensing, 11(2): #159[DOI: 10.3390/rs11020159]
Fernandez-Redondo M, Hernandez-Espinosa C and Torres-Sospedra J. 2004. Hyperspectral image classification by ensembles of multilayer feedforward networks//Proceedings of 2004 IEEE International Joint Conference on Neural Networks. Budapest, Hungary: IEEE: 1145-1149[DOI: 10.1109/IJCNN.2004.1380097http://dx.doi.org/10.1109/IJCNN.2004.1380097]
Howard A G, Zhu M L, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M and Adam H. 2017. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL].https://arxiv.org/pdf/1704.04861.pdfhttps://arxiv.org/pdf/1704.04861.pdf
Hu W, Huang Y Y, Li W, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: #258619[DOI: 10.1155/2015/258619]
Huang G, Liu Z, van der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE: 2261-2269[DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]
Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): #582[DOI: 10.3390/rs12030582]
Li T, Sun J G, Zhang X J and Wang X. 2016. Spectral-spatial joint classification method of hyperspectral remote sensing image. Chinese Journal of Scientific Instrument, 37(6): 1379-1389
李铁, 孙劲光, 张新君, 王星. 2016. 高光谱遥感图像空谱联合分类方法研究. 仪器仪表学报, 37(6): 1379-1389 [DOI: 10.3969/j.issn.0254-3087.2016.06.023]
Li Y, Zhang H K and Shen Q. 2017. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1): #67[DOI: 10.3390/rs9010067]
Liu Q C, Xiao L, Liu F and Xu J H. 2020. SSCDenseNet: a spectral-spatial convolutional dense network for hyperspectral image classification. Acta Electronica Sinica, 48(4): 751-762
刘启超, 肖亮, 刘芳, 徐金环. 2020. SSCDenseNet: 一种空-谱卷积稠密网络的高光谱图像分类算法. 电子学报, 48(4): 751-762 [DOI: 10.3969/j.issn.0372-2112.2020.04.017]
Liu Q C, Xiao L, Yang J X and Chan J C W. 2020. Content-guided convolutional neural network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(9): 6124-6137[DOI: 10.1109/TGRS.2020.2974134]
Makantasis K, Karantzalos K, Doulamis A and Doulamis N. 2015. Deep supervised learning for hyperspectral data classification through convolutional neural networks//2015 IEEE International Geoscience and Remote Sensing Symposium. Milan, Italy: IEEE: 4959-4962[DOI: 10.1109/IGARSS.2015.7326945http://dx.doi.org/10.1109/IGARSS.2015.7326945]
Mei X G, Pan E T, Ma Y, Dai X B, Huang J, Fan F, Du Q L, Zheng H and Ma J Y. 2019. Spectral-spatial attention networks for hyperspectral image classification. Remote Sensing, 11(8): #963[DOI: 10.3390/rs11080963]
Mou L C, Ghamisi P and Zhu X X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3639-3655[DOI: 10.1109/TGRS.2016.2636241]
Qin A Y, Shang Z W, Tian J Y, Wang Y L, Zhang T P and Tang Y Y. 2019. Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 16(2): 241-245[DOI: 10.1109/LGRS.2018.2869563]
Roy S K, Krishna G, Dubey S R and Chaudhuri B B. 2020. HybridSN: exploring 3-D——2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2): 277-281[DOI: 10.1109/LGRS.2019.2918719]
Wan S, Gong C, Zhong P, Du B, Zhang L F and Yang J. 2020. Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(5): 3162-3177[DOI: 10.1109/TGRS.2019.2949180]
Wang L G and Wei F J. 2013. Band selection for hyperspectral imagery based on combination of genetic algorithm and ant colony algorithm. Journal of Image and Graphics, 18(2): 235-242
王立国, 魏芳洁. 2013. 结合遗传算法和蚁群算法的高光谱图像波段选择. 中国图象图形学报, 18(2): 235-242 [DOI: 10.11834/jig.20130216]
Wang W J, Dou S G, Jiang Z M and Sun L J. 2018. A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sensing, 10(7): #1068[DOI: 10.3390/rs10071068]
Yang J X, Zhao Y Q and Chan J C W. 2017. Learning and transferring deep joint spectral-spatial features for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing, 55(8): 4729-4742[DOI: 10.1109/TGRS.2017.2698503]
Zhang H K, Li Y and Jiang Y N. 2018. Deep learning for hyperspectral imagery classification: the state of the art and prospects. Acta Automatica Sinica, 44(6): 961-977
张号逵, 李映, 姜晔楠. 2018. 深度学习在高光谱图像分类领域的研究现状与展望. 自动化学报, 44(6): 961-977 [DOI: 10.16383/j.aas.2018.c170190]
Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858[DOI:10.1109/TGRS.2017.2755542]
相关文章
相关作者
相关机构
京公网安备11010802024621