浏览全部资源
扫码关注微信
纸质出版日期： 2021-12-16 ，
录用日期： 2020-11-04
移动端阅览
李鑫伟, 李彦胜, 张永军. 弱监督深度语义分割网络的多源遥感影像水体检测[J]. 中国图象图形学报, 2021,26(12):3015-3026.
Xinwei Li, Yansheng Li, Yongjun Zhang. Weakly supervised deep semantic segmentation network for water body extraction based on multi-source remote sensing imagery[J]. Journal of Image and Graphics, 2021,26(12):3015-3026.
目的
2
深度语义分割网络的优良性能高度依赖于大规模和高质量的像素级标签数据。在现实任务中，收集大规模、高质量的像素级水体标签数据将耗费大量人力物力。为了减少标注工作量，本文提出使用已有的公开水体覆盖产品来创建遥感影像对应的水体标签，然而已有的公开水体覆盖产品的空间分辨率低且存在一定错误。对此，提出采用弱监督深度学习方法训练深度语义分割网络。
方法
2
在训练阶段，将原始数据集划分为多个互不重叠的子数据集，分别训练深度语义分割网络，并将训练得到的多个深度语义分割网络协同更新标签，然后利用更新后的标签重复前述过程，重新训练深度语义分割网络，多次迭代后可以获得好的深度语义分割网络。在测试阶段，多源遥感影像经多个代表不同视角的深度语义分割网络分别预测，然后投票产生最后的水体检测结果。
结果
2
为了验证本文方法的有效性，基于原始多源遥感影像数据创建了一个面向水体检测的多源遥感影像数据集，并与基于传统的水体指数阈值分割法和基于低质量水体标签直接学习的深度语义分割网络进行比较，交并比（intersection-over-union，IoU）分别提升了5.5%和7.2%。
结论
2
实验结果表明，本文方法具有收敛性，并且光学影像和合成孔径雷达（synthetic aperture radar，SAR）影像的融合有助于提高水体检测性能。在使用分辨率低、噪声多的水体标签进行训练的情况下，训练所得多视角模型的水体检测精度明显优于基于传统的水体指数阈值分割法和基于低质量水体标签直接学习的深度语义分割网络。
Objective
2
Water body detection has shown important applications in flood disaster assessment
water resource value estimation and ecological environment protection based on remote sensing imagery. Deep semantic segmentation network has achieved great success in the pixel-level remote sensing image classification. Water body detection performance can be reasonably expected based on the deep semantic segmentation network. However
the excellent performance of deep semantic segmentation network is highly dependent on the large-scale and high-quality pixel-level labels. This research paper has intended to leverage the existing open water cover products to create water labels corresponding to remote sensing images in order to reduce the workload of labeling and meantime maintain the fair detection accuracy. The existing open water cover products have a low spatial resolution and contain a certain degree of errors. The noisy low-resolution water labels have inevitably affected the training of deep semantic segmentation network for water body detection. A weakly supervised deep learning method to train deep semantic segmentation network have been taken into consideration to resolve the difficulties. The optimization method to train deep semantic segmentation network using the noisy low-resolution labels for the high accuracy of water detection has been presented based on minimizing the manual annotation cost.
Method
2
In the training stage
the original dataset has been divided into several non-overlapped sub-datasets. The deep semantic segmentation network has been trained on each sub-dataset. The trained deep semantic segmentation networks with different sub-datasets have updated the labels simultaneously. As the non-overlapped sub-datasets generally have different data distributions
the detection performance of different networks with different sub-datasets is also complementary. The prediction of the same region by different networks is different
so the multi-perspective deep semantic segmentation network can realize the collaborative update of labels. The updated labels have been used to repeat the above process to re-train new deep semantic segmentation networks. Following each step of iteration
the output of the network has been used as the new labels. The noisy labels have been removed with the iteration process. The range of truth value of the water has also be expanded continuously along with the iteration process. Several good deep semantic segmentation networks can be obtained after a few iterations. In the test stage
the multi-source remote sensing images have been predicted by several deep semantic segmentation networks representing different perspectives and producing the final water detection voting results.
Result
2
The multi-source remote sensing image training dataset
validation dataset and testing dataset have been built up for verification. The multi-source remote sensing imagery has composed of Sentinel-1 SAR (synthetic aperture radar) images and Sentinel-2 optical images. The training dataset has contained 150 000 multi-source remote sensing samples with the size of 256×256 pixels. The labels of the training dataset have been intercepted with the public MODIS (moderate-resolution imaging spectroradiometer) water coverage products in geographic scale. The spatial resolution of the training dataset is low and contains massive noise. The validation dataset has contained 100 samples with the size of 256×256 pixels and the testing dataset have contained 400 samples with the size of 256×256 pixels
and the labels from the validation and testing datasets have accurately annotated with the aid of domain experts. The training
validation and testing datasets have not been overlapped each and the dataset can geographically cover in global scale. Experimental results have shown that the proposed method is convergent
and the accuracy tends to be stable based on four iterations. The fusion of optical and SAR images can improve the accuracy of water body detection. The IoU (intersection over union) has increased by 5.5% compared with the traditional water index segmentation method. The IoU has increases by 7.2% compared with the deep semantic segmentation network directly using the noisy low-resolution water labels.
Conclusion
2
The experimental results have shown that the current method can converge fast
and the fusion of optical and SAR images can improve the detection results. On the premise of the usage of the noisy low-resolution water labels
the water body detection accuracy of the trained multi-perspective model is obviously better than the traditional water index segmentation method and the deep semantic segmentation network based on the direct learning of the noisy low-resolution water labels. The accuracy of the traditional deep semantic segmentation method is slightly lower than that of the traditional water index method
which indicates that the effectiveness of deep learning highly depends on the quality of the training data labels. The noisy low-resolution water labels have reduced the effect of deep learning. The effect of the proposed method on small rivers and lakes has been analyzed. The accuracy on small rivers and lakes has decreased slightly. The result has still higher than the traditional water index method and the deep learning method with the direct training of the noisy low-resolution water labels.
水体检测多源遥感影像低分辨率噪声标签弱监督深度语义分割网络
water body detectionmulti-source remote sensing imagenoisy low-resolution labelsweakly supervised deep semantic segmentation network
Bearman A, Russakovsky O, Ferrari V and Li F F. 2016. What's the point: semantic segmentation with point supervision//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer: 549-565[DOI: 10.1007/978-3-319-46478-7_34http://dx.doi.org/10.1007/978-3-319-46478-7_34]
Carroll M L, Townshend J R, DiMiceli C M, Noojipady P and Sohlberg R A. 2009. A new global raster water mask at 250 m resolution. International Journal of Digital Earth, 2(4): 291-308[DOI:10.1080/17538940902951401]
Chen J B, Liu S X, Wang C Y, You S C and Wang Z W. 2013. Research on urban water body extraction using knowledge-based decision Tree. Remote Sensing Information, 28(1): 29-33, 37
陈静波, 刘顺喜, 汪承义, 尤淑撑, 王忠武. 2013. 基于知识决策树的城市水体提取方法研究. 遥感信息, 28(1): 29-33, 37 [DOI:10.3969/j.issn.1000-3177.2013.01.007]
Chen Q, Zheng L J, Li X J, Xu C B, Wu Y, Xie D H and Liu L. 2019. Water body extraction from high-resolution satellite remote sensing images based on deep learning. Geography and Geo-Information Science, 35(4): 43-49
陈前, 郑利娟, 李小娟, 徐崇斌, 吴俣, 谢东海, 刘亮. 2019. 基于深度学习的高分遥感影像水体提取模型研究. 地理与地理信息科学, 35(4): 43-49 [DOI:10.3969/j.issn.1672-0504.2019.04.007]
Dai J F, He K M and Sun J. 2015. BoxSup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE: 1635-1643[DOI: 10.1109/iccv.2015.191http://dx.doi.org/10.1109/iccv.2015.191]
Ding F. 2009. A new method for fast information extraction of water bodies using remotely sensed data. Remote Sensing Technology and Application, 24(2): 167-171
丁凤. 2009. 一种基于遥感数据快速提取水体信息的新方法. 遥感技术与应用, 24(2): 167-171 [DOI:10.11873/j.issn.1004-0323.2009.2.167]
Duan Q Y, Meng L K, Fan Z W, Hu W G and Xie W J. 2015. Applicability of the water information extraction method based on GF-1 image. Remote Sensing for Land&Resources, 27(4): 79-84
段秋亚, 孟令奎, 樊志伟, 胡卫国, 谢文君. 2015. GF-1卫星影像水体信息提取方法的适用性研究. 国土资源遥感, 27(4): 79-84 [DOI:10.6046/gtzyyg.2015.04.13]
Friedl M A, Mciver D K, Hodges J C F, Zhang X Y, Muchoney D, Strahler A H, Woodcock C E, Gopal S, Schneider A, Cooper A, Baccini A, Gao F and Schaaf C. 2002. Global land cover mapping from MODIS: algorithms and early results. Remote Sensing of Environment, 83(1/2): 287-302[DOI:10.1016/S0034-4257(02)00078-0]
Ghosh A, Kumar H and Sastry P S. 2017. Robust loss functions under label noise for deep neural networks//Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI: 1919-1925
Guo Y M, Liu Y, Georgiou T and Lew M S. 2017. A review of semantic segmentation using deep neural networks. International Journal of Multimedia Information Retrieval, 7(2): 87-93[DOI:10.1007/s13735-017-0141-z]
He H Q, Du J, Chen T and Chen X Y. 2017. Remote sensing image water body extraction combing NDWI with convolutional neural network. Remote Sensing Information, 32(5): 82-86
何海清, 杜敬, 陈婷, 陈晓勇. 2017. 结合水体指数与卷积神经网络的遥感水体提取. 遥感信息, 32(5): 82-86 [DOI:10.3969/j.issn.1000-3177.2017.05.013]
Hung W C, Tsai Y, Liou Y, Lin Y Y and Yang M H. 2018. Adversarial learning for semi-supervised semantic segmentation[EB/OL]. [2020-06-05].https://arxiv.org/pdf/1802.07934.pdfhttps://arxiv.org/pdf/1802.07934.pdf
Kingma D P and Ba J L. 2015. Adam: a method for stochastic optimization[EB/OL]. [2020-06-05].https://arxiv.org/pdf/1412.6980.pdfhttps://arxiv.org/pdf/1412.6980.pdf
Khoreva A, Benenson R, Hosang J, Hein M and Schiele B. 2017. Simple does it: weakly supervised instance and semantic segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE: 1665-1674[DOI: 10.1109/CVPR.2017.181http://dx.doi.org/10.1109/CVPR.2017.181]
Li Y S, Zhang Y J, Huang X, Zhu H and Ma J Y. 2018a. Large-scale remote sensing image retrieval by deep hashing neural networks. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 950-965[DOI:10.1109/TGRS.2017.2756911]
Li Y S, Tao C, Tan Y H, Shang K and Tian J W. 2016. Unsupervised multilayer feature learning for satellite image scene classification. IEEE Geoscience and Remote Sensing Letters, 13(2): 157-161[DOI:10.1109/LGRS.2015.2503142]
Li Y S, Zhang Y J, Huang X and Ma J Y. 2018b. Learning source-invariant deep hashing convolutional neural networks for cross-source remote sensing image retrieval. IEEE Transactions on Geoscience and Remote Sensing, 56(11): 6521-6536[DOI:10.1109/TGRS.2018.2839705]
Li Y S, Zhang Y J, Huang X and Yuille A L. 2018c. Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 146: 182-196[DOI:10.1016/j.isprsjprs.2018.09.014]
Li Y S, Zhang Y J and Zhu Z H. 2021. Error-tolerant deep learning for remote sensing image scene classification. IEEE Transactions on Cybernetics, 51(4): 1756-1768[DOI:10.1109/TCYB.2020.2989241]
Li Y S, Chen W, Zhang Y J, Tao C, Xiao R and Tan Y H. 2020. Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning. Remote Sensing of Environment, 250: #112045[DOI:10.1016/j.rse.2020.112045]
Liao AP, Chen L J, Chen J, He C Y, Cao X, Chen J, Peng S, Sun F D and Gong P. 2014. High-resolution remote sensing mapping of global land water. Science China Earth Sciences, 57(10): 2305-2316
廖安平, 陈利军, 陈军, 何超英, 曹鑫, 陈晋, 彭舒, 孙芳蒂, 宫鹏. 2014. 全球陆表水体高分辨率遥感制图. 中国科学: 地球科学, 44(8): 1634-1645 [DOI:10.1007/s11430-014-4918-0]
Lin D, Dai J F, Jia J Y, He K M and Sun J. 2016. ScribbleSup: scribble-supervised convolutional networks for semantic segmentation//Proceedings of 2016 IEEE Conference on Computer Vision and PatternRecognition (CVPR). Las Vegas, USA: IEEE: 3159-3167[DOI: 10.1109/cvpr.2016.344http://dx.doi.org/10.1109/cvpr.2016.344]
Lu S L, Wu B F, Yan N N and Wang H. 2011. Water body mapping method with HJ-1A/B satellite imagery. International Journal of Applied Earth Observation and Geoinformation, 13(3): 428-434[DOI:10.1016/j.jag.2010.09.006]
Mcfeeters S K. 1996. The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features. International Journal of Remote Sensing, 17(7): 1425-1432[DOI:10.1080/01431169608948714]
Mo W H, Sun H, Zhong S Q, Huang Y L and He L. 2007. Research on the CIWI model and its application. Remote Sensing Information, (5): 16-21
莫伟华, 孙涵, 钟仕全, 黄永璘, 何立. 2007. MODIS水体指数模型(CIWI)研究及其应用. 遥感信息, (5): 16-21 [DOI:10.3969/j.issn.1000-3177.2007.05.004]
Pinheiro P O and Collobert R. 2015. From image-level to pixel-level labeling with convolutional networks//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE: 1713-1721[DOI: 10.1109/CVPR.2015.7298780http://dx.doi.org/10.1109/CVPR.2015.7298780]
Reed S, Lee H, Anguelov D, Szegedy C, Erhan D and Rabinovich A. 2015. Training deep neural networks on noisy labels with bootstrapping[EB/OL]. [2020-06-05].https://arxiv.org/pdf/1412.6596v1.pdfhttps://arxiv.org/pdf/1412.6596v1.pdf
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241[DOI: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]
Schmitt M, Prexl J, Ebel P, Liebel L and Zhu X X. 2020. Weakly supervised semantic segmentation of satellite images for land cover mapping-challenges and opportunities. ISPRS Annals of Photogrammetry, Remote Sensing&Spatial Information Sciences, 5(3): 795-802[DOI:10.5194/isprs-annals-V-3-2020-795-2020]
Souly N, Spampinato C and Shah M. 2017. Semi supervised semantic segmentation using generative adversarial network//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE: 5689-5697[DOI: 10.1109/ICCV.2017.606http://dx.doi.org/10.1109/ICCV.2017.606]
Sukhbaatar S, Bruna J, Paluri M, Bourdev L and Fergus R. 2015. Training convolutional networks with noisy labels[EB/OL]. [2020-06-05].https://arxiv.org/pdf/1406.2080v4.pdfhttps://arxiv.org/pdf/1406.2080v4.pdf
Tan Y H, Xiong S Z and Li Y S. 2018. Automatic extraction of built-up areas from panchromatic and multispectral remote sensing images using double-stream deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(11): 3988-4004[DOI:10.1109/JSTARS.2018.2871046]
Wang X, Sui L C, Zhong M Q, Li D M and Dang L L. 2018. Fully convolution neural networks for water extraction of remote sensing images. Bulletin of Surveying and Mapping, (6): 41-45
王雪, 隋立春, 钟棉卿, 李顶萌, 党丽丽. 2018. 全卷积神经网络用于遥感影像水体提取. 测绘通报, (6): 41-45 [DOI:10.13474/j.cnki.11-2246.2018.0173]
Xu H Q. 2005. A study on information extraction of water body with the modified normalized difference water index (MNDWI). Journal of Remote Sensing, 9(5): 589-595
徐涵秋. 2005. 利用改进的归一化差异水体指数(MNDWI)提取水体信息的研究. 遥感学报, 9(5): 589-595 [DOI:10.11834/jrs.20050586]
Yang Y and Ruan R Z. 2010. Extraction of plain lake water body based on TM imagery. Remote Sensing Information, (3): 60-64
杨莹, 阮仁宗. 2010. 基于TM影像的平原湖泊水体信息提取的研究. 遥感信息, (3): 60-64 [DOI:10.3969/j.issn.1000-3177.2010.03.013]
Yin Y Q, Li J G, Yu T, Yang H Y and Zhang Y H. 2015. The study of object-oriented water body extraction method based on high resolution RS image. Bulletin of Surveying and Mapping, (1): 81-85
殷亚秋, 李家国, 余涛, 杨红艳, 张永红. 2015. 基于高分辨率遥感影像的面向对象水体提取方法研究. 测绘通报, (1): 81-85 [DOI:10.13474/j.cnki.11-2246.2015.0016]
Zhou B L, Khosla A, Lapedriza A, Oliva A and Torralba A. 2016. Learning deep features for discriminative localization//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE: 2921-2929[DOI: 10.1109/CVPR.2016.319http://dx.doi.org/10.1109/CVPR.2016.319]
相关文章
相关作者
相关机构
京公网安备11010802024621