Semantic segmentation benchmark dataset for coastal ecosystem monitoring based on unmanned aerial vehicle （UAV）

Wang Shengke; Wang Xiandong; Qu Liang; Yao Fengqin; Liu Yingying; Li Conghui; Wang Yuzhen; Zhong Guoqiang

doi:10.11834/jig.230670

Fusion and Intelligent Interpretation for Multi-source Remote Sensing Data | Views : 0 下载量: 573 CSCD: 0

PDF
Export
Share
Collection
Album

Semantic segmentation benchmark dataset for coastal ecosystem monitoring based on unmanned aerial vehicle （UAV）
Vol. 29, Issue 8, Pages: 2162-2174(2024)
Received：28 September 2023，

Revised：10 January 2024，

Published：16 August 2024
DOI： 10.11834/jig.230670
稿件说明：

移动端阅览

王胜科，王贤栋，曲亮，姚凤芹，刘莹莹，李聪慧，王玉瑱，仲国强. 2024. 面向无人机海岸带生态系统监测的语义分割基准数据集. 中国图象图形学报， 29(08):2162-2174 DOI： 10.11834/jig.230670.

Wang Shengke， Wang Xiandong， Qu Liang， Yao Fengqin， Liu Yingying， Li Conghui， Wang Yuzhen， Zhong Guoqiang. 2024. Semantic segmentation benchmark dataset for coastal ecosystem monitoring based on unmanned aerial vehicle （UAV）. Journal of Image and Graphics， 29(08):2162-2174 DOI： 10.11834/jig.230670.

摘要

目的

海岸带生态系统的监测是维护生态平衡和可持续发展的重要任务。语义分割技术在遥感影像中的应用为海岸带生态系统的精准监测提供了有效手段。然而，目前该领域仍面临一个挑战，即缺乏公开的专门面向海岸带生态系统的语义分割基准数据集。

方法

受到自然环境和人为因素的共同影响，海岸带外部形态变化快，目前的卫星遥感和常规的人工测量、船载作业等监测方式已难以满足实时观测的要求。因此本文首先利用无人机对海岸带典型生态群落进行拍摄、采集和标注，构建了面向海岸带生态系统的基准数据集，名为OUC-UAV-SEG。接着，通过统计的方法，对OUC-UAV-SEG进行定量分析，并对数据集中存在的挑战进行了深入探讨。最后，使用典型视觉语义分割算法对OUC-UAV-SEG展开评估。

结果

对OUC-UAV-SEG数据集进行了基准测试，评估了当前可用的语义分割算法在该数据集上的性能，并揭示了这些算法的局限性。

结论

OUC-UAV-SEG数据集的建立为海岸带生态系统的监测提供了新的资源，有助于科学家、生态学家和决策者们更好地了解海岸带生态系统的现状和变化趋势，为海洋生态保护和管理提供更精确的信息。该数据集链接：

https://github.com/OucCVLab/OUC-UAV-SEG

。

Abstract

Objective

The coastal ecosystem is a natural system composed of biological communities and their interactions with the environment， including typical coastal ecosystems such as mangroves， salt marshes， coral reefs， seagrass beds， oyster reefs， sandy shores， as well as complex ecosystems like estuaries and bays. These ecosystems play a crucial role in maintaining high-quality ecological environments and fostering rich marine biodiversity. A healthy coastal ecosystem is not only a crucial support for the sustainable economic development of China but also an essential component of the ecological security of the country.The application of semantic segmentation techniques in remote sensing imagery has provided an effective means for the precise monitoring of coastal ecosystems， which offers scientists， ecologists， and decision makers clear and highly comprehensive information to understand the current state and changing trends of coastal ecosystems. However， a significant challenge persists， that is， a specialized， comprehensive， and fine-grained data support system for coastal ecosystems is lacking， which causes difficulty in accurately understanding the distribution， area， and changes in ecosystems such as salt marshes， seagrass beds， and reed beds. This challenge has become a pressing issue in the current national marine ecological conservation efforts. Currently， the monitoring of coastal ecosystems relies primarily on satellite remote sensing and traditional surveying methods. Satellite remote sensing， with its unique advantages of all-weather， all-day， large-scale， and long-time observation， is widely used for monitoring marine ecology and resources through the analysis of satellite data. However， the spatial resolution of satellite remote sensing images has limitations， which introduces errors in cases that require fine-scale monitoring. For example， in narrow rivers， small wetland areas， or islands， the limited spatial resolution may result in unclear visibility of small features in the images. Traditional surveying methods often require professionals to conduct field surveys， which poses safety hazards in complex and risky environments. In addition， these methods are susceptible to human and natural factors， which leads to challenges in precise positioning and depiction， coupled with extended and inefficient monitoring cycles. Field surveys and mapping work for coastal ecosystems face significant challenges using traditional methods.

Method

This study utilized unmanned aerial vehicles （UAVs） to capture， collect， and annotate data from typical coastal ecosystems in real time for addressing urgent issues in marine ecological conservation. This effort led to the establishment of the OUC-UAV-SEG dataset， which includes various typical vegetation types found in coastal ecosystems， such as reed， seagrass beds， spartin， and crucially， addresses a key marine event—oil spills.

Result

In contrast to previous research that predominantly focused on individual dataset scenarios， such as spartin or oil spills， using traditional remote sensing methods for data analysis， this study stands out by using statistical methods to conduct a detailed quantitative analysis of OUC-UAV-SEG， which covers various categories and their respective quantities. In discussing the challenges posed by the dataset， including the discrete clustered morphology of eelgrass， interlaced spiderweb-like oil spills， fragile tubular Sargassum， and mottled tufted seagrass beds， existing segmentation algorithms exhibited suboptimal performance in handling these distinctive features. Finally， classical visual semantic segmentation algorithms were applied to evaluate OUC-UAV-SEG by conducting benchmark tests to assess the performance of currently available semantic segmentation algorithms on this dataset and revealing their limitations.

Conclusion

The establishment of the OUC-UAV-SEG dataset marks a significant step forward in providing a novel resource for monitoring coastal ecosystems. This dataset empowers scientists， ecologists， and decision makers with a more comprehensive understanding of the current conditions and evolving trends in coastal ecosystems. These insights are crucial for furnishing more accurate information to support marine ecological conservation and management efforts， which holds positive significance. Future efforts will focus on expanding the diversity of the dataset by collecting data on marine ecological environments under more extensive geographic and meteorological conditions to comprehensively reflect ecosystem diversity. Furthermore， by regularly organizing workshops and challenges， the aim is to advance cutting-edge research in the field of coastal ecosystem monitoring. Finally， active use of social media and other channels will be employed to engage with the public for raising awareness of marine environmental issues and inspiring more people to participate in the monitoring and protection of coastal

ecosystems. For comprehensive details about this dataset， please refer to the project link：

https://github.com/OucCVLab/OUC-UAV-SEG

关键词

Keywords

references

Chen L C ， Zhu Y K ， Papandreou G ， Schroff F and Adam H . 2018 . Encoder-decoder with atrous separable convolution for semantic image segmentation // Proceedings of the 15th European Conference on Computer Vision . Munich， Germany ： Springer： 801 - 818 ［ DOI： 10.1007/978-3-030-01234-2_49 http://dx.doi.org/10.1007/978-3-030-01234-2_49 ］

Cheng B M ， Misra I ， Schwing A G ， Kirillov A and Girdhar R . 2022 . Masked-attention mask Transformer for universal image segmentation // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans， USA ： IEEE： 1290 - 1299 ［ DOI： 10.1109/CVPR52688.2022.00135 http://dx.doi.org/10.1109/CVPR52688.2022.00135 ］

Dong Y N ， Fan B K ， Li S P and Shan J J . 2022 . Research on ecological protection and high-quality development of the yellow river basins from the perspective of ecological civilization . Ecological Economy ， 38 （ 2 ）： 217 - 222

董亚宁，范博凯，李少鹏，单菁菁 . 2022 . 生态文明视角下黄河流域生态保护和高质量发展研究 . 生态经济， 38 （ 2 ）： 217 - 222

Fu J ， Liu J ， Tian H J ， Li Y ， Bao Y J ， Fang Z W and Lu H Q . 2019 . Dual attention network for scene segmentation // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach， USA ： IEEE： 3146 - 3154 ［ DOI： 10.1109/CVPR.2019.00326 http://dx.doi.org/10.1109/CVPR.2019.00326 ］

Jiang X W ， He X Q ， Lin M S ， Gong F ， Ye X M and Pan D L . 2019 . Progresses on ocean satellite remote sensing application in China . Haiyang Xuebao ， 41 （ 10 ）： 113 - 124

蒋兴伟，何贤强，林明森，龚芳，叶小敏，潘德炉 . 2019 . 中国海洋卫星遥感应用进展 . 海洋学报， 41 （ 10 ）： 113 - 124 ［ DOI： 10.3969/j.issn.0253-4193.2019.10.008 http://dx.doi.org/10.3969/j.issn.0253-4193.2019.10.008 ］

Liu Q C ， Xiao L and Yang J X . 2021 . Parallel implementation of content-guided deep convolutional network for hyperspectral image classification . Journal of Image and Graphics ， 26 （ 8 ）： 1926 - 1939

刘启超，肖亮，杨劲翔 . 2021 . 面向高光谱图像分类的内容引导卷积深度网络并行实现 . 中国图象图形学报， 26 （ 8 ）： 1926 - 1939 ［ DOI： 10.11834/jig.200411 http://dx.doi.org/10.11834/jig.200411 ］

Liu T S ， Lam K M ， Zhao R and Qiu G P . 2022 . Deep cross-modal representation learning and distillation for illumination-invariant pedestrian detection . IEEE Transactions on Circuits and Systems for Video Technology ， 32 （ 1 ）： 315 - 329 ［ DOI： 10.1109/TCSVT.2021.3060162 http://dx.doi.org/10.1109/TCSVT.2021.3060162 ］

Liu Z Y and Liu Q S . 2023 . Study on the effect and mode of ecological civilization construction incoastal areas of China . Environmental Protection ， 51 （ 9/10 ）： 55 - 60

刘志媛，刘青松 . 2023 . 我国滨海地区生态文明建设成效及模式研究 . 环境保护， 51 （ 9/10 ）： 55 - 60 ［ DOI： 10.14026/j.cnki.0253-9705.2023.z2.014 http://dx.doi.org/10.14026/j.cnki.0253-9705.2023.z2.014 ］

Min Y K ， Ke Y H ， Han Y ， Yin X L and Zhou D M . 2023 . Dynamic monitoring of invasive Spartina alterniflora clearance via fusion of Sentinel-2 and GF-1 time series images . National Remote Sensing Bulletin ， 27 （ 6 ）： 1467 - 1479

闵钰魁，柯樱海，韩月，尹小岚，周德民 . 2023 . 融合Sentinel-2和GF-1时序影像的入侵植物互花米草清除动态监测 . 遥感学报， 27 （ 6 ）： 1467 - 1479 ［ DOI： 10.11834/jrs.20232279 http://dx.doi.org/10.11834/jrs.20232279 ］

Ronneberger O ， Fischer P and Brox T . 2015 . U-Net： convolutional networks for biomedical image segmentation // Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich， Germany ： Springer： 234 - 241 ［ DOI： 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28 ］

Shelhamer E ， Long J and Darrell T . 2017 . Fully convolutional networks for semantic segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence ， 39 （ 4 ）： 640 - 651 ［ DOI： 10.1109/TPAMI.2016.2572683 http://dx.doi.org/10.1109/TPAMI.2016.2572683 ］

Wada K . 2016 . Labelme： image polygonal annotation with python ［CP/OL］ . ［2023-09-28］ . https://doi.org/10.5281/zenodo.5711226 https://doi.org/10.5281/zenodo.5711226

Wang S Q ， Yang J X ， Shao Y T and Xiao L . 2023 . Non-negative sparse component decomposition based modeling and robust unmixing for hyperspectral images . Journal of Image and Graphics ， 28 （ 2 ）： 613 - 627

汪顺清，杨劲翔，邵远天，肖亮 . 2023 . 高光谱图像非负稀疏分量分解建模与鲁棒性解混方法 . 中国图象图形学报， 28 （ 2 ）： 613 - 627 ［ DOI： 10.11834/jig.211054 http://dx.doi.org/10.11834/jig.211054 ］

Wu T Y ， Tang S ， Zhang R ， Cao J and Zhang Y D . 2020 . CGNet： a light-weight context guided network for semantic segmentation . IEEE Transactions on Image Processing ， 30 ： 1169 - 1179 ［ DOI： 10.1109/TIP.2020.3042065 http://dx.doi.org/10.1109/TIP.2020.3042065 ］

Xie E Z ， Wang W H ， Yu Z D ， Anandkumar A ， Álvarez J M and Luo P . 2021 . SegFormer： simple and efficient design for semantic segmentation with Transformers //Proceedings of the 35th International Conference on Neural Information Processing Systems. ［s.l.］：［s.n.］

Xu J C ， Xiong Z X and Bhattacharyya S P . 2023 . PIDNet： a real-time semantic segmentation network inspired by PID controllers // Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver， Canada ： IEEE： 19529 - 19539 ［ DOI： 10.1109/CVPR52729.2023.01871 http://dx.doi.org/10.1109/CVPR52729.2023.01871 ］

Xu K J ， Deng P F and Huang H . 2021 . HSRS-SC： a hyperspectral image dataset for remote sensing scene classification . Journal of Image and Graphics ， 26 （ 8 ）： 1809 - 1822

徐科杰，邓培芳，黄鸿 . 2021 . HSRS-SC：面向遥感场景分类的高光谱图像数据集 . 中国图象图形学报， 26 （ 8 ）： 1809 - 1822 ［ DOI： 10.11834/jig.200835 http://dx.doi.org/10.11834/jig.200835 ］

Yu C Q ， Gao C X ， Wang J B ， Yu G ， Shen C H and Sang N . 2021 . BiSeNet V2： bilateral network with guided aggregation for real-time semantic segmentation . International Journal of Computer Vision ， 129 （ 11 ）： 3051 - 3068 ［ DOI： 10.1007/s11263-021-01515-2 http://dx.doi.org/10.1007/s11263-021-01515-2 ］

Zhang L ， Chen W and Wang Y H . 2023 . Key sub-region feature fusion network for fine-grained ship detection and recognition in remote sensing images . Journal of Image and Graphics ， 28 （ 9 ）： 2940 - 2955

张磊，陈文，王岳环 . 2023 . 用于遥感舰船细粒度检测与识别的关键子区域融合网络 . 中国图象图形学报， 28 （ 9 ）： 2940 - 2955 ［ DOI： 10.11834/jig.220671 http://dx.doi.org/10.11834/jig.220671 ］

Zhao H S ， Shi J P ， Qi X J ， Wang X G and Jia J Y . 2017 . Pyramid scene parsing network // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu， USA ： IEEE： 2881 - 2890 ［ DOI： 10.1109/CVPR.2017.660 http://dx.doi.org/10.1109/CVPR.2017.660 ］

Zheng J H ， Sun C ， Lin Y ， Li L and Liu Y C . 2023 . Classification of salt marsh vegetation based on pixel-level time series from Landsat images . National Remote Sensing Bulletin ， 27 （ 6 ）： 1400 - 1413

郑嘉豪，孙超，林昀，李璐，刘永超 . 2023 . 基于Landsat像元级时间序列的海岸带盐沼植被分类 . 遥感学报， 27 （ 6 ）： 1400 - 1413 ［ DOI： 10.11834/jrs.20232461 http://dx.doi.org/10.11834/jrs.20232461 ］

Zhu X B ， Shen Y F ， Liu J Q ， Ding J ， Jiao J N ， Ju W M and Lu Y C . 2023 . Optical extraction of oil spills based on sunglint reflection difference in HY-1C CZI images . National Remote Sensing Bulletin ， 27 （ 1 ）： 197 - 208

朱小波，沈亚峰，刘建强，丁静，焦俊男，居为民，陆应诚 . 2023 . 基于耀光反射差异的海面溢油遥感识别提取 . 遥感学报， 27 （ 1 ）： 197 - 208 ［ DOI： 10.11834/jrs.20221688 http://dx.doi.org/10.11834/jrs.20221688 ］

Alert me when the article has been cited

提交

HSRS-SC: a hyperspectral image dataset for remote sensing scene classification

Semantic segmentation of light-field angle cue representations

Bilateral cross enhancement with self-attention compensation for semantic segmentation of point clouds

Multimodal remote sensing image registration： a survey

Weakly supervised semantic segmentation based on deep learning