图增强与多重神经网络优化的多维图对比学习
Multidimensional graph contrastive learning based on graph enhancement and multi-neural networks
- 2025年 页码:1-14
收稿日期:2024-10-09,
修回日期:2025-04-08,
录用日期:2025-04-09,
网络出版日期:2025-04-09
DOI: 10.11834/jig.240612
移动端阅览
浏览全部资源
扫码关注微信
收稿日期:2024-10-09,
修回日期:2025-04-08,
录用日期:2025-04-09,
网络出版日期:2025-04-09,
移动端阅览
目的
2
图表示学习在社交网络、生物信息及推荐系统等领域应用广泛。无监督图对比学习因其无需大量标注数据即可获取高质量节点表示而备受关注,但现有方法普遍存在增强策略单一、对比粒度粗放等问题,影响嵌入表示质量。
方法
2
针对上述问题,本文提出一种结合局部-全局图增强技术与多重神经网络协同建模的多维度图对比学习模型(local augmentation and SVD based on triple network for multi-dimensional graph comparative learning,LAST-MGCL)。首先,构建局部增强图神经网络和奇异值分解增强模块,分别从节点邻域信息和整体拓扑模式出发,对原始图数据进行多粒度增强;其次,设计由多头注意力图神经网络构成的三重编码网络,分别处理原始图和增强图,通过跨网络信息交互强化多视图融合表示;最后,提出跨网络对比、跨视图对比与邻居对比相结合的多维度对比损失,协同优化图表示质量。
结论
2
在节点分类任务上,LAST-MGCL模型在Cora、Citeseer和PubMed数据集上的平均分类准确率分别达到82.5%、72.5%和81.6%,整体优于当前主流对比学习方法,体现出较好的分类性能与鲁棒性;同时,在可视化任务中,LAST-MGCL生成的节点嵌入表现出更紧密的类内聚合与更清晰的类间边界,进一步验证了模型在表征学习中的有效性。综上,本文提出的LAST-MGCL面向无标签图数据场景,对现有图对比学习框架进行了系统性增强,为无监督图表征学习提供了一种有效解决方案。
Method
2
First, LAST-MGCL incorporates a local-global graph augmentation strategy, which combines two complementary techniques: a Conditional Variational Autoencoder (CVAE) for local enhancement and an SVD-based module for global enhancement. The CVAE is designed to enrich feature representations within local neighborhoods, enabling the model to better capture the intricate relationships between nodes that have limited neighborhood connectivity. This is particularly beneficial for sparse graphs, where local structure is often underrepresented. By generating richer local feature representations, the CVAE ensures that the model can more effectively process nodes with few neighbors, ultimately improving the overall quality of graph embeddings. On the other hand, the SVD-based module focuses on global structural patterns, leveraging singular value decomposition to capture the topological essence of the graph at a broader scale. This global enhancement ensures that key topological features are preserved, facilitating the model's ability to generalize across different graph types. By combining these local and global enhancement techniques, LAST-MGCL creates a multi-granularity augmentation strategy that provides diverse views of the graph, enriching the learning process and improving the expressiveness of various graph neural networks (GNNs). Second, LAST-MGCL adopts a triple encoding network architecture, which leverages the power of multi-head attention to process both the original and augmented graph data. In this architecture, the graph data is passed through three sub-networks, each guided by a multi-head attention mechanism that enables the model to focus on different aspects of the graph's structure. The multi-head attention mechanism is designed to capture diverse, multi-scale dependencies across the graph, making it particularly effective at integrating information from various views of the graph. Through cross-network information exchange, the sub-networks collaborate, strengthening the model's ability to integrate representations from different graph perspectives. This cross-network collaboration enhances the multi-view fusion, which is critical for improving the robustness and stability of the learned graph embeddings. By ensuring that information is effectively shared between sub-networks, the model can integrate complementary information from both original and augmented graph views, thus improving the overall representation quality. Third, LAST-MGCL introduces an innovative multi-dimensional contrastive learning optimization framework to further refine the learning process. This novel framework integrates multiple contrastive learning objectives to optimize graph representation learning across various dimensions. The contrastive loss is designed to combine cross-network contrastive learning, which aligns representations between the original and augmented graphs, with cross-view contrastive learning that enhances generalization across different augmented perspectives. Additionally, neighbor contrastive learning is incorporated to maintain local semantic coherence by focusing on the relationships between neighboring nodes within the graph. These objectives work together to reinforce the structural consistency and semantic alignment of graph representations at multiple granularities, ensuring that both local and global dependencies are effectively captured. By applying this multi-dimensional contrastive framework, LAST-MGCL addresses critical challenges such as the underutilization of contrastive information in traditional methods, the reliance on negative samples, and the difficulty of aligning representations across different graph views.
Conclusion
2
In node classification tasks, the LAST-MGCL model demonstrates strong performance across several benchmark datasets, achieving classification accuracies of 82.5% on Cora, 72.5% on Citeseer, and 81.6% on PubMed. These results indicate that LAST-MGCL consistently outperforms state-of-the-art contrastive learning methods, offering superior classification accuracy and robustness. Additionally, the node embeddings generated by LAST-MGCL exhibit more compact intra-cluster cohesion and clearer inter-cluster boundaries, highlighting the model's ability to effectively capture and distinguish graph structures. Ablation experiments were conducted to assess the contribution of each model component. The results revealed that removing key components significantly degraded performance. For example, removing multi-dimensional contrastive learning resulted in the most significant performance drop, with a reduction of 9.7%. These findings underscore the importance of the combined local-global graph augmentation approach, which captures both local and global graph information, and the role of multi-dimensional contrastive learning in enhancing node interactions and clustering. Furthermore, a hyperparameter sensitivity analysis was performed to optimize model performance. Finally, t-SNE visualizations comparing LAST-MGCL to the best baseline models show that LAST-MGCL excels in node clustering and maintains clear class boundaries across all datasets, further validating its superior performance in representation learning. In summary, LAST-MGCL enhances the existing graph contrastive learning framework by integrating local-global graph augmentation, multi-view representation learning, and multi-dimensional contrastive optimization. Specifically designed for unsupervised graph learning, it provides an effective solution for learning high-quality node representations from unlabeled graph data.
Purpose
2
Graph representation learning has been widely applied across various domains, including social networks, bioinformatics, and recommendation systems, due to its ability to effectively capture and encode structural and relational information within graph-structured data. Among existing approaches, unsupervised graph contrastive learning (GCL) has gained significant attention as it enables high-quality node representations without relying on extensive labeled data, making it particularly suitable for real-world applications where labeled annotations are costly and scarce. However, despite its advantages, current GCL methods suffer from several inherent limitations. Most existing graph contrastive learning techniques rely on single-perspective augmentation strategies, such as randomly removing edges or nodes, which can only capture a limited range of structural variations within a graph. However, graphs often exhibit intricate, multi-level dependencies that a single augmentation approach cannot adequately represent. For instance, a graph may contain subgraphs with varying node connectivity patterns or may encode higher-order relationships that are overlooked by such simple augmentations. As a result, relying solely on one perspective reduces the richness and diversity of learned node representations, limiting the model's ability to generalize across different graph structures. Moreover, conventional contrastive learning frameworks often employ coarse-grained contrastive mechanisms, comparing entire subgraphs or large sets of nodes at a high level of abstraction. While this can work in certain contexts, it fails to capture finer-grained distinctions in local node structures and semantic attributes, leading to suboptimal node embeddings. These limitations hinder the model’s ability to learn discriminative representations, thereby affecting its effectiveness in tasks such as node classification, clustering, and link prediction.
Bing R , Yuan G , Meng F R , Wang S Z , Qiao S J and Wang Z X . 2023 . Multi-view Contrastive Enhanced Heterogeneous Graph Structure Learning . Journal of Software , 34 ( 10 ): 4477 - 4500
邴睿 , 袁冠 , 孟凡荣 , 王森章 , 乔少杰 , 王志晓 . 2023 . 多视图对比增强的异质图结构学习方法 . 软件学报 , 34 ( 10 ): 4477 - 4500 .[ DOI: 10.13328/j.cnki.jos.006883 http://dx.doi.org/10.13328/j.cnki.jos.006883 ]
Cai D and Lam W . 2020 . Graph transformer for graph-to-sequence learning . Proceedings of the AAAI conference on artificial intelligence , 34 ( 05 ): 7464 - 7471 .[ DOI:10.1609/aaai.v34i05.6243]
Chien E , Peng J , Li P and Milenkovic O . 2020 . Adaptive universal generalized pagerank graph neural network. arXiv preprint arXiv:2006. 07988 . [ DOI: 10.48550/arXiv.2006.07988 http://dx.doi.org/10.48550/arXiv.2006.07988 ]
Han X , Jiang Z , Liu N and Hu X . 2022 . G-mixup: Graph data augmentation for graph classification. In International Conference on Machine Learning (pp . 8230 - 8248 ). PMLR .[ DOI: 10.48550/arXiv.2202.07179 http://dx.doi.org/10.48550/arXiv.2202.07179 ]
Hassani K and Khasahmadi A H . 2020 . Contrastive multi-view representation learning on graphs. In International conference on machine learning (pp . 4116 - 4126 ). PMLR . [ DOI: 10.48550/arXiv.2006.05582 http://dx.doi.org/10.48550/arXiv.2006.05582 ]
He M , Wei Z and Xu H . 2021 . Bernnet: Learning arbitrary graph spectral filters via bernstein approximation . Advances in Neural Information Processing Systems , 34 , 14239 - 14251 . [ DOI: 10.48550/arXiv.2106.10994 http://dx.doi.org/10.48550/arXiv.2106.10994 ]
Kipf T N and Welling M . 2016 . Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609. 02907 .[ DOI: 10.48550/arXiv.1609.02907 http://dx.doi.org/10.48550/arXiv.1609.02907 ]
Lee N , Lee J and Park C . 2022 . Augmentation-free self-supervised learning on graphs . In Proceedings of the AAAI conference on artificial intelligence (Vol . 36 , No. 7 , pp. 7372 - 7380 ).[ DOI: 10.1609/aaai.v36i7.20700 http://dx.doi.org/10.1609/aaai.v36i7.20700 ]
Li X , Cai B and Hu N B . 2024 . Research on graph contrastive learning method based on ternary mutual information . CAAI Transactions on Intelligent Systems , 19 ( 5 ): 1257 - 1267 .
李旭 , 蔡彪 , 胡能兵 . 2024 . 基于三元互信息的图对比学习方法研究 . 智能系统学报 , 19 ( 5 ): 1257 - 1267 . DOI:10.11992/tis.202308004.[ DOI: 10.11992/tis.202308004 http://dx.doi.org/10.11992/tis.202308004 ]
Liu Y B , Li H N , Zhang C Q , Xiao Z T , Zhang F , Wei Y , Gao Y Z , Shi F , Shan F and Shen D G . 2022 . Diagnosis of COVID-19 by using structural attention graph neural network . Journal of Image and Graphics , 27 ( 3 ): 750 - 761 .
刘彦北 , 李赫南 , 张长青 , 肖志涛 , 张芳 , 隗英 , 高耀宗 , 石峰 , 单飞 , 沈定刚 . 2022 . 结构图注意力网络的新冠肺炎轻重症诊断 . 中国图象图形学报 , 27 ( 03 ): 750 - 761 .[ DOI: 10.11834/jig.210682 http://dx.doi.org/10.11834/jig.210682 ]
Mo Y , Peng L , Xu J , Shi X and Zhu X . 2022 . Simple unsupervised graph representation learning . In Proceedings of the AAAI conference on artificial intelligence (Vol . 36 , No. 7 , pp. 7797 - 7805 ).[ DOI: 10.1609/aaai.v36i7.20748 http://dx.doi.org/10.1609/aaai.v36i7.20748 ]
Pei H , Wei B , Chang K C C , Lei Y and Yang, B . 2020 . Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002. 05287 .[ DOI: 10.48550/arXiv.2002.05287 http://dx.doi.org/10.48550/arXiv.2002.05287 ]
Peng Z , Huang W , Luo M , Zheng Q , Rong Y , Xu T and Huang J . 2020 . Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference 2020 (pp . 259 - 270 ).[ DOI: 10.1145/3366423.3380112 http://dx.doi.org/10.1145/3366423.3380112 ]
Ranjan E , Sanyal S and Talukdar P . 2020 . Asap: Adaptive structure aware pooling for learning hierarchical graph representations . In Proceedings of the AAAI conference on artificial intelligence (Vol . 34 , No. 04 , pp. 5470 - 5477 ). [ DOI: 10.1609/aaai.v34i04.5997 http://dx.doi.org/10.1609/aaai.v34i04.5997 ]
Shen X , Sun D , Pan S , Zhou X and Yang L T . 2023 . Neighbor contrastive learning on learnable graph augmentation . In Proceedings of the AAAI conference on artificial intelligence (Vol . 37 , No. 8 , pp. 9782 - 9791 ). [ DOI: 10.1609/aaai.v37i8.26168 http://dx.doi.org/10.1609/aaai.v37i8.26168 ]
Veličković P , Fedus W , Hamilton W L , Liò P , Bengio Y and Hjelm D . 2018 . Deep graph infomax. arXiv preprint arXiv:1809. 10341 .[ DOI: 10.48550/arXiv.1809.10341 http://dx.doi.org/10.48550/arXiv.1809.10341 ]
Veličković P , Cucurull G , Casanova A , Romero A , Lio P and Bengio Y . 2017 . Graph attention networks. arXiv preprint arXiv:1710. 10903 .[ DOI: 10.48550/arXiv.1710.10903 http://dx.doi.org/10.48550/arXiv.1710.10903 ]
Wan S , Zhan Y , Liu L , Yu B , Pan S and Gong C . 2021 . Contrastive graph poisson networks: Semi-supervised learning with extremely limited labels . Advances in Neural Information Processing Systems , 34 , 6316 - 6327 .[ DOI: 10.5555/3540261.3540744 http://dx.doi.org/10.5555/3540261.3540744 ]
Xu Y F and Fan H X . 2024 . Self-Supervised Graph Representation Learning Method Based on Data and Feature Augmentation . Journal of Computer Engineering & Applications , 60 17
) (许云峰 , 范贺荀 . 2024 . 基于数据与特征增强的自监督图表示学习方法 . 计算机工程与应用 , 60 ( 17 ): 148 - 157 . DOI: 10.3778/j.issn.1002-8331.2306-0254. [DOI:10.3778/j.issn.1002-8331.2306-0254 http://dx.doi.org/10.3778/j.issn.1002-8331.2306-0254.[DOI:10.3778/j.issn.1002-8331.2306-0254 ]
Yang K X , Liu L , Fu X D , Liu L J and Peng W . 2025 . Scale feature representation learning network for retinal vessels image segmentation . Journal of Image and Graphics , 30 ( 03 ): 0855 - 0869
杨可欣 , 刘骊 , 付晓东 , 刘利军 , 彭玮 . 2025 . 视网膜血管图像分割的尺度特征表示学习网络 . 中国图象图形学报 , 30 ( 03 ): 855 - 869 .[ DOI: 10.11834/jig.240120 http://dx.doi.org/10.11834/jig.240120 ]
Yin Y , Wang Q , Huang S , Xiong H and Zhang X . 2022 . Autogcl: Automated graph contrastive learning via learnable view generators . In Proceedings of the AAAI conference on artificial intelligence (Vol . 36 , No. 8 , pp. 8892 - 8900 ). [ DOI: 10.1609/aaai.v36i8.20871 http://dx.doi.org/10.1609/aaai.v36i8.20871 ]
You Y , Chen T , Sui Y , Chen T , Wang Z and Shen Y . 2020 . Graph contrastive learning with augmentations . Advances in neural information processing systems , 33 , 5812 - 5823 .[ DOI: 10.48550/arXiv.2010.13902 http://dx.doi.org/10.48550/arXiv.2010.13902 ]
Zhu Y , Xu Y , Yu F , Liu Q , Wu S and Wang L . 2020 . Deep graph contrastive representation learning. arXiv preprint arXiv:2006. 04131 . [ DOI: 10.48550/arXiv.2006.04131 http://dx.doi.org/10.48550/arXiv.2006.04131 ]
Zhu Y , Xu Y , Yu F , Liu Q , Wu S and Wang L . 2021 . Graph contrastive learning with adaptive augmentation . In Proceedings of the web conference 2021 (pp. 2069-2080).[ DOI: 10.1145/3442381.3449802 http://dx.doi.org/10.1145/3442381.3449802 ]
Zhu J , Yan Y , Zhao L , Heimann M , Akoglu L and Koutra D . 2020 . Beyond homophily in graph neural networks: Current limitations and effective designs . Advances in neural information processing systems , 33 , 7793 - 7804 .[ DOI: 10.48550/arXiv.2006.11468 http://dx.doi.org/10.48550/arXiv.2006.11468 ]
Zhou B , Jiang Y , Wang Y , Liang J , Gao J , Pan S and Zhang X . 2023 . Robust graph representation learning for local corruption recovery. In Proceedings of the ACM Web Conference 2023 (pp . 438 - 448 ). [ DOI: 10.1145/3543507.3583399 http://dx.doi.org/10.1145/3543507.3583399 ]
相关作者
相关机构