CS Paper Radar

MAGE: Meta-path Adaptive Graph Evidence for Transferable Multimodal Graph Learning

Haiyang Jiang, Tong Chen, Xinyi Gao, Hao Wang, Hongzhi Yin

cs.IR ·submitted 2026-11-16 03:18 UTC · v1

Abstract

Multimodal graphs integrate textual and visual attributes with relational structures, providing a rich representation of complex real-world data. To enable the broad applicability and reuse of graph models, it is crucial for models trained on source graphs to generalize to unseen graph domains. However, this is challenging because cross-modal interactions among neighbors and the neighboring information needed for prediction can vary across target nodes and graph domains. This gives rise to two key challenges: adaptive cross-modal neighborhood fusion, which captures and integrates cross-modal interactions among neighbors to accommodate variations across nodes and domains; and target-specific evidence selection, which identifies the neighboring textual and visual information needed for each target node to support LLM-based prediction. In light of these challenges, we propose MAGE (Meta-path Adaptive Graph Evidence Prompting), a framework for transferable multimodal graph learning. MAGE represents textual and visual node views separately in a heterogeneous graph and introduces two complementary components. Meta-path Adaptive Multimodal Propagation (MAMP) adaptively aggregates neighborhood information along heterogeneous meta-paths and compresses it into graph-aware soft tokens. Evidence-Guided Instruction Tuning (EGIT) selects relevant neighboring evidence for each query and incorporates its graph-aware representations into instructions for a frozen multimodal LLM. Together, these components support transfer from multiple source graphs to unseen target graphs without target-domain parameter updates. Experiments with three source graphs and four unseen target graphs demonstrate strong cross-domain transfer performance on node classification and link prediction, without requiring parameter updates on the target domains.

Closest papers

Nearest neighbours by meaning across the whole index.

TEAM: Topology-Aware, Semantic-Sensitive, General Transformer for Multimodal-Attributed Graphs

cs.LG ·2026-11-08 ·Yushuo Li, Xunkai Li, Zhengyu Wu, Ronghua Li, Guoren Wang
similarity 0.78
Abstract · 10 pages, 6 figures, 5 tables

Multimodal-Attributed Graphs (MAGs) organize heterogeneous data in graphs whose nodes are characterized by modalities such as text and images rather than traditional unimodal attributes. However, many representative MAG methods organize propagation around node-level states and topology-defined neighborhoods, which can obscure fine-grained modality contributions: a bounded neighborhood may contain semantically weak evidence, while compatible context need not be confined to it. To address both limitations, we propose TEAM, a modality-centric framework that constructs and encodes graph context with node-modality tokens rather than treating each node as a single interaction unit. Concretely, TEAM unfolds the local topology into a Modality Encoding Tree and performs tree-anchored semantic evidence acquisition, organizing local structural context while expanding the accessible evidence space outside the constructed local Tree. We further introduce the Modality-Source Context Transformer (MSCT), which encodes the selected node-modality tokens and coordinates interactions among modality-source groups. By retaining textual and visual attributes as separate interaction units until readout, it supports fine-grained cross-node cross-modal exchange without first collapsing them into whole-node features. Across the evaluated baselines, TEAM attains the highest reported mean on all graph-learning metrics, all three G2Text metrics, and both G2Image metrics. On Toys, it surpasses the strongest baseline by 3.37 and 2.93 percentage points in Accuracy and Macro-F1, respectively.

0.21

Multimodal Graph Learning: A Survey of Models, Benchmarks, and Frontiers

cs.LG ·2026-11-08 ·Rui Xue, Tianfu Wu
survey similarity 0.77
Abstract

Multimodal attributed graphs (MAGs), in which nodes carry text, images, and other modalities while edges encode relations among entities, underlie e-commerce, social media, content platforms, and creative applications. Learning on MAGs has developed along two complementary lines: graph-centric methods redesign how modalities are fused, propagated, and connected within message passing, while foundation-model-centric methods pretrain transferable multimodal graph encoders or bring graph context into multimodal large language models for prediction and generation. In parallel, several benchmarks have appeared with different datasets and evaluation protocols. This survey takes an algorithm-centric and evaluation-grounded view of the field and makes three contributions. Modeling: we introduce a unified message-passing formulation and organize methods into five integration levels (feature, propagation, topology, learning, and foundation model), covering discriminative and graph-conditioned generative models; within this formulation we derive representative methods as special cases, compare their assumptions, expressiveness, and cost, and identify cross-modal sensitivity as a property that separates otherwise similar architectures. Datasets and evaluation: we review recent benchmarks, summarize their datasets and metrics, define model-independent descriptors (label homophily, average degree, clustering, and edge provenance), and map each evaluation protocol to the baselines it uses. Frontiers: we connect multimodal graph learning to AI agents, covering the multimodal graphs that agents perceive and remember and graph engineering, the practice of organizing agent workflows as explicit graphs; we then discuss open problems, concrete efforts for the community, and research directions, including when graph structure helps in the presence of strong multimodal foundation models.

0.19

ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning

cs.LG ·2026-08-24 ·Rui Xue, Tianfu Wu
hot direction similarity 0.75
Abstract

Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes, yet existing methods largely decouple these processes: prior multimodal graph neural networks (GNNs) focus on aligning modalities in a shared embedding space while operating on fixed or weakly adapted graph structures, and graph structure learning approaches infer topology from unimodal node representations without accounting for multimodal interactions. This separation fundamentally limits the ability of GNNs to capture semantically meaningful relationships in multimodal settings, where observed edges are often noisy, incomplete, or misaligned with underlying semantics. We propose ReCoG (Reciprocal Co-Evolution for Multimodal Graph Learning), a new learning paradigm that tightly couples graph structure learning and multimodal representation learning through end-to-end reciprocal interaction. Concretely, ReCoG integrates (i) a multimodal graph refiner that infers and corrects edges using cross-modal semantic evidence, and (ii) a coupled cross-modal message passing mechanism that performs joint intra- and inter-modality propagation over the refined graph. This unified design yields greater expressiveness than decoupled or two-stage formulations and allows dynamic interaction between topology and representation learning. Across diverse benchmarks for node classification and link prediction, ReCoG consistently outperforms strong multimodal graph structure learning baselines, including graph foundation models. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning.

0.45

Rethinking Evidence Modeling: Context-Coupled Graph Learning for Multimodal Recommendation

cs.IR ·2026-10-11 ·Mingqiao Zhang, Qiyao Peng, Yumeng Wang, Wenzhe Dong, Hongtao Liu +1 more
similarity 0.74
Abstract

Multimodal graph recommendation integrates observed interactions with item images and text to learn user preferences. These inputs constitute Semantic-Behavioral Evidence (SBE). Many existing graph methods incorporate collaborative and multimodal semantic relations, yet their aggregation and state updates remain context-separated, limiting the continuous composition of preference evidence across layers and the joint use of multiple sources. To address this limitation, we propose CC-MGRec, a context-coupled graph learning method that organizes SBE within a common graph learning context for cross-relation modeling and compositional propagation. Its context-coupled multimodal graph integrates user-item collaborative relations and item-item semantic relations, coordinating their message scales through joint normalization. Recommendation-grounded semantic calibration uses training behavior to adjust semantic neighbor weights while preserving normalized collaborative propagation coefficients. Collaborative-semantic compositional propagation updates shared node states layer by layer to continuously compose collaborative and multimodal semantic evidence, implicitly forming Preference Evidence Chains (PECs), including collaborative-semantic and visual-textual PECs. Joint execution of relation messages on the same shared node states within each layer preserves propagation outputs while reducing intermediate output processing and fusion overhead. We provide theoretical analysis of the proposed mechanisms. Experiments on three Amazon datasets demonstrate improved recommendation quality and faster inference during graph propagation.

0.14

MM-DyGraph: A Dataset, Benchmark, and Model for Multimodal Dynamic Graphs

cs.AI cs.LG ·2026-11-06 ·Zongyuan Wu, Pengjie Wang, Lingshan Chen, Tianhang Wan, Yuan Meng +3 more
benchmark similarity 0.73
Abstract

Multimodal dynamic graphs are pervasive in real-world applications, where nodes and edges are accompanied by multimodal signals, including text and images. Despite their practical importance, benchmarks and methods that explicitly study multimodal learning in dynamic graphs remain scarce, hindering the full understanding and effective utilization of multimodal information for downstream tasks. To address this gap, we introduce MM-DyGraph, to the best of our knowledge, the first benchmark for multimodal dynamic graph learning, consisting of diverse real-world datasets with fine-grained temporal graph evolution and rich multimodal attributes. MM-DyGraph provides three evaluation tasks to comprehensively assess model performance in dynamic settings. We benchmark representative state-of-the-art dynamic graph methods and reveal that naive feature concatenation yields target-level inconsistencies: multimodal inputs benefit certain prediction targets while harming others. This observation indicates that the modality informativeness depends on the specific prediction target over time. Building on these observations, we propose a Target-aware Spatial-Temporal Multimodal Fusion (TST-MF) framework, which jointly conditions multimodal fusion on the prediction target and spatial-temporal relations, enabling context-adaptive cross-modal interplay for target-specific representations. Extensive experiments over diverse datasets and tasks show that TST-MF delivers robust and competitive performance, achieving clear gains in multimodal settings.

0.25

ReCoG: Relative Node Contexts for Multi-Domain Graph Pre-training

cs.LG ·2026-11-01 ·Yupeng Tong, Lianze Shan, Jitao Zhao, Dongxiao He
similarity 0.73
Abstract · 13 pages, 7 figures, 8 tables

The success of foundation models in language and vision has motivated Graph Foundation Models (GFMs), which learn transferable knowledge through multi-domain graph pre-training. Existing methods advance GFMs through feature unification and structural learning, and in doing so they rely on the raw graph structure to organize node contexts. However, we observe that removing a large proportion of edges from source graphs during pre-training improves cross-domain transfer accuracy across four representative methods. Our theoretical analysis identifies the raw graph structure as a source of conflict in cross-graph model sharing. We therefore advocate defining node contexts using criteria consistent across graphs. Achieving this requires (1) constructing node contexts consistently across graphs without a shared positional structure, such as the token order of text or the pixel grid of images, and (2) retaining the structural and neighborhood information of the original graphs. To this end, we introduce Relative Contexts for Graphs (ReCoG) for multi-domain graph pre-training. Specifically, we first construct attribute contexts to provide a comparison rule independent of the raw graph structure. Furthermore, we construct structural contexts based on common neighbors and semantic contexts based on neighborhood attribute summaries to retain the topological and neighborhood information of the original graphs. Since all graphs share these construction rules, we pre-train a single graph encoder by reconstructing node attributes over these contexts, without domain-specific modules. We conduct extensive experiments on 18 target datasets, which demonstrate the effectiveness of ReCoG. In particular, ReCoG exceeds the strongest baseline by 7.61 percentage points in mean 1-shot cross-domain node classification accuracy and by 4.47 points in cross-type transfer to heterophilous graphs.

0.13

Toward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approach

cs.LG ·2026-10-02 ·Xunkai Li, Chenxi Wan, Yinlin Zhu, Wang Luo, Hongchao Qin +2 more
similarity 0.71
Abstract

Multimodal graph foundation models (MGFMs) seek to learn generalizable representations from large-scale graphs with heterogeneous node modalities. However, real-world Multimodal-Attributed Graphs (MAGs) often contain incomplete node attributes, limiting the scale and diversity of available pretraining corpora. Besides, existing MGFMs primarily incorporate graph topology as structural context, overlooking its role in guiding multimodal binding and shaping a unified representation space. To address these challenges, we propose GraphBind, a topology-driven approach that uses graph topology to bind rich modality information into a unified shared space. GraphBind is motivated by the stability of graph topology, which provides structural references and complementary semantic information for multimodal binding. Concretely, GraphBind uses topology to organize self semantics and reliable neighborhood semantics into a global shared space that integrates structure and semantics, and adapts this space to discriminative and generative tasks through lightweight interfaces. Extensive experiments against 11 representative baselines demonstrate that GraphBind achieves leading performance on both discriminative and generative tasks, with relative improvements of up to 28.1% over the strongest baseline.

0.12

MOVE: Multimodal Open-world Verification and Expansion for Graph Learning

cs.LG ·2026-09-24 ·Zekai Chen, Jiayang Xing, Xun Wu, Miao Zhang, Xunkai Li +5 more
similarity 0.71
Abstract

Multimodal graph learning faces a fundamental challenge: new classes may emerge after deployment, while models are trained with a fixed label space. Existing approaches typically detect unknown nodes and use LLMs to generate candidate class descriptions, but they do not determine whether existing classes are insufficient to cover these nodes or whether a generated class is reliable enough to expand the class space. Our empirical study reveals three challenges: multimodal information beyond individual modalities is required for unknown-node identification, LLM-generated class descriptions may not fully capture multimodal class characteristics, and directly adding candidate classes can introduce redundant categories. Based on these observations, we propose MOVE, a multimodal open-world class verification and expansion framework. MOVE identifies nodes that cannot be assigned to existing classes by jointly considering visual tokens, textual attributes, and graph context, leverages a multimodal LLM to generate candidate classes, and selectively expands the class space only when candidates are consistently supported by multimodal evidence without introducing unnecessary categories. Experiments demonstrate that MOVE achieves an average improvement of 11.87\% across unknown recognition, open-domain annotation, and downstream graph learning tasks.

0.24