CS Paper Radar

Columnar-Embedder: A Biologically Inspired Cortical Architecture for Binary Sparse Distributed Graph Representations

Mohamed Abidalrekab, Dan Hammerstrom

cs.NE ·submitted 2026-08-12 03:31 UTC · v1

Abstract

Finding a representative description of graph entities that captures their structural roles and homophily is a challenging goal for graph embedding techniques due to the non-Euclidean nature of graphs. Traditionally, Graph embeddings achieve top performance via random-walk methods and graph neural networks. However, these methods are transductive and utilize an expensive global optimization via softmax or a dense representation trained in an end-to-end pipeline with gradient descent. Nonetheless, other variants of GNNs can map to unseen nodes; they still rely on iterative message passing and backpropagation, incurring high computational and memory costs. Conversely, the mammalian cortex solves structurally similar problems by learning to map its input stream of patterns into a compact representation for downstream regions. We present the biologically inspired Columnar-Embedder architecture for learning binary Sparse Distributed Representations (SDRs) of graph nodes. The learning is driven by a local Bienenstock-Cooper-Munro (BCM) Hebbian rule modulated by positive pointwise mutual information (PPMI) computed from online streams of random walks. Continuous learning from streaming random-walk pairs without labels, backpropagation, or supervision enables the architecture to exhibit natural resistance to catastrophic forgetting. Across five graph benchmarks, the performance of SDRs is competitive with that of real-valued dense embeddings on node classification and link prediction, while the architecture exhibits portability, resilience to noise, and robustness to data corruption.

Closest papers

Nearest neighbours by meaning across the whole index.

Extremely Fast and Compact Binary Graph Representations via Randomized Operator Sketching

cs.LG ·2026-09-26 ·Srajan Agarwal, Megha P, Bikas C Das, Zakaria Laskar, Saptarshi Bej
similarity 0.70
Abstract

Graph neural networks typically rely on dense, floating-point node representations, which can impose substantial memory and computational costs. Binary graph hashing offers an alternative by encoding node information as compact bit strings. However, existing approaches either sacrifice global topological information for computational efficiency or incur substantial generation costs. We introduce an ultra-fast, entirely algebraic hashing method that constructs binary node representations directly from graph structure, without requiring node features or gradient-based training. Our method approximates a high-order structural transition matrix using randomized column sampling inspired by the Nyström method and combines it with an efficient label-safe semantic propagation mechanism. The resulting continuous representations are discretized through column-wise thresholding to obtain compact binary codes. Experiments on ten node classification datasets show that the proposed method consistently improves classification accuracy over existing feature-free binary baselines while requiring sub-second code generation on many datasets. The resulting binary representations are also naturally suited to event-driven computation, making them compatible with neuromorphic spiking neural networks and gradient-free learning rules. These results demonstrate that simple algebraic approximations can provide an efficient alternative to learned pipelines for discrete graph representation learning.

0.18

Topology-induced Operators Reveal Complementary Graph Representations without Training

cs.LG cs.SI ·2026-09-08 ·Meng Qin, Jinqiang Cui, Hongwei Zheng, Weihua Li, Sen Pei
similarity 0.69
Abstract

Graph representation learning has largely focused on designing increasingly sophisticated models to transform graph topology into vector representations, or embeddings. However, the extent to which embedding quality depends on model learning, rather than on the underlying topological transformations, remains unclear. Here, we show that informative embeddings can be derived without complicated model design and gradient-based training. Propagating random features through implicit hierarchical structures induced by random walks and anonymous walks yields embeddings that capture node proximity and structural role, respectively. These two training-free embeddings preserve complementary aspects of graph organization and perform competitively with classic and recent methods across various node-, edge-, and graph-level tasks. They often require substantially less computation, resulting in a favorable quality-efficiency trade-off. Combining the two types of embeddings further improves inference quality of some tasks compared with using either embedding type alone. Our results suggest that informative graph embeddings can arise from carefully chosen topological transformations before any learning operation is applied.

0.04

HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases

cs.AI stat.MLcs.LG ·2026-09-27 ·Quan D. Bui, Nguyen Do, An Nguyen Dang, Huyen Nguyen, Nhu Duc Minh Nguyen +1 more
similarity 0.68
Abstract

Existing interpretable graph additive models still face limitations in either computational scalability or modeling flexibility. In terms of structural modeling, previous approaches either face quadratic scaling costs or sacrifice explicit source-to-target contribution decomposition. In terms of feature components, they rely either on per-feature neural networks or on single shared bases with limited feature specialization. We address both problems by introducing HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases, an interpretable-by-design framework. For feature modeling, HARMONIA introduces a Mixture of Neural Bases (MoNB), which routes features to specialized basis experts, enabling parameter sharing without sacrificing feature-specific specialization. For structural modeling, HARMONIA uses Relative Random Walk Probabilities (RRWP) to capture multi-hop and multi-path relationships, and proposes Sparse RRWP Aggregation (SRA) to compute these interactions through sparse graph propagation without quadratic pairwise complexity. HARMONIA retains a simple additive form in which predictions decompose into feature responses modulated by structural influence. Empirically, HARMONIA achieves stronger explanation recovery than existing interpretable graph baselines while maintaining competitive predictive performance and scaling to graphs with millions of nodes. These results show that interpretable graph learning can remain both faithful and scalable without sacrificing predictive effectiveness.

0.21

Message-passing graph persistence

cs.LG cs.AIstat.ML ·2026-11-04 ·Jorge Luiz Franco, Arthur Queiroz Moura, Moacir Ponti, Amauri Souza
benchmarktheory similarity 0.68
Abstract

Message-passing graph neural networks (GNNs) are limited in expressive power by the one-dimensional Weisfeiler-Leman test. Persistent homology (PH) provides topological descriptors that can complement GNN representations, but existing approaches typically compute persistence separately from the representation learning process. In this paper, we introduce Message-Passing Graph Persistence (MGP), a family of topological descriptors that incorporates message passing into graph filtrations. At each filtration step, node embeddings are updated using the current subgraph and associated with persistence events. These embeddings also determine the representatives of connected components during persistence computation. We prove that MGP is invariant under graph isomorphism and that it is strictly more expressive than the PH-based descriptors it extends. We evaluate MGP on an expressivity benchmark and on several graph classification and regression datasets. Experiments show that MGP outperforms other PH-based descriptors on most of these datasets.

0.17

Scalable GNN-based Knowledge Graph Representation Learning with Efficient Message Passing

cs.LG cs.AI ·2026-09-28 ·Huu Tan Mai, Cuong Xuan Chu, Heiko Paulheim, Daria Stepanova
published similarity 0.68
Abstract · Accepted at the Posters and Demos Track of the International Semantic Web Conference (ISWC) 2026

Graph neural networks (GNNs) excel at representation learning on Knowledge Graphs (KGs), achieving stateof-the-art performance on tasks like link prediction or entity classification. However, their high computational complexity, inherent to their user-defined message passing (MP) algorithm, still prohibits their widespread adoption, especially for large KGs. Current efforts to mitigate the scalability bottlenecks of GNNs on KGs, such as subgraph sampling, are often task- and model-specific, and do not reliably guarantee lossless (if applicable) runtime/space reductions. To address this, we extend Relational Sparse Matrix Multiplication (RSPMM), originally designed to losslessly lower the space complexity of composition-based MP with pointwise composition functions, to support more expressive functions (e.g., 2x2 block-diagonal matrix multiplication, Givens rotation, circular correlation). Our method delivers significant task-independent reductions in runtime and space for current GNNs on KGs and facilitates efficient re-implementations of GNNs that maintain near state-of-the-art performance on challenging KG tasks, for a fraction of computational costs.

0.26

Spatially-Aware Self-Organizing Graph Autoencoders via Localized Hebbian Learning

cs.CV ·2026-10-25 ·Robin Koç, Hazal Moğultay Özcan, Fatoş T. Yarman Vural
published similarity 0.67
Abstract · 6 pages, 5 figures. Accepted to the UBMK conference (not presented)

Traditional autoencoders rely on static network architectures to learn latent representations. Recently we suggested a dynamic architecture inspired by Hebbian plasticity, enabling topological self-organization among convolutional feature maps (R. Koç et al., Proceedings of the SIU, 2026). Early implementations of this paradigm infer relationships among feature maps using global average pooling and model their interactions through scalar-valued connections. This approach falls short to fully exploit the underlying spatial information, embedded in input images. Furthermore, abrupt transitions between different loss functions and training dynamics can introduce representation shocks, leading to unstable feature learning. In this paper, we introduce a Spatially-Aware Self-Organizing Graph Autoencoder (S-LooM) to address these structural bottlenecks. By using spatial cross-correlation, and spatial kernels, S-LooM establishes dynamic connections among feature maps. We replace static scalar message passing of autoencoders with learnable spatial kernels and eliminate high-frequency deconvolutional artifacts observed in decoders using nearest-neighbor upsampling. Via a soft-blending training curriculum, the latent topology aligns smoothly during training. Our experiments show that S-LooM yields a more stable topology, compared to the state of the art dynamic architectures. Meaning, the graph structure that emerge has more resistance to feature entanglement and noise. It forms functional communities responsible for visual attributes, such as illumination, shape and texture even under high connection densities.

0.24

Dynamic Heterogeneous Graph Representation Learning: A Survey

cs.LG cs.AIcs.SI ·2026-09-04 ·Huan Liu, Pengfei Jiao, Jie Yin, Hongjiang Chen, Zhidong Zhao
top venuesurvey similarity 0.67
Abstract · IJCAI 2026 Survey Track

Graph representation learning (GRL) serves as a canonical paradigm for modeling complex networks. However, real-world AI systems inherently manifest as evolving heterogeneous entities with complex interactions, posing significant challenges to static or homogeneous modeling. To address these complexities, representation learning for Dynamic Heterogeneous Graphs (DHGs) has emerged as a vital approach for learning low-dimensional representations that simultaneously preserve structural semantics and temporal dynamics. This survey presents the first systematic review of DHG representation learning methods. We first introduce a unified formal definition that encompasses both discrete-time and continuous-time DHGs from the perspective of temporal granularity. Building upon this formulation, we propose a novel algorithm-centric taxonomy that categorizes existing literature, including early embedding-based approaches, graph neural network (GNN)-based models, and relatively recent Transformer-based DHG methods, while explicitly highlighting their intrinsic modeling biases with respect to dynamic granularity. Furthermore, we summarize representative applications of DHG representation learning, along with commonly used datasets and benchmarks. Finally, we discuss promising research directions that guide future advances in this rapidly evolving field.

0.38

DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning

cs.LG cs.AIcs.NE ·2026-10-12 ·Yuxiang Zeng, Kun Xie, Yang Wang, Jigang Wen, Xiaocan Li +3 more
top venue similarity 0.67
Abstract · Accepted at NeurIPS 2026. 38 pages, 6 figures

Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. Neural architectures are operation-labeled directed acyclic graphs (DAGs), where edge directions determine computational roles and execution dependencies. Existing graph Transformer predictors incorporate topological and directional information in different ways, but primarily focus on specific mechanisms rather than coordinated encoder-wide modeling of these semantics. We introduce encoder-wide directional consistency, a design principle that coordinates the modeling of directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature updates. To implement this principle, we propose DTA-GT, a Direction- and Topology-Aware Graph Transformer that integrates direction- and topology-aware node initialization, magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative RNN-, CNN-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and magnetic spectral controls support the contribution of the direction-aware stages and orientation-sensitive spectral information, while mechanistic diagnostics characterize module behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.

0.61