CS Paper Radar

A Bochner Weitzenböck Decomposition for Cellular Sheaf Laplacians, Curvature and Neural Sheaf Diffusion

Francesco Maria Guadagnuolo, Alessio Borgi, Dorina Thanou, Pietro Liò

cs.CG cs.LG ·submitted 2026-10-20 16:13 UTC · v1
theory

Abstract

We extend the Bochner-Weitzenböck decomposition to cellular sheaf Laplacians, splitting it into a positive semidefinite term, the Bochner sheaf Laplacian, and a block diagonal term, the Forman curvature. We show that the Bochner Laplacians form a class of cellular sheaves that extends both connection and matrix-weighted Laplacians, and generalizes signed graphs to $d$ dimensions. We give a holonomy characterization of their kernel, prove the related Cheeger inequalities, and attach to them a random walk that generalizes the Gremban expansion. For the Forman term, we prove a discrete Bochner theorem, which bounds the first cohomology of a cellular sheaf with nonnegative curvature, and we read the whole decomposition as an opinion-dynamics model. We then apply this machinery to Sheaf Neural Networks, whose learnt Laplacian is otherwise unobserved. The Bochner interpretation yields a node-level conviction score that acts as an operational importance measure under frozen-model interventions, while the Forman profile provides a local descriptor of the learnt operator.

Closest papers

Nearest neighbours by meaning across the whole index.

Demystifying Oversmoothing in Sheaf Neural Networks: An Index-Theoretic Criterion

cs.LG math.DG ·2026-08-17 ·Junwen Dong, Yuhan Peng, Hao Li, Huitao Feng, Kelin Xia
similarity 0.62
Abstract

To combat oversmoothing in Graph Convolutional Networks, Sheaf Neural Networks (SNNs) were proposed as a generalization by equipping the graph with a sheaf structure and replacing the graph Laplacian with a sheaf Laplacian $\mathcal{L}$. Existing analyses connect sheaf diffusion to oversmoothing via the harmonic space ($\ker\mathcal{L}$), taking its absolute dimension as an indicator of anti-oversmoothing capacity. However, absolute dimension alone is not a reliable measure: certain sheaf configurations inflate $\dim \ker \mathcal{L}$ while their harmonic sections remain entirely constant, without enriching discriminative capacity. We instead introduce the first relative, geometric approach, yielding a precise characterisation of anti-oversmoothing capacity. Under natural conditions on stalk transportation and global sheaf structure, we establish an index-theoretic comparison criterion showing that one sheaf's harmonic space genuinely contains another's beyond trivial inflation. We illustrate this with a concrete instance and further introduce \textit{GyroSheaf}, a sheaf with curved gyrovector-space stalks, extending the criterion to the non-linear setting via local tangent-space linearization. Experiments across ten models confirm the theoretical criterion: sheaf models violating the criterion collapse despite possessing index jumps, while compliant models maintain depth-stable representations.

0.45

A Finslerian Approach for Embedding Directed Data

cs.LG stat.ML ·2026-09-29 ·Gwendal Debaussart-Joniec, Théau Blanchard, Argyris Kalogeratos
theory similarity 0.58
Abstract

Many datasets carry an intrinsic directionality: citations point backward in time, cells differentiate along lineages, and traffic follows preferred routes. Spectral embedding methods, including most of their extensions to directed graphs, discard this information: they symmetrize the data and map it into a Euclidean space where asymmetry cannot be represented. We instead model directed data as sampled from a Finsler manifold, whose distance depends on the direction of travel, and study the kernel operator built from this asymmetric distance. Through a moment expansion of this operator, we show that its symmetric and antisymmetric parts separate geometry from direction. As the bandwidth of the kernel vanishes, the symmetric part converges to a weighted Laplacian, recovering diffusion maps in the Riemannian case, while the antisymmetric part converges to a first-order transport operator that encodes the directionality. We prove that the corresponding graph operators, built from finitely many samples, converge uniformly and almost surely to these limits. For Randers metrics, this vector field is explicit and yields an embedding algorithm recovering both the manifold structure, from the spectrum of the symmetric part, and the underlying drift. We illustrate the approach on synthetic directed graphs and point-clouds.

0.23

Geometric Structures on Graphs: a Holonomy-Based Discretization of Curvature

cs.LG math.DG ·2026-08-23 ·Hao Li, Yuhan Peng, Junwen Dong
unusual similarity 0.57
Abstract

We propose a holonomy-based framework for discretizing curvature on graphs equipped with local symmetric positive-definite metrics. Each vertex carries a fibre metric \(g_i\), and each directed edge carries a reversible metric-compatible transport \(F_{ij}\). The ordered product around an oriented triangular loop \(\mathcal C\) gives a holonomy \(H_{\mathcal C}\), whose normalized logarithm \(Ω_{\mathcal C}=-s_{\mathcal C}^{-1}\operatorname{Log}(H_{\mathcal C})\) is used as a finite-loop curvature observation. Thus the construction discretizes the geometric principle that infinitesimal holonomy is controlled by curvature, rather than treating holonomy as a heuristic feature. Since \(Ω_{\mathcal C}\) lies in the \(g_i\)-orthogonal Lie algebra, it is not itself a velocity of an SPD metric. We therefore introduce two aggregation mechanisms: a commutator with a symmetric response matrix, producing symmetric Ricci-type metric responses, and an incidence-aware covariant divergence of curvature-induced edge fluxes, reflecting the relation between trace and covariant divergence. The resulting responses are locally orthogonal-gauge equivariant and can drive exponential updates that preserve positive definiteness. We also give a reversible metric-compatible parametrization of edge transports, allowing orthogonal edge factors, loop scales, weights, and response matrices to be learned while respecting the graph geometry. Known-geometry calibrations on the unit sphere test the holonomy--curvature relation, curvature preservation under nontrivial local metric representations, and the empirical recovery of edge transports from local observations.

0.27

Emergent Low-Dimensional Topological Dynamics Dictate Generalization and Escape Mechanics in Billion-Parameter Neural Networks

cs.LG ·2026-09-07 ·Dongqing Cao
code/datahot direction similarity 0.56
Abstract · 16 pages, 6 figures

The remarkable generalization of modern over-parameterized neural networks defies classical statistical learning bounds. Here we establish that optimization trajectories in billion-parameter models do not sample the ambient dimension $\mathbb{R}^N$ homogeneously; instead, they are dynamically constrained to a low-dimensional execution submanifold $\mathcal{M}_{\text{exec}}$ (with empirically observed effective dimension $d_{\text{eff}} \le 12$). By equipping the parameter space with a regularized Hessian metric $g_{ij} = \nabla_i \nabla_j L + εδ_{ij}$ and analyzing stochastic Langevin diffusion on loss sublevel sets, we derive a rigorous Topological Generalization Bound grounded in projected subspace persistent homology. Furthermore, by linking Morse index transitions to Betti number annihilation ($β_1 \to 0$), we formulate an exact mathematical mechanism for exponential escape from strict saddles under anisotropic noise. We validate our framework across 80+ architectures (ResNets, Vision Transformers, and 7B LLMs). To bridge theoretical topology with computational feasibility, we resolve the curse of dimensionality by computing projected persistent homology proxies via randomized Lanczos iterations restricted strictly within active Hessian subspaces. Building upon this mathematical foundation, we introduce Topological-Adaptive Optimization (Top-SGD and Top-AdamW)---a family of wrapper algorithms that dynamically modulate learning rates using active curvature signatures. Extensive benchmarks demonstrate a 14.1\% reduction in total wall-clock training time on ImageNet with all computational and memory overheads fully accounted for. Code is publicly available at https://github.com/imslab-research/Topological-SGD.

0.50

Mitigating Over-squashing without Rewiring: A Sheaf Effective Resistance Perspective

cs.LG stat.ML ·2026-10-03 ·André Ribeiro, Germano Barcelos, Amauri H. Souza, Diego Mesquita, Ana Luiza Tenório
top venuetheory similarity 0.55
Abstract · Accepted to NeurIPS 2026

Graph Neural Networks (GNNs) often struggle to capture long-range dependencies due to over-squashing -- a phenomenon in which the repeated compression of node embeddings into finite-size messages causes representations to collapse. Over-squashing is most often diagnosed as a property of the graph topology, with effective resistance serving as a principled measure of the bottleneck. We provide a complementary view on the matter: building on cellular sheaves, we introduce sheaf effective resistance, a generalization of effective resistance that depends on the sheaf attached to the graph, and we prove that for flat vector bundles, the over-squashing sensitivity in the Jacobian sense is upper bounded by a quantity related to the sheaf effective resistance between the nodes. The bottleneck thus need not lie in the graph itself: it can be relocated, and reduced, by adjusting the sheaf. We instantiate this idea in FlatNSD, a simple message-passing variant of Neural Sheaf Diffusion, and show that it implicitly learns to modulate total sheaf effective resistance, performing well on benchmarks designed to stress over-squashing without altering the original graph topology.

0.54

Learning Pseudo-Riemannian Manifolds for Heterophilic Graphs via Graph Signature

cs.LG cs.AI ·2026-11-04 ·Yun Young Choi, Asung Kil, Sun Woo Park, Minho Lee, Seokhwan Kim
top venuetheory similarity 0.55
Abstract · Accepted at NeurIPS 2026. 41 pages, including appendix

Several widely used families of Graph Neural Networks carry a positive-definite geometric prior. Graph convolution read as heat diffusion needs a positive semi-definite operator, hyperbolic models place features on positive-definite Riemannian manifolds, and dot-product and GAT-style attention scores use positive-definite inner products. On heterophilic graphs, where edges often signify dissimilarity or structural repulsion, that prior is a mismatch: no positive-definite form can assign a negative squared distance to a direction the labels separate. To measure the mismatch, we introduce the Graph Signature Index ($s_{\mathcal{G}}$), the inertia of a class-compatibility matrix built from label topology alone, and we prove that no value of the edge or node homophily index in $(0,1)$ rules out a negative eigenvalue. We then propose the Pseudo-Riemannian Attention Network (PRAT), whose attention logit is a signed squared pseudo-distance under a learnable diagonal metric initialized from $s_{\mathcal{G}}$. The design is diagnose-then-learn: the signature supplies a sign structure that unconstrained training keeps outside near-zero directions, and the projection learns the rest of the metric around it. Across fifteen node-classification benchmarks PRAT outperforms or matches the strongest heterophily-adapted models. A controlled ablation ties that performance to the sign structure itself: with architecture and protocol fixed, forbidding the metric to be indefinite costs accuracy on the graphs $s_{\mathcal{G}}$ marks as indefinite and leaves the others unchanged.

0.43

Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

cs.CL cs.LG ·2026-10-12 ·Bryce Grant, Peng Wang
publishedcode/data similarity 0.54
Abstract · NeurIPS 2026 Poster, ICLR Unifying Concept Representation Learning

Prior research into interpretability does not test whether a representation stays consistent under paraphrasing. Each pair of paraphrases can be thought of as a local witness that two activations should be the same. From these local witnesses we can build a cellular sheaf on the paraphrase graph. The question of what is consistent under paraphrasing can be answered by the Laplacian $L_\mathcal{F}$ given a frozen model. No training required. Our kernel $H^0$ gives us a content subspace. We show that with only 20 dims the kernel outperforms the subspace that LEACE retains with all dims on held-out CounterFact fact retrieval by $+17.1$\,pp on average across 8 models. By constraining all paraphrases of a fact together we see an additional increase of $+6.5$\,pp. Our spectral complement shows us where the causal influence is concentrated. Ablating the coordinates that vary the most when paraphrased (which approximates ablating the spectral complement at the coordinate level) leads to $5.6$ to $26.5\times$ larger changes in model outputs compared to ablating coordinates with matched per-coordinate variability. We find this across 9 models from 124M to 13B params. By stacking layers we can create a grid which has $b_1 = N(L-1)$ algebraic 1-cycles. The harmonic mass derived from the model varies by nearly 3 orders of magnitude between the 6 architectures. It also tracks the order of steering fragility (Spearman's $ρ= 0.90$, exact one-sided permutation $p = 0.014$). The kernel, its spectral complement and the harmonic mass all come from the same paraphrase graph. Project page: https://cwru-aism.github.io/gluing-lc-page/.

0.71

A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds

cs.LG ·2026-08-24 ·Alessio Marta, Paola Causin
theoryunusual similarity 0.54
Abstract · 48 pages, 15 figures

Speciation in generative diffusion models denotes the emergence of distinct stable branches during denoising, through which initially undifferentiated trajectories progressively commit to different data classes. In this work we develop an intrinsic theory of speciation for diffusion models supported on compact Riemannian manifolds: the aim is to go beyond existing theoretical descriptions, which usually identify speciation with a symmetric pitchfork bifurcation and assume to work in a large-dimensional space. We characterize speciation by bifurcations of the critical points of the evolving probability density. A spectral heat-kernel representation makes explicit the role of the manifold geometry, while Poincaré-Hopf and Morse theory impose global constraints on the number and type of score equilibria and reveal topologically-imposed geometrical modes. For mixtures of heat kernels, we prove that generic speciation events have a one-dimensional critical kernel and admit an A2 fold normal form; pitchforks and simultaneous multidirectional transitions arise from nongeneric symmetric configurations. We derive geometry-dependent estimates of speciation times for bimodal mixtures and Riemannian regular simplices. We further establish structural stability of nondegenerate folds under score perturbations and show that the first-order time shift is determined solely by the component of the score error along the critical direction. The theory is illustrated on the sphere using mixtures of von Mises-Fisher distributions, where pitchfork and saddle-node bifurcations, topological modes, and hierarchical multiple speciations are observed. Finally, a chart-based intrinsic score-learning scheme based on neural networks contrasts the theoretically predicted transitions on prototypal and more complex datasets.

0.49