CS Paper Radar

Certified Topological Interaction in Neural Representations: Exact Tests and the Statistic They Require

Sushovan Majhi

cs.LG math.AT ·submitted 2026-09-08 10:51 UTC · v2
code/dataunusual

Abstract

Class disentanglement--the separation of a representation's class-conditional point clouds along depth and over training--is measured by descriptive curves: the sentence such a study wants to write, layer l+1 is more disentangled than layer l, is an eyeball judgement with no null. We supply the inferential layer for a topological measurement of class overlap, the Intersection Euler Characteristic Profile: the Euler characteristic of the overlap of the clouds' ball unions as a function of scale, from one Alpha-complex sweep with no boundary-matrix reduction. Every number carries a test--exact permutation tests in both directions, a guarded separation certificate the invariant requires, and a paired sign-flip test for comparative claims. Building that test taught a lesson outliving this invariant: its statistic must be scale-free. On the raw profile mass, which has units of feature length, 12,375 paired tests return 5,633 significant steps of which every one at the first epoch points the wrong way, certifying feature-norm dynamics as disentanglement; the dimensionless statistic returns 2,707, with 2,026 decreases. Across 111 networks and 52,650 measurements, disentanglement is depth-graded and early, and interaction quotients rank class pairs by confusability (rho=0.83), on par with cheap separability statistics. In a 96-model factorial, augmentation is the one training choice that separates classes relative to chance; weight decay compresses the overlap without separating. Only a k-fold statistic can pose the structural question: the joint entanglement of a class triple sits below its strongest pair in 97% of triple-layer cells and 99.5% of deep cells, at median ratios far below a measured null floor, in vision encoders and frozen language models--a regularity, not a law. The unnormalized mass predicts test accuracy (R^2=0.94), the quotient does not, and neither beats a linear probe.

Comment
36 pages, 9 figures, 4 tables. Code and measurement records: https://github.com/sushovan4/disentanglement

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Homology: Non-Abelian and Orientability Invariants Sensitivity in Cellular Networks

cs.CG ·2026-10-22 ·Raghuwansh Raj, George Panagopoulos, Jun Pang
benchmark similarity 0.61
Abstract

Topological deep learning (TDL) architectures are typically evaluated on synthetic complex pairs that differ in a target invariant. Yet, whether a model recovers that invariant or an artifact of the chosen discretization is seldom audited. To examine this, we design and test pairs of complexes that look identical to standard homological and local message-passing tests, have the same Hasse graphs, yet differ in the target invariant. We study two beyond-homology regimes: nonAbelian fundamental-group structure and homotopy-matched orientability. For the former, we construct genus-2 pairs that match homology, cell statistics, local incidence profiles, and rank-colored Hasse 1-WL behavior while differing in their fundamental groups. For the latter, we evaluate punctured torus-Klein bottle pairs and show that distance, boundary, cycle, and spectral signals can confound apparent orientability detection, motivating stronger matched controls. Under the tested settings, subcomplex-lifted architectures separate some pairs beyond the matched homological, local-incidence, and Hasse 1-WL controls, while a depth sweep reveals an empirical depth-dependent trend without establishing a general barrier or direct computation of the target invariants. These results show that claims of topological expressivity must be stated relative to the model representation, explicitly controlled information channels, and the precise scope of the benchmark.

0.23

Measuring Memory and Generalization as Separable Geometric Channels: The Topo^2 Framework

cs.LG cs.AI ·2026-08-31 ·Zhanbo Zhang, Ming Liu, Qing Wang
similarity 0.52
Abstract · 14 pages, 6 figures

Deep networks trained on noisy labels simultaneously generalize on clean data and memorize flipped labels. These are usually conflated as pressures on one capacity. We present Topo^2, a measurement framework that makes them causally separable, measurable, and law-governed. Persistent-homology H1 structure of the representation space separates into a within-class manifold channel (a function of the training stopping point) and a cross-class channel (a monotone readout of memorized flipped samples). An intervention, the FM0 prescription (zero loss on flipped samples from epoch 0), reaches each setting's generalization ceiling while memorizing essentially nothing. Within the framework we establish a law set with graded evidence: (L2) FM0 separation prescription (9/9); (L1) the within-channel as a training-position function (mid-rise 6/6; convergence-back CIFAR 3/3, SVHN 2/3); (L3) a ring-construction identity (definitional, not a law); and TLS (memory-generalization topological layering): memory is causally additive, anchored (silencing clean collapses the representation), invertible (stripping memory restores near-ceiling generalization), and quantitatively billable (the memorization cost law, effective slope coefficient C ~ 0.38 at the reference capacity: CIFAR-10 0.3801 / SVHN 0.3806 / CIFAR-100 0.384 / VGG 0.3715, capacity-dependent in general and traced to clean-sample feature displacement). We also publish the framework's boundaries: a falsification ledger of nine dead ends, and an instrument-vindication section that excludes six families of global statistics as explanations of the within-channel. The framework turns "memorization" from an ill-defined capacity into a measurable, separable, invertible topological layer.

0.30

Beyond Uniform Local Isometry and Topology: FactoMap for Disentangled Representations

cs.LG cs.AI ·2026-08-25 ·Sohini Gupta, Michael Gao, Bahareh Tolooshams
similarity 0.50
Abstract

Many disentanglement methods represent generative factors using Euclidean product coordinates, although the underlying factor spaces may wrap, collapse, or have position-dependent geometry. We introduce factor-space structure, which combines factor domains, generator-induced identifications, and position-dependent scales to distinguish topologically equivalent spaces with different factor geometries. We show that statistical independence does not imply uniform product geometry: independent hue and scale can affect observations at different, scale-dependent rates, producing anisotropy that no fixed rescaling can remove. We propose the Factor-Space Topographic Map (FactoMap), which learns interpretable prototypes indexed by a structured factor-space lattice. Topographic learning transfers the lattice's periodicity, collapses boundaries, and non-uniform extent to the representation. Experiments show that matching this structure better preserves factor continuity and improves disentanglement of the underlying factors.

0.24

$Ψ$-Resilience: Model-Free Feature Importance from 1D Topological Signals

cs.LG cs.AI ·2026-10-01 ·Fabian Galis, Darian Onchis, Pedro Real Jurado
similarity 0.50
Abstract

We introduce $Ψ$-Resilience, a model-free feature importance method that derives explanations directly from the data itself via 1D topological signals. Our method constructs a class-disagreement landscape by estimating class-conditional densities and taking their pointwise absolute difference along the feature axis. Then, the 0-dimensional persistence of this 1D signal defines a resilience functional that aggregates only those topological features that survive perturbations up to a robustness scale which is set by the user. This gives us a context-robust importance score that is inherently auditable via the underlying 1D landscapes and their persistence. We evaluate our method on both synthetic and real datasets. On synthetic generators with specified ground-truth importance, $Ψ$-Resilience recovers the ranking of features with high fidelity, achieving Spearman rank correlations up to 0.8 and performing competitively with multiple feature importance methods, including SHAP and mutual information. On real datasets with no known ground truth, our technique agrees with these methods, with correlations up to 0.9. These results show that $Ψ$-Resilience is a stable explanation method that enables rigorous, distribution-level auditing of feature importance without relying on a predictive model.

0.31

TopoSteer: Topological Layer Selection for Activation Steering in LLMs

cs.CL cs.LG ·2026-11-01 ·Tuc Nguyen, Marcos Gutiérrez-del-Pozo, Eduardo Paluzo-Hidalgo, Yifan Hu, Adaku Uchendu +1 more
position similarity 0.50
Abstract · 42 pages, 8 figures

Activation steering controls large language models at inference time by adding a vector to their internal representations. Most work asks how to build that vector, and far less attention has gone to where to add it, even though steering at a poorly chosen layer can be worse than not steering at all. Prior selectors choose a layer without ever steering the model to test it: they score how separable the target behavior is at that layer, and how consistently the individual examples point along the steering direction. We argue that what matters is not the separability of the two activation clouds, nor the alignment of individual displacements with their mean, but the extent of the cloud of per-example displacements that mean-difference steering averages. We propose a topology-based layer selection method, called TopoSteer, which scores a layer by the total H_0 persistence of its displacement cloud, computed as the length of the cloud's Euclidean minimum spanning tree. We show that the score focuses on the displacement cloud's relative geometry, while class separation affects it only through standardization. We also provide a theoretical explanation for our method by proving that, as example-level variation increases, TopoSteer and discriminability move in opposite directions, so the two criteria can rank layers in opposite ways. Across multiple language models and steering tasks, TopoSteer provides a principled alternative to discriminability- and separability-based layer selection and yields more effective steering without requiring held-out validation.

0.58

Topological Deep Learning Challenge 2026: Bridging the Gap

cs.LG cs.AI ·2026-11-05 ·Guillermo Bernárdez, Lev Telyatnikov, Mathilde Papillon, Marco Montagna, Louisa Cornelis +66 more
publishedbenchmark similarity 0.50
Abstract

This paper describes the 2026 edition of the Topological Deep Learning Challenge: Bridging the Gap, hosted at the second Topology, Algebra, and Geometry in Data Science (TAG-DS) Conference. Where previous editions grew the modelling and data ecosystem of Topological Deep Learning (TDL), this edition set out to place Graph Neural Networks (GNNs) and Topological Neural Networks (TNNs) side by side under a single evaluation pipeline. Participants implemented recent state-of-the-art architectures in TopoBench, with Track 1 covering GNNs and Track 2 covering TNNs. They evaluated these implementations on shared synthetic benchmarks generated with GraphUniverse, which independently controls homophily, average degree and degree-distribution heaviness. The challenge yielded 68 qualifying submissions from 34 teams, spanning the graph, simplicial, cellular, hypergraph and combinatorial domains. For the first time, submissions were also tracked on a public leaderboard, which we summarize and analyze here. Our analysis finds that the differences between the submitted tracks are strongly task-dependent: the mean accuracy gap is modest on community detection, while on triangle counting the Track 1 mean error is 1.85 times the Track 2 mean. Several structurally informed GNNs also outperform the Track 2 mean on triangle counting.

0.34

Decodable but Not Accessible: Auditing Distance-Based Reliability Estimation on Disentangled Skin-Lesion Representations

eess.IV cs.CV ·2026-08-10 ·Duc-Vinh Tran
similarity 0.49
Abstract

Distance-based reliability estimation assumes that a representation's geometry reflects its trustworthiness, yet this assumption is rarely tested under training interventions that reshape geometry directly. We audit this assumption under domain-adversarial representation learning using a disentanglement dose-response ladder. Three checkpoint families share the same architecture and a 16-dimensional representation, differing only in orthogonality strength (lambda = 0, 1, 5). Representation geometry changed substantially with disentanglement strength: the condition number shifted by two orders of magnitude (Kendall tau = 0.84, exact p = 2.8e-5). This change was not accompanied by improved reliability estimation: Mahalanobis-distance AUROC (ISIC-test vs. PAD-UFES) remained flat and below chance (about 0.40) at every level, with no significant association with any of five geometry metrics tested. The same failure was observed for cosine-to-centroid and pooled k-nearest-neighbor scorers, plus three non-distance-based scorers: an energy-based confidence score, Virtual-Logit Matching, and a kernel density estimator. Seven of eight scorers converged on the same result; the energy-based score showed an isolated upward trend that we report but do not treat as evidence against the overall pattern. A supervised probe with no access to the training objective recovered domain membership from the identical embeddings at 0.72-0.81 AUROC across every level, showing that the relevant information was not absent from the representation. These findings indicate that classification performance alone can overlook whether information in a learned representation is organized in a form that downstream reliability estimators can use. Information can remain decodable while becoming largely inaccessible to non-probing reliability estimators.

0.04

What Visual Generators Need from Teachers: Rethinking Representation Alignment

cs.CV ·2026-09-28 ·Yongcong Wang, Hingchin Chen, Mingyu Fan, Shuo Jiang, Teer Zhang +4 more
hot direction similarity 0.49
Abstract

Representation alignment speeds up diffusion transformer training by pulling an intermediate block of the model (student) toward features of a frozen pretrained encoder (teacher). Which teacher layer to align, and for how long, is still set by convention, and each alternative costs a training run. We find that alignment helps where the student cannot linearly recover the teacher's features, not where it already resembles them. Since a deep teacher layer is largely predictable from the one below, we isolate what each layer adds, its increment, and measure how much of it an unaligned student recovers. The student fills the teacher's hierarchy from the bottom up and stalls near the top, which we call hierarchy filling: even after 400K steps it recovers almost none of the deepest. The recoverability gap is the unrecovered share of an increment, read from one unaligned checkpoint. In short runs that each align one teacher layer at one block, the gap nearly reproduces their ranking by FID improvement, and CKA, a measure of feature similarity, largely reverses it. Representation Alignment and Recoverability Estimation (RARE) picks the teacher layer with the largest gap before training. During training, it tracks each token's remaining distance to that layer, the online counterpart of the gap, weights tokens by it, and phases out the loss once the average distance stops falling. With SiT-B/2 on ImageNet $256\times256$, RARE reaches an FID of 18.02 without guidance and 4.46 with it, ahead of seven alignment baselines including REPA, iREPA and HASTE. It also trains in 14% fewer GPU-hours than iREPA. Its FID stays below iREPA's across model scales, teachers, datasets and backbones.

0.60