CS Paper Radar

Higher Order Component Attribution via Learned Surrogates in High Energy Physics

Lauri Laatu, Aritra Bal, Benedikt Maier, Markus Klute, Michael Spannowsky

cs.LG hep-exhep-ph ·submitted 2026-11-05 20:05 UTC · v1
unusual

Abstract

Experimental High Energy Physics (HEP) has adopted the transformer architecture for identification of jets, particularly the state-of-the-art Particle Transformer (ParT) which injects explicit information about particle correlations into the attention mechanism. While able to achieve state-of-the-art results in accuracy, interpretability to its effectiveness has been limited. In this paper we propose component attribution methods using Kolmogorov-Arnold Networks (KANs) that directly tie back to the physical descriptors of jets, to probe whether mechanisms such as Multi-Head Attention (MHA), or the ParT architecture, are able to learn these as discriminating features.

Closest papers

Nearest neighbours by meaning across the whole index.

RINO: Renormalization Group Distillation with No Labels

cs.AI hep-ex ·2026-10-11 ·Zichun Hao, Raghav Kansal, Abhijith Gandrakota, Chang Sun, Haoyang Li +4 more
code/datahot direction similarity 0.54
Abstract

Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel $H \to 4q$ by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO.

0.37

This is SParTA: A modular, efficient and scalable transformer architecture for jet-tagging

cs.LG hep-phhep-ex ·2026-10-19 ·Rajneil Baruah, Subhadeep Mondal, Sunando Kumar Patra, Satyajit Roy
benchmark similarity 0.54
Abstract · 31 pages, 5 figures, 10 tables

We present the Sparse Particle-interaction Transformer Architecture (SParTA), a modular, scalable, and efficient transformer for jet tagging at high-energy colliders. Exploiting the steeply falling transverse-momentum ($p_T$) spectrum of jet constituents, SParTA factorizes constituent processing into two topological regimes. Ordered by descending $p_T$, only the $N$ hardest constituents (the hard core) participate in quadratic $O(N^2)$ differential interaction attention driven by pairwise physical and learned relational features. The remaining soft constituents are integrated via linear $O(lN)$ cross-attention to the hard core, preserving soft-radiation and color-flow information while avoiding the $O(l^2)$ memory bottleneck of full self-attention over all $l$ constituents. Pairwise interaction features are computed dynamically in-network from constituent four-momenta and conditioned on global jet-level substructure observables. On the quark-gluon tagging benchmark, a 300K-parameter SParTA model achieves an AUC of 92.02% and accuracy of 85.06%, matching the 2.14M-parameter Particle Transformer (ParT) at one-seventh the parameter count and outperforming IAFormer by 21% in background rejection at 30% signal efficiency. On the boosted top-tagging benchmark, SParTA outperforms ParT and ParticleNet with 511K parameters, while we analyze the architectural trade-offs relative to fully dense multi-layer interaction networks.

0.37

Functional anatomy of Pythia-Herwig differences with Kolmogorov-Arnold networks

cs.LG hep-ph ·2026-08-16 ·Arghya Chattopadhyay
similarity 0.53
Abstract · 32 pages, 4 appendix

Differences between high-energy event generators can arise at several stages of the collision simulation, from the hard scattering through parton showering and hadronization to the final event. These differences are usually summarized using observable distributions or global classifier scores. While these quantify the disagreement, they do not reveal which observable-level structures carry it or whether those structures persist through different stages of event generation. In this work, we formulate this problem as a staged functional analysis of generator-model differences. Following the same hard dijet events through Pythia and Herwig at shower-only, hadronized, and full-generator levels, we use an additive Kolmogorov-Arnold network (KAN) representation of the classifier-derived log density ratio to decompose the learned discrepancy into explicit one-dimensional observable responses that can be isolated, recomposed, and transported between generator stages. Within the same eight-observable jet representation, the Pythia-Herwig difference is driven mainly by multiplicity at shower level, shifts toward jet mass and shape after hadronization, and develops a mixed shape-multiplicity driven structure in the full-generator configuration. Transporting the individual shower-level functional components downstream shows that shower-level multiplicity information can retain its reweighting power, whereas the corresponding shape responses need not do so even though shape becomes important again at later stages. The jet-mass factors, meanwhile, are limited by poor statistical support. This KAN-based framework therefore provides a functional anatomy of generator-model dependence, exposing both persistent structures and support failures that are hidden inside a single global classifier-derived reweighting function.

0.10

HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives

cs.CV cs.AI ·2026-09-01 ·Sathiyamohan Nishankar, Pubudu Sanjeewani, Asanka Perera, Selvarajah Thuseethan
theory similarity 0.53
Abstract

Vision Transformer (ViT) design has become increasingly diverse, with backbones combining convolutional stems, windowed, linear, or multi-axis attention, patch merging, and spatial reduction in various configurations. This diversity poses challenges for existing attribution methods, whose assumptions often do not hold across ViT variants: Grad-CAM requires a terminal spatial feature map, attention rollout assumes global softmax attention, and layer-wise relevance propagation (LRP) requires module-specific rules. To the best of our knowledge, no existing method provides a unified attribution framework across this architectural space. We show that this architectural diversity can be captured by a simpler underlying structure. The attention and resolution-reduction operators in current ViTs can be decomposed into four operation types: linear maps, bilinear mixing, normalization or gating, and reindexing. Each operation admits a relevance rule that satisfies conservation. Based on these rules, HiLRP supports new backbones by construction rather than by architecture-specific derivation, and its attribution maps decompose the prediction rather than relying on heuristic assumptions. We prove conservation and conditional equivariance and verify both to machine precision. Across 14 attribution methods and 10 architectures, we find that no prior method remains reliable across ViT families, while Faithfulness Correlation becomes uninformative for backbones robust to spatial masking. HiLRP alone preserves conservation across windowed, spatial-reduction, multi-axis, and linear-attention models, where naive extensions can produce zero or inflated relevance. It also localizes attribution failures in class activation mapping, achieving 0.97 Pointing compared with 0.55 for competing methods on EfficientViT.

0.28

Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time

cs.CL ·2026-09-04 ·Lisa Bouger, Yannick Teglia, Philippe Loubet Moundi
similarity 0.51
Abstract

We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence on the logits with structural contribution within the residual stream, enabling a multi-scale analysis at the head, layer, and network levels. Applied to a DeBERTa model specialized for prompt injection detection, our framework reveals distinct decision behaviours between correct and erroneous predictions. Our method provides an effective compromise between fine-grained circuit analysis and global output-based methods, and offers a systematic way to study decision mechanisms in Transformer classifiers.

0.22

"Transforming" LHCb: self-supervised maps of heavy-flavour decays

cs.LG hep-ex ·2026-09-08 ·Marko Stamenkovic, Greg Landsberg
similarity 0.51
Abstract · 31 pages, 14 figures

Decays of beauty and charm hadrons provide sensitive probes of physics beyond the standard model, including decays with invisible particles, in which part of the final state leaves no reconstructed detector signature. The large heavy-flavour data samples recorded by the LHCb experiment at the CERN LHC, together with its precise tracking, displaced vertex reconstruction, and particle identification, make it particularly well suited to learning a map of reconstructed heavy-hadron decay environments directly from data. We propose to bring recent advances in jet flavour tagging at ATLAS and CMS to significantly improve on the performance of the current LHCb taggers and extend them to the reconstruction of heavy-flavour decays with several invisible particles in the final state. To achieve this, we introduce a self-supervised transformer architecture that learns the decay maps without flavour or exclusive-decay labels by inferring masked particle identification information and completing jets from which constituents have been removed. Across five classification tasks in simulated LHCb Open Data, the self-supervised model outperforms an otherwise identical transformer with random weights, and performs comparably to a fully supervised transformer. We achieve a tagging power of about 10\%. In addition, removing constituents from reconstructed exclusive decays also systematically increases the model anomaly score relative to random removals from the same heavy hadrons. We confirm this behaviour directly in 2017 LHCb proton-proton collision Open Data: the score increases for all eight studied heavy-flavour channels, and the signal region response exceeds that in the adjacent sidebands. These studies provide a proof of principle that mapping heavy-flavour decay environments through jets can transform flavour tagging in LHCb and extend the discovery reach for incomplete or otherwise unusual decays.

0.36

NObSP: Functional Decomposition of Neural Networks via Oblique Subspace Projections

cs.LG ·2026-09-15 ·Alexander Caicedo, Víctor De La Hoz, Santiago Alférez
benchmarktheorycode/data similarity 0.51
Abstract · 24 pages, including 6 pages of supplementary material. Code available at https://github.com/santialferez/nobsp_lib

Understanding how deep neural networks make decisions remains a fundamental challenge. We present NObSP (Nonlinear Oblique Subspace Projections), a framework that decomposes predictions into explicit per feature contribution functions and an interaction residual. NObSP exploits the linear final layer of a trained network and uses oblique projections in sample space to reduce double counting when learned feature subspaces overlap, thereby supporting both local explanations and global functional analysis. We establish connections to functional ANOVA and the Kolmogorov-Arnold representation theorem and derive an efficient partial regression algorithm for out of sample evaluation. For convolutional networks, NObSP-CAM produces class activation maps without backward passes after a one time calibration. Experiments on tabular and vision benchmarks show faithfulness comparable to established attribution methods. On a synthetic benchmark with known component functions, NObSP obtains a Function Reproduction Score of 0.989, compared with 0.966 for KernelSHAP and 0.922 for Integrated Gradients. On TinyImageNet, contribution vector embeddings improve mean nearest neighbor class purity from 0.654 for raw activations to 0.713 and reduce mean neighbor distance by more than half. These results indicate that NObSP complements scalar attribution methods by recovering functional contribution profiles with separable positive and negative evidence.

0.36

The proton structure one mode at a time: a Neural Tangent Kernel analysis of a DIS fit

cs.LG hep-ph ·2026-10-26 ·Amedeo Chiefa, Luigi Del Debbio, Richard Kenway
similarity 0.51
Abstract · 70 pages, 37 figures

We study the training dynamics of a feed-forward neural network (NN) through the analysis of the neural tangent kernel (NTK) in a phenomenologically relevant fit of parton distribution functions (PDFs) from deep inelastic scattering (DIS) data. The exact integration of the gradient flow derived in a previous single-flavour analysis is generalised to the multi-flavour case, yielding an analytical perspective on the relationship between data and methodology during training. The Shapley value recently used in the context of PDFs is derived within our statistical framework for observables that depend linearly on the PDFs, and we outline a geometric interpretation of training. We expose the role of gradient-based optimisation algorithms and of the parametrisation in the regularisation of the ill-posed inverse problem that affects the determination of PDFs from experimental data, demonstrating that the regularisation amounts to the incremental inclusion of NTK modes, each learned at the timescale set by the corresponding eigenvalue. We find that only a few leading modes of the NTK are necessary to reconstruct the target PDFs, while the noise of the data is carried by the sub-leading ones learned at later stages of training. Finally, we consider the impact of the initialisation on the training dynamics, and recover a transient behaviour observed in the previous analysis and in the machine learning (ML) literature. Beyond sharpening our understanding of the training dynamics, our framework can be applied to other parametrisations (such as fixed functional forms) and ML applications in physics, as well as to stress tests in the presence of inconsistent data.

0.26