CS Paper Radar

RINO: Renormalization Group Distillation with No Labels

Zichun Hao, Raghav Kansal, Abhijith Gandrakota, Chang Sun, Haoyang Li, Zihan Zhao, Ngadiuba Jennifer, Javier Duarte, Maria Spiropulu

cs.AI hep-ex ·submitted 2026-10-11 21:03 UTC · v1
code/datahot direction

Abstract

Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel $H \to 4q$ by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO.

Closest papers

Nearest neighbours by meaning across the whole index.

Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

cs.LG hep-ex ·2026-09-15 ·Ho Fung Tsoi, Dylan Rankin
similarity 0.73
Abstract · 9th Workshop on Machine Learning and the Physical Sciences (ML4PS 2026)

Many self-supervised methods for training foundation models at the Large Hadron Collider (LHC) rely on data augmentations to encourage the model to embed events into a representation space invariant to certain physical or detector symmetries. A common challenge arises from the large freedom in choosing a proper set of augmentations on which downstream performance depends. The implementation of augmentations involves either modifying existing events, potentially breaking the event fidelity, or simulating more event variants, which is computationally intensive. In this work, we present a data-driven method of pairing events by their similarity via the energy mover's distance (EMD), which measures how similar two events are in terms of the work required to transform one into the other. With this approach, distinct events are sampled and matched by their similarity to serve as views for learning invariance, keeping the physics content of each event intact without handcrafted distortions. We demonstrate this augmentation-free pairing method by pre-training on QCD jets via self-distillation and show that it can yield semantic jet embeddings with downstream discrimination power comparable to or better than an augmentation-based baseline.

0.26

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

cs.LG hep-ex ·2026-10-26 ·Dylan Sheldon Rankin, Ho Fung Tsoi
similarity 0.72
Abstract · 20 pages, 3 figures. Submission to SciPost

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

0.25

"Transforming" LHCb: self-supervised maps of heavy-flavour decays

cs.LG hep-ex ·2026-09-08 ·Marko Stamenkovic, Greg Landsberg
similarity 0.65
Abstract · 31 pages, 14 figures

Decays of beauty and charm hadrons provide sensitive probes of physics beyond the standard model, including decays with invisible particles, in which part of the final state leaves no reconstructed detector signature. The large heavy-flavour data samples recorded by the LHCb experiment at the CERN LHC, together with its precise tracking, displaced vertex reconstruction, and particle identification, make it particularly well suited to learning a map of reconstructed heavy-hadron decay environments directly from data. We propose to bring recent advances in jet flavour tagging at ATLAS and CMS to significantly improve on the performance of the current LHCb taggers and extend them to the reconstruction of heavy-flavour decays with several invisible particles in the final state. To achieve this, we introduce a self-supervised transformer architecture that learns the decay maps without flavour or exclusive-decay labels by inferring masked particle identification information and completing jets from which constituents have been removed. Across five classification tasks in simulated LHCb Open Data, the self-supervised model outperforms an otherwise identical transformer with random weights, and performs comparably to a fully supervised transformer. We achieve a tagging power of about 10\%. In addition, removing constituents from reconstructed exclusive decays also systematically increases the model anomaly score relative to random removals from the same heavy hadrons. We confirm this behaviour directly in 2017 LHCb proton-proton collision Open Data: the score increases for all eight studied heavy-flavour channels, and the signal region response exceeds that in the adjacent sidebands. These studies provide a proof of principle that mapping heavy-flavour decay environments through jets can transform flavour tagging in LHCb and extend the discovery reach for incomplete or otherwise unusual decays.

0.36

Self-Supervised Learning for Robust Resonance Mass Regression in Cascade Decays

cs.LG hep-ex ·2026-09-15 ·Ho Fung Tsoi, Alex Yang, Luis Felipe Gutierrez Zagazeta, Shion Chen, Dylan Rankin
similarity 0.64
Abstract · 9th Workshop on Machine Learning and the Physical Sciences (ML4PS 2026)

Reconstructing the mass of a heavy resonance from its decay products with missing energy is one of the central tasks that directly determine the sensitivity in new physics searches at collider experiments. Supervised learning approaches to this problem often struggle to generalize well due to the presence of various systematic uncertainties and distribution shifts. Exhausting all possible variations in the labeled data can be very compute-intensive, while a failure of the model to generalize can corrupt the reconstructed resonance widths that are critical in peak-hunting analyses. In this work, following the foundation model paradigm, we use a self-supervised approach to pre-train a transformer encoder with VICReg to learn an embedding invariant to various corruptions, then fine-tune it for mass regression on a heavy resonance with masses ranging from 2.5 to 6.5 TeV and a SUSY-like cascade decay into an eleven-body final state. We show that the pre-trained model reconstructs sharper resonance peaks and has a more stable performance under various realistic corruptions, compared to a supervised model of the same architecture trained on the same augmented data from scratch.

0.17

Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems

cs.LG cs.AIphysics.comp-ph ·2026-10-06 ·Kasper Helverskov Petersen, Rasmus Hannibal Tirsgaard, François R J Cornet, Mikkel Jordahn, Mikkel N. Schmidt
code/data similarity 0.63
Abstract

Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self-supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom-JEPA, a self-supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom-level and substructure-level objectives inspired by joint-embedding predictive architectures. We pretrain Atom-JEPA on large-scale molecular and crystalline datasets and evaluate its transfer performance by fine-tuning on a diverse set of downstream property prediction tasks. Atom-JEPA achieves state-of-the-art performance on molecular ADMET and quantum-chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent-space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at https://github.com/khelverskovp/atom-jepa

0.49

ResoSeg: Resonance Tagger using Transformer and Segment Model

cs.AI hep-ex ·2026-09-11 ·Chunkai Li, Junhao Yin, Ke Li, Jingde Chen
code/data similarity 0.63
Abstract · 12 pages, 10 figures

Deep learning has been widely applied across many areas of experimental high-energy physics, yet existing models address only event-level classification or object tagging and therefore still require reconstruction algorithms tailored to each decay channel. We present the first application of segmentation to resonance tagging at BESIII and introduce ResoSeg, a deep learning model that jointly performs particle-level segmentation and event-level classification, enabling a one-pass analysis of resonance to anything decays while precisely reconstructing the relevant resonance properties. We demonstrate the reconstruction of $η_c$ with $e^+e^-\toπ^+π^-h_c$, $h_c\toγη_c$, $η_c\to\text{anything}$. The model is trained on BESIII-$η_c$ dataset, which is constructed with per-track true labels obtained via a Truth-Matching Algorithm. Experimental results show that the average combined efficiency of ResoSeg is more than double that of the conventional 16-channel approach across energy points from 4.19 to 4.60\,GeV. The model generalizes to unseen energy points, adapts to other $η_c$ production modes through transfer learning, and remains robust against variations in the $η_c$ mass, width, and branching fractions, providing a general, resonance-aware model applicable beyond $η_c$ and BESIII. The source code is available at https://github.com/oashen/ResoSeg.

0.45

How Far Can Foundation Models Be Distilled for Efficient Jet Tagging?

cs.LG hep-exhep-phphysics.data-an ·2026-10-21 ·Jay Agarwal, Siddharth Khare, Dhruv Kumar
benchmarkcode/data similarity 0.62
Abstract · 34 pages, 16 figures, 23 tables. Code and model weights: https://github.com/JayAgarwal25/distilling-jet-taggers

Jet tagging is one of the most important applications of machine learning at the Large Hadron Collider. Recent models build in more physics knowledge or rely on large-scale pretraining, but the experiments place strict limits on their inference cost. Knowledge distillation can transfer their performance to small students that are cheap to run. We ask which properties of a teacher help the student, how small a student can get and how close it can come to the state of the art. On a jet tagging benchmark, we distil six teachers that differ in architecture, symmetry and pretraining, including two foundation models, into five small students. Every teacher improves every student, and the gain depends mainly on the strength of the teacher. We then shrink our two strongest distilled students, Transformer and L-GATr-slim, in width and in depth. With 20 thousand parameters, a distilled L-GATr-slim outperforms every published jet tagger trained from scratch at a fraction of the inference cost of every teacher. Even with two thousand parameters, it matches ParT, a model with a thousand times as many parameters. We also study what the students learn from their teachers. Our benchmark of inference cost shows that students of the same size can differ by almost an order of magnitude in speed. We release the code and the weights of all trained models.

0.24

Machine Learning Meets High-Energy Nuclear Physics: From Pattern Recognition to Physics-Integrated Discovery

cs.AI hep-phhep-lathep-th+1 ·2026-10-08 ·Xun Chen, Weiyao Ke, Yu-Gang Ma, Long-Gang Pang, Kai Zhou
publishedsurvey similarity 0.62
Abstract · 37 pages, 22 figures, NST accepted

Machine learning (ML) in high-energy nuclear physics (HENP) is entering a new stage in which physical knowledge is incorporated more directly into data analysis, simulation, and physics inference. This mini-review focuses on developments that have matured in the past several years. Whereas earlier applications emphasized event classification, pattern recognition, and surrogate models for selected observables, recent work has moved toward physics-integrated workflows: calibrated Bayesian extraction of QCD matter properties, dense-matter equation-of-state inference from heavy-ion and neutron-star data, generative event modeling, neural unfolding of weak physical signals, differentiable inverse solvers, gauge-equivariant and diffusion-based lattice-field samplers, and neural reconstruction of model functions in holographic QCD. We survey recent applications of ML in heavy-ion collisions, neutron-star physics, lattice QFT, and holographic or continuum QCD. The emphasis is not on ML architectures alone, but on how they enter concrete physics workflows, how physical constraints such as symmetries, conservation laws, causality, thermodynamic stability, and topology are imposed, and how uncertainty quantification and validation determine whether an AI-assisted result can support a reliable physics conclusion.

0.58