CS Paper Radar

Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

Ho Fung Tsoi, Dylan Rankin

cs.LG hep-ex ·submitted 2026-09-15 18:44 UTC · v2

Abstract

Many self-supervised methods for training foundation models at the Large Hadron Collider (LHC) rely on data augmentations to encourage the model to embed events into a representation space invariant to certain physical or detector symmetries. A common challenge arises from the large freedom in choosing a proper set of augmentations on which downstream performance depends. The implementation of augmentations involves either modifying existing events, potentially breaking the event fidelity, or simulating more event variants, which is computationally intensive. In this work, we present a data-driven method of pairing events by their similarity via the energy mover's distance (EMD), which measures how similar two events are in terms of the work required to transform one into the other. With this approach, distinct events are sampled and matched by their similarity to serve as views for learning invariance, keeping the physics content of each event intact without handcrafted distortions. We demonstrate this augmentation-free pairing method by pre-training on QCD jets via self-distillation and show that it can yield semantic jet embeddings with downstream discrimination power comparable to or better than an augmentation-based baseline.

Comment
9th Workshop on Machine Learning and the Physical Sciences (ML4PS 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

RINO: Renormalization Group Distillation with No Labels

cs.AI hep-ex ·2026-10-11 ·Zichun Hao, Raghav Kansal, Abhijith Gandrakota, Chang Sun, Haoyang Li +4 more
code/datahot direction similarity 0.73
Abstract

Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel $H \to 4q$ by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO.

0.37

Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC

cs.AI hep-exhep-phphysics.data-an ·2026-10-09 ·Donatella Genovese, Philip Ploner, Simone Scardapane, Stefano Giagu, Thea Klaeboe Aarrestad
benchmark similarity 0.71
Abstract · Submission to SciPost

Anomaly detection, the identification of collision events that deviate from Standard Model expectations, is one of the most promising model-agnostic strategies for new-physics searches at the Large Hadron Collider. Searching for rare events in learned low-dimensional representations removes the need to commit to hand-crafted observables a priori, but sacrifices the immediate physical interpretation those observables provide. For anomaly detection to enter a discovery pipeline, flagged events must therefore be diagnosed locally, both to distinguish potential new physics from mismodeling or detector effects and to identify observables for targeted follow-up searches. We evaluate supervised and self-supervised neural embedding strategies on COLLIDE-2V, a 750-million-event dataset simulated under realistic High-Luminosity LHC conditions, using the reconstruction error of an autoencoder trained in the learned latent space as the anomaly score. Across the configurations and benchmark signals considered, supervised objectives, which exploit simulation process labels during training, generally outperform their self-supervised counterparts and, in their best configurations, improve on an autoencoder trained directly on kinematic features, raising the signal efficiency at 10% background acceptance on all held-out processes. We further introduce a local diagnostic pipeline that fits a Gaussian Mixture Model to the SM latent space, profiles each component through high-level observables without process labels, localizes the flagged events, and ranks observables by their local discriminating power. The ranking recovers the defining kinematics of the held-out processes, while correlating the anomaly score with the same observables quantifies how closely the score tracks them, revealing cases where the most discriminating observable is not the one the score follows most closely.

0.11

Learning the Geometry of Collider Events with Metric-Aware Deep Sets

cs.LG hep-ph ·2026-09-10 ·Lauren Hay, Rishabh Jain, Matt LeBlanc, Jennifer Roloff
similarity 0.69
Abstract · 14 pages including appendices, 5 figures. Preliminary

Optimal transport gives structured data a geometry, but exact evaluation is costly in large pairwise analyses that exploit relationships among distances. Learned surrogates are faster, but need not preserve this metric structure. We develop a Deep Sets surrogate for OT between variable-size weighted point clouds that enforces non-negativity, exchange symmetry, and zero self-distance, leaving the triangle inequality unconstrained. Applied to the Energy Mover's Distance between collider events in a particle physics application, the Metric-Aware Particle Flow Network achieves percent-level mean absolute percentage error while significantly improving inference throughput over other exact and approximate methods surveyed. The architectural constraints are found to improve properties that are not explicitly enforced: across $10^6$ held-out event triplets, triangle-inequality violations fall from 199 for a matched unconstrained network to 2, and the maximum from 149.5 to 5.8 GeV. These results demonstrate that targeted inductive biases can yield fast neural surrogates with substantially improved geometric fidelity.

0.36

"Transforming" LHCb: self-supervised maps of heavy-flavour decays

cs.LG hep-ex ·2026-09-08 ·Marko Stamenkovic, Greg Landsberg
similarity 0.69
Abstract · 31 pages, 14 figures

Decays of beauty and charm hadrons provide sensitive probes of physics beyond the standard model, including decays with invisible particles, in which part of the final state leaves no reconstructed detector signature. The large heavy-flavour data samples recorded by the LHCb experiment at the CERN LHC, together with its precise tracking, displaced vertex reconstruction, and particle identification, make it particularly well suited to learning a map of reconstructed heavy-hadron decay environments directly from data. We propose to bring recent advances in jet flavour tagging at ATLAS and CMS to significantly improve on the performance of the current LHCb taggers and extend them to the reconstruction of heavy-flavour decays with several invisible particles in the final state. To achieve this, we introduce a self-supervised transformer architecture that learns the decay maps without flavour or exclusive-decay labels by inferring masked particle identification information and completing jets from which constituents have been removed. Across five classification tasks in simulated LHCb Open Data, the self-supervised model outperforms an otherwise identical transformer with random weights, and performs comparably to a fully supervised transformer. We achieve a tagging power of about 10\%. In addition, removing constituents from reconstructed exclusive decays also systematically increases the model anomaly score relative to random removals from the same heavy hadrons. We confirm this behaviour directly in 2017 LHCb proton-proton collision Open Data: the score increases for all eight studied heavy-flavour channels, and the signal region response exceeds that in the adjacent sidebands. These studies provide a proof of principle that mapping heavy-flavour decay environments through jets can transform flavour tagging in LHCb and extend the discovery reach for incomplete or otherwise unusual decays.

0.36

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

cs.LG hep-ex ·2026-10-26 ·Dylan Sheldon Rankin, Ho Fung Tsoi
similarity 0.69
Abstract · 20 pages, 3 figures. Submission to SciPost

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

0.25

Comprehensive reconstruction of collider events with hypergraph representation learning and graph-conditioned diffusion

cs.LG hep-phhep-ex ·2026-09-16 ·Lining Mao, Yvonne Peters, Ethan Simpson, Zihan Zhang
similarity 0.66
Abstract · 23 pages, 9 figures, to be submitted to PRX Intelligence

In particle collider experiments, event reconstruction is the task of inferring the kinematics of short-lived particles produced in the hard scatter from the stable final states recorded by detectors. We decompose event reconstruction into two primary tasks: assigning measured jets and charged leptons to parent particles, and predicting unmeasured neutrino kinematics. We present VyPER, a novel geometric learning framework that represents collider events as hypergraphs with a physics-inspired topology. VyPER combines the supervised classification of hyperedges for particle assignment with a diffusion model for predicting neutrino kinematics, leveraging a joint loss function to optimize both reconstruction tasks within a unified framework. We showcase VyPER across several proton-proton collision processes, comparing its performance to existing analytical and machine-learning-based reconstruction techniques. In doing so, we demonstrate that accurate event reconstruction is achievable across a diverse range of Standard Model physics processes, opening new avenues for precision measurements in the Higgs boson, electroweak, and top-quark sectors.

0.17

Scaling Collider Event Generation with Residual-Quantized Tokens

cs.LG hep-exhep-phphysics.data-an ·2026-09-30 ·Dan Godi, Dmitrii Kobylianskii, Eilam Gross
similarity 0.66
Abstract · 10 pages + 11 pages of appendices, 12 figures, 10 tables

Full detector simulation and reconstruction of collider events are projected to become major bottlenecks at the High-Luminosity Large Hadron Collider, motivating the development of fast, ML-based surrogates. At the same time, LLMs have driven fast progress in generative discrete modeling: autoregressive transformers trained on tokenized data now represent the state of the art across a range of generative tasks. We extend the discrete modeling paradigm by introducing a particle-level generative model trained on residual-quantized full-event data. We demonstrate the ability of this model family to perform conditional generation from detector-stable particles; we study its scaling behavior across a range of dataset and model sizes, characterize the effects of repeated data exposure and demonstrate that token-level loss systematically predicts downstream physical fidelity. These results provide an empirical framework for scalable collider full-event generation based on residual-quantized representations.

0.39

How Architecture and Training Affect TPC Representations Across Experiments

cs.LG cs.CVnucl-exphysics.ins-det ·2026-08-22 ·Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp +10 more
similarity 0.64
Abstract

Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional embeddings for four datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.

0.11