CS Paper Radar

Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics

Samuel Young, César Jesús-Valls, Kazuhiro Terao

cs.CV hep-ex ·submitted 2026-09-01 02:56 UTC · v1

Abstract

Foundation models are increasingly being pursued in particle and nuclear physics, but existing approaches remain strongly tied to individual experiments through detector-specific architectures or pre-training objectives, limiting their reuse across sensing modalities. We show that a point cloud self-distillation framework yields a substantially more general sensor-level pre-training recipe. We show that the same refined architecture and objective can be independently pre-trained with minimal changes on three qualitatively different detector modalities: liquid argon time projection chamber (LArTPC), collider TPC, and water Cherenkov. Using 1,000 labeled images for downstream task adaptation, Panda V2 matches or exceeds specialized foundation-model baselines trained with orders of magnitude more supervision, matching state-of-the-art particle-clustering performance with 70x fewer labeled events on sPHENIX while substantially improving particle identification, and on LArTPC data matching Panda (arXiv:2512.01324) particle reconstruction with up to 1,000x fewer labels. Beyond reconstruction, simple linear probes reveal physically meaningful latent structure associated with particle causality and track curvature.

Comment
24 pages, 11 figures, preprint

Closest papers

Nearest neighbours by meaning across the whole index.

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

cs.LG hep-ex ·2026-10-26 ·Dylan Sheldon Rankin, Ho Fung Tsoi
similarity 0.66
Abstract · 20 pages, 3 figures. Submission to SciPost

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

0.25

Track finding and particle identification performance with particle physics foundation models

cs.LG physics.data-anhep-exnucl-ex ·2026-10-22 ·Shuhang Li, Haiwang Yu, Gabor Galgoczi, Yeonju Go, Yi Huang +7 more
similarity 0.65
Abstract

Modern high energy nuclear and particle physics experiments produce large amounts of sparse data that must be reduced significantly in size before performing physics analysis. Recently, there has been an effort to evaluate Foundation Models as a potential tool for performing data analysis. These models have been shown to learn features in high dimensional datasets based on the multi-modal data they are trained on. To show their effectiveness at performing complex pattern recognition tasks, such as reconstructing charged particles in a magnetic field, we evaluated the performance of a Foundation Model based reconstruction on several downstream tasks with a dataset composed of spacepoints in a Time Projection Chamber. We show that the performance of the Foundation Model based data analysis significantly improves upon that of traditional methods, and furthermore can be adapted to learn additional tasks that are not necessarily related to the initial training objective.

0.20

How Architecture and Training Affect TPC Representations Across Experiments

cs.LG cs.CVnucl-exphysics.ins-det ·2026-08-22 ·Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp +10 more
similarity 0.65
Abstract

Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional embeddings for four datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.

0.11

SensOFormer: Foundation Models for Particle Accelerators

cs.LG physics.acc-ph ·2026-11-01 ·Mahindra Rautela, Alexander Scheinker
code/data similarity 0.62
Abstract

Large-scale scientific facilities, such as particle accelerators, are complex systems composed of thousands of interacting components that must operate collectively to support diagnostics and control, and ensure safe, reliable operation. These facilities rely on extensive sensor networks that generate large volumes of heterogeneous, incomplete, and noisy telemetry, often sampled at different rates across subsystems. In this paper, we adopt the foundation-model paradigm to learn general-purpose sensor representations from historical accelerator data through self-supervised pretraining. We introduce SensOFormer, a denoising masked transformer with a Perceiver-style encoder-decoder architecture designed to handle variable sensor sets, accommodate heterogeneous sampling rates, and learn robust representations from noisy measurements. The model is pretrained on multiple particle-accelerator datasets and transferred to downstream tasks, including missing-data imputation, anomaly detection, cavity identification, and fault identification. Through extensive evaluations and comparisons, we demonstrate that a single self-supervised model can learn reusable representations for diverse operational tasks in large-scale scientific facilities. Project page: https://mahindrautela.github.io/sensoformer.

0.30

Leveraging Industrial Foundation Models at the Edge of Particle Physics Detectors via Distillation Learning and Hardware Co-design

cs.LG physics.ins-dethep-ex ·2026-09-20 ·Gia Ancone, Qibin Liu, Liangyu Wu, Julia Gonski
similarity 0.62
Abstract · 7 pages, 1 figure, 1 table

Data acquisition (DAQ) systems at future particle physics experiments stand to benefit from the extremes of AI/ML development: large-scale foundation models can enhance the performance of feature extraction algorithms, and small-scale on-detector deployments can enable real-time intelligent data handling. This work provides the first fine-tuning of an industrial foundation model for particle physics DAQ. Starting from the backbone of Google Research's TimesFM (Time Series Foundation Model), we demonstrate fine-tuning on real-time regression tasks for drift chamber trackers and dual-readout calorimeters. Furthermore, the fine-tuned TimesFM model is distilled into a student and co-designed with FPGA implementation to enable these models to run in real-time at future colliders. The fine-tuned distillations meet or exceed the performance of previously published AI/ML solutions for each task. Further, the pipeline of distillation and model compression from TimesFM is generic and can be easily adapted to a variety of 1D waveform tasks across domains.

0.28

Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC

cs.LG hep-ex ·2026-09-15 ·Ho Fung Tsoi, Dylan Rankin
similarity 0.61
Abstract · 9th Workshop on Machine Learning and the Physical Sciences (ML4PS 2026)

Many self-supervised methods for training foundation models at the Large Hadron Collider (LHC) rely on data augmentations to encourage the model to embed events into a representation space invariant to certain physical or detector symmetries. A common challenge arises from the large freedom in choosing a proper set of augmentations on which downstream performance depends. The implementation of augmentations involves either modifying existing events, potentially breaking the event fidelity, or simulating more event variants, which is computationally intensive. In this work, we present a data-driven method of pairing events by their similarity via the energy mover's distance (EMD), which measures how similar two events are in terms of the work required to transform one into the other. With this approach, distinct events are sampled and matched by their similarity to serve as views for learning invariance, keeping the physics content of each event intact without handcrafted distortions. We demonstrate this augmentation-free pairing method by pre-training on QCD jets via self-distillation and show that it can yield semantic jet embeddings with downstream discrimination power comparable to or better than an augmentation-based baseline.

0.26

Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders

cs.AI astro-ph.HEcs.LGhep-ex ·2026-08-26 ·Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov, Vincenzo Piuri
similarity 0.61
Abstract · Submitted to SciPost Physics

We present a first application of sparse-autoencoder-based mechanistic interpretability to particle physics. Studying a neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction, we identify a validated atlas of physical concepts in the model representation, using a strict validation protocol consisting of held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Causal interventions show that the direction head barely draws on this atlas. Motivated by this underused information, we train an uncertainty head on the same event-level representation to predict the model's angular reconstruction error. Unlike the direction head, it depends causally on quality and brightness features from the atlas. These results suggest that mechanistic interpretability can reveal learned latent physics encoded within a model's internal representation and help design downstream tasks that exploit it.

0.41

Domain Adaptive Graph Neural Networks for Neutrino Experiments

cs.LG physics.data-anhep-ex ·2026-11-06 ·Hemalata Nayak, Giuseppe Cerati, Aleksandra Ćiprijanović, Tammy Walton
similarity 0.61
Abstract · This was also accepted in the ML4PS 2026 workshop

Weakly interacting particles such as neutrinos require highly efficient and precise detection methods to enable searches for new physics. Deep learning, and particularly Graph Neural Networks (GNNs), has shown promise for reconstruction tasks in liquid argon time projection chamber (LArTPC) neutrino experiments. However, differences between simulations and experimental data introduce domain shifts that can degrade model performance and potentially mimic or obscure new-physics signals. Here, we demonstrate that robust learning across datasets can still be achieved by using domain adaptation during training. We introduce, to our knowledge for the first time, domain-adaptation-enabled GNNs for robust LArTPC reconstruction tasks. We use MicroBooNE open dataset as our labeled source domain, and introduce four perturbed datasets as our unlabeled target domains-- including a fixed scale variation, moderate and severe gaussian noise, and a signal variation as a function of the position in the detector. The inclusion of adversarial-based domain adaptation during training improves performance for all four target domains, without a substantial drop in performance in the source domain. Precision in the target domain was improved up to 0.19 and recall up to 0.37, depending on the variation and the network task. These results demonstrate a potential of domain adaptation to bridge the gap between simulations and experiments, and build more trustworthy models for current and future LArTPC neutrino experiments.

0.27