CS Paper Radar

Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders

Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Inar Timiryasov, Vincenzo Piuri

cs.AI astro-ph.HEcs.LGhep-ex ·submitted 2026-08-26 17:53 UTC · v2

Abstract

We present a first application of sparse-autoencoder-based mechanistic interpretability to particle physics. Studying a neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction, we identify a validated atlas of physical concepts in the model representation, using a strict validation protocol consisting of held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Causal interventions show that the direction head barely draws on this atlas. Motivated by this underused information, we train an uncertainty head on the same event-level representation to predict the model's angular reconstruction error. Unlike the direction head, it depends causally on quality and brightness features from the atlas. These results suggest that mechanistic interpretability can reveal learned latent physics encoded within a model's internal representation and help design downstream tasks that exploit it.

Comment
Submitted to SciPost Physics

Closest papers

Nearest neighbours by meaning across the whole index.

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

cs.LG hep-ex ·2026-10-26 ·Dylan Sheldon Rankin, Ho Fung Tsoi
similarity 0.67
Abstract · 20 pages, 3 figures. Submission to SciPost

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

0.25

How Architecture and Training Affect TPC Representations Across Experiments

cs.LG cs.CVnucl-exphysics.ins-det ·2026-08-22 ·Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp +10 more
similarity 0.65
Abstract

Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional embeddings for four datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.

0.11

Domain Adaptive Graph Neural Networks for Neutrino Experiments

cs.LG physics.data-anhep-ex ·2026-11-06 ·Hemalata Nayak, Giuseppe Cerati, Aleksandra Ćiprijanović, Tammy Walton
similarity 0.64
Abstract · This was also accepted in the ML4PS 2026 workshop

Weakly interacting particles such as neutrinos require highly efficient and precise detection methods to enable searches for new physics. Deep learning, and particularly Graph Neural Networks (GNNs), has shown promise for reconstruction tasks in liquid argon time projection chamber (LArTPC) neutrino experiments. However, differences between simulations and experimental data introduce domain shifts that can degrade model performance and potentially mimic or obscure new-physics signals. Here, we demonstrate that robust learning across datasets can still be achieved by using domain adaptation during training. We introduce, to our knowledge for the first time, domain-adaptation-enabled GNNs for robust LArTPC reconstruction tasks. We use MicroBooNE open dataset as our labeled source domain, and introduce four perturbed datasets as our unlabeled target domains-- including a fixed scale variation, moderate and severe gaussian noise, and a signal variation as a function of the position in the detector. The inclusion of adversarial-based domain adaptation during training improves performance for all four target domains, without a substantial drop in performance in the source domain. Precision in the target domain was improved up to 0.19 and recall up to 0.37, depending on the variation and the network task. These results demonstrate a potential of domain adaptation to bridge the gap between simulations and experiments, and build more trustworthy models for current and future LArTPC neutrino experiments.

0.27

pi-Net: Empirical Latent Representations for Differentiable Physics Heads in Data-Scarce Systems

cs.LG cs.AI ·2026-10-12 ·Akshay Dhalpe, Arnab Chattopadhyay, Shahriar Bakrani Balani, Eric Cortanéa, Sasu Tarkoma
similarity 0.63
Abstract

Developing reliable predictive models for multi-physics systems often forces a compromise between computationally demanding physical simulators and data-hungry, opaque machine learning (ML) algorithms. While incorporating dimensional analysis into directed computation graphs offers a structurally sound alternative to explicitly solving partial differential equations (PDEs), such formulations typically yield underdetermined systems that act as imperfect simulators. We introduce pi-Net, a physics-informed latent variable model that resolves these imperfect simulators. The hybrid pi-Net architecture consists of an encoder that maps inputs into physically normalized latent space, which are then decoded into final predictions by a differentiable physics head. Evaluated on a real-world, data-scarce Gas Metal Arc Welding (GMAW) case study, the framework achieves an R2 of 0.864 with only 60 experimental samples. The results demonstrate that pi-Net significantly outperforms purely data-driven baselines in data-scarce regimes while providing causal, process-level explainability.

0.11

PhysFieldBench: Can Multimodal Models Understand Physical Fields?

cs.AI ·2026-09-28 ·Yuezhou Ma, Huikun Weng, Jialong Wu, Chenyi Zhao, Hang Zhou +3 more
benchmarkcode/data similarity 0.63
Abstract

Multimodal large language models (MLLMs) are increasingly envisioned as core components of scientific and engineering agents, yet their ability to interpret physical fields remains poorly understood. Existing physics benchmarks largely emphasize textbook problem solving or intuitive physical reasoning, leaving open whether MLLMs can infer physically meaningful information from continuous field observations. We introduce PhysFieldBench, a benchmark comprising 24 tasks and 1,160 evaluation examples across controlled equation fields, simulated physical fields, and observed physical fields. The tasks assess three forms of inference: identifying physical mechanisms, comparing latent control variables, and predicting outcome properties. Across representative open-source and proprietary MLLMs, zero-shot performance is low: the best model achieves a chance-normalized score of 29.3, while several open-source models remain near chance. In contrast, a task-specific supervised vision transformer performs substantially better, demonstrating that the inputs contain learnable physical information. To diagnose these failures, a structured self-explanation analysis attributes most errors to missed visual patterns and incorrect visual-to-physical mappings. Further, to explore whether post-training can improve physical inference and generalize to unseen tasks, we compare supervised fine-tuning with final answers or chain-of-thought supervision and reinforcement learning. Final-answer supervision performs best overall but transfers less effectively, whereas reinforcement learning after chain-of-thought supervision achieves the best generalization. Together, these findings highlight the need to improve visual-to-physical grounding and cross-task generalization for MLLMs to reliably interpret physical fields in scientific and engineering workflows.

0.32

Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC

cs.AI hep-exhep-phphysics.data-an ·2026-10-09 ·Donatella Genovese, Philip Ploner, Simone Scardapane, Stefano Giagu, Thea Klaeboe Aarrestad
benchmark similarity 0.62
Abstract · Submission to SciPost

Anomaly detection, the identification of collision events that deviate from Standard Model expectations, is one of the most promising model-agnostic strategies for new-physics searches at the Large Hadron Collider. Searching for rare events in learned low-dimensional representations removes the need to commit to hand-crafted observables a priori, but sacrifices the immediate physical interpretation those observables provide. For anomaly detection to enter a discovery pipeline, flagged events must therefore be diagnosed locally, both to distinguish potential new physics from mismodeling or detector effects and to identify observables for targeted follow-up searches. We evaluate supervised and self-supervised neural embedding strategies on COLLIDE-2V, a 750-million-event dataset simulated under realistic High-Luminosity LHC conditions, using the reconstruction error of an autoencoder trained in the learned latent space as the anomaly score. Across the configurations and benchmark signals considered, supervised objectives, which exploit simulation process labels during training, generally outperform their self-supervised counterparts and, in their best configurations, improve on an autoencoder trained directly on kinematic features, raising the signal efficiency at 10% background acceptance on all held-out processes. We further introduce a local diagnostic pipeline that fits a Gaussian Mixture Model to the SM latent space, profiles each component through high-level observables without process labels, localizes the flagged events, and ranks observables by their local discriminating power. The ranking recovers the defining kinematics of the held-out processes, while correlating the anomaly score with the same observables quantifies how closely the score tracks them, revealing cases where the most discriminating observable is not the one the score follows most closely.

0.11

SensOFormer: Foundation Models for Particle Accelerators

cs.LG physics.acc-ph ·2026-11-01 ·Mahindra Rautela, Alexander Scheinker
code/data similarity 0.61
Abstract

Large-scale scientific facilities, such as particle accelerators, are complex systems composed of thousands of interacting components that must operate collectively to support diagnostics and control, and ensure safe, reliable operation. These facilities rely on extensive sensor networks that generate large volumes of heterogeneous, incomplete, and noisy telemetry, often sampled at different rates across subsystems. In this paper, we adopt the foundation-model paradigm to learn general-purpose sensor representations from historical accelerator data through self-supervised pretraining. We introduce SensOFormer, a denoising masked transformer with a Perceiver-style encoder-decoder architecture designed to handle variable sensor sets, accommodate heterogeneous sampling rates, and learn robust representations from noisy measurements. The model is pretrained on multiple particle-accelerator datasets and transferred to downstream tasks, including missing-data imputation, anomaly detection, cavity identification, and fault identification. Through extensive evaluations and comparisons, we demonstrate that a single self-supervised model can learn reusable representations for diverse operational tasks in large-scale scientific facilities. Project page: https://mahindrautela.github.io/sensoformer.

0.30

Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics

cs.CV hep-ex ·2026-09-01 ·Samuel Young, César Jesús-Valls, Kazuhiro Terao
similarity 0.61
Abstract · 24 pages, 11 figures, preprint

Foundation models are increasingly being pursued in particle and nuclear physics, but existing approaches remain strongly tied to individual experiments through detector-specific architectures or pre-training objectives, limiting their reuse across sensing modalities. We show that a point cloud self-distillation framework yields a substantially more general sensor-level pre-training recipe. We show that the same refined architecture and objective can be independently pre-trained with minimal changes on three qualitatively different detector modalities: liquid argon time projection chamber (LArTPC), collider TPC, and water Cherenkov. Using 1,000 labeled images for downstream task adaptation, Panda V2 matches or exceeds specialized foundation-model baselines trained with orders of magnitude more supervision, matching state-of-the-art particle-clustering performance with 70x fewer labeled events on sPHENIX while substantially improving particle identification, and on LArTPC data matching Panda (arXiv:2512.01324) particle reconstruction with up to 1,000x fewer labels. Beyond reconstruction, simple linear probes reveal physically meaningful latent structure associated with particle causality and track curvature.

0.47