CS Paper Radar

A survey detection channel overrides the pixels in an astronomical foundation model, and biases tomographic mean redshifts

Ihor Kendiukhov

cs.AI astro-ph.IM ·submitted 2026-08-23 01:10 UTC · v1
survey

Abstract

Foundation models for astronomy are trained on survey pixels together with the catalogue products derived from those pixels. Those catalogues are incomplete at a measurable rate, and a model trained on both inherits that incompleteness as a systematic. We audit AION-1, a 39-modality transformer trained on more than 200 million objects, using causal interventions on its inputs. Holding the image tokens byte-identical and editing only the survey segmentation map changes every quantity the model reports -- flux, size, ellipticity, redshift -- by 110-4400 times a matched placebo. The mechanism is detection gating, presence at the field centre (r = 0.47), not the light the mask encloses (r = 0.30); across 322 real blends the model ignores how the pipeline partitioned the light (R = -0.006). Nor is the preference specific to that channel: contradicted catalogue photometry leaves the model nine times worse than supplying no metadata at all. The Legacy Survey pipeline leaves 3.68% of targets with no segment covering their position. Propagating that rate, with a miss represented by the fields the pipeline actually returns, shifts tomographic mean redshifts by a median 0.71 times the LSST DESC requirement over 40 assignments and exceeds it in 12; observed positional errors take the worst bin to 8.3 times. Drawing the misses by their measured magnitude dependence rather than uniformly does not change it. Spectroscopy removes the effect, withholding the detection channel removes it at no measurable cost, and the effect grows with model scale. Two further limits lie in the tokeniser: its image codec resolves 28 effective states on source patches against 934 for the spectrum codec, and the redshift readout is quantisation-limited. Sparse dictionaries are unreliable causal handles: across 15, recovery spans 26-75% and moves up to 18 points on the seed alone.

Closest papers

Nearest neighbours by meaning across the whole index.

A Mechanistic Interpretability Study of an Astronomy Foundation Model

cs.LG astro-ph.IM ·2026-11-11 ·Rachel C. Zhang, Alberto Bietti, Wes Gurnee, Alessandro Favero, Miles Cranmer
published similarity 0.64
Abstract · Accepted at NeurIPS 2026. A condensed version was selected for an oral presentation at the NeurIPS 2026 Workshop on Interpretability for Dis…

Foundation models are increasingly used across the sciences, but whether their internal computations correspond to known scientific operations or proceed through different mechanisms remains unclear. Understanding these mechanisms is essential for assessing reliability, identifying failure modes, and learning new science from the models. We present, to our knowledge, the first causal mechanistic interpretability study of an astronomy foundation model, AION-1, focusing on its multimodal advantage in redshift prediction, where combining imaging with photometry outperforms photometry alone. We find that this advantage is largest for spatially extended and dust-attenuated galaxies, and that most of it is localized to the first of the encoder's twelve blocks. Within that block, how well a property is linearly decodable does not predict whether it can be localized. Spatial extent is causally used, and one attention head dominates linear probing for it, yet ablating that head has negligible effect on redshift prediction: the information is distributed across heads and is decodable again one layer after we erase it. Dust attenuation shows the opposite pattern: it is only weakly decodable anywhere in the encoder, yet a single head is necessary for most of the multimodal advantage, particularly for the most dust-attenuated galaxies, and keeping that head as the only active head in its block preserves nearly all of the advantage. None of the astrophysically motivated operations we tested explains what this head computes. Whether this reflects a limitation of current interpretability methods or a computation that does not correspond to any known astrophysical operation remains open, and resolving it is a prerequisite for using interpretability as a tool for scientific discovery.

0.28

Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts

cs.CV astro-ph.IM ·2026-11-04 ·Eraldo Pereira Marinho
similarity 0.63
Abstract · 17 pages and 3 figures

Reported performance in small astronomical image datasets can reflect object reuse, field-of-view differences, acquisition settings, and catalogue-derived properties as well as astronomical morphology. We designed \dataset{} to measure these effects in a four-class task comprising galaxies, globular clusters, nebulae, and open clusters. The dataset contains 370 catalogue-supported physical objects, each represented by three Digitised Sky Survey FITS bands, and uses a group-constrained split of 222 training, 75 validation, and 73 test objects. We compared a custom CNN--Transformer trained from scratch with ImageNet-pretrained ResNet-50 and ViT-B/16 under an adaptive full-field representation, a common angular field, and two central-region-masked controls, using three fixed seeds. In the one-pass object-independent test, mean balanced accuracy on the adaptive view was $0.773\pm0.014$, $0.874\pm0.054$, and $0.866\pm0.023$ for the custom model, ResNet-50, and ViT-B/16, respectively. No primary architecture contrast rejected its null after Holm correction; ResNet-50 and ViT-B/16 differed by 0.007 balanced-accuracy points (95\% cluster-bootstrap interval $[-0.048,0.062]$). Controls based on acquisition and catalogue metadata reached 0.721--0.735, and central-region-masked adaptive views reached 0.731--0.800. These results demonstrate generalisation to unseen physical objects, but also show that image context, acquisition settings, and catalogue-derived object properties carry substantial class information. In this regime, high accuracy cannot be interpreted as evidence that a classifier learned target morphology, and the comparison does not isolate architecture from pretraining.

0.20

Star Light, Star Bright: Can Multimodal Models Parse an Astronomy Visual Question Answering Dataset Right?

cs.CV astro-ph.IMcs.DLcs.IR ·2026-10-15 ·J. P. Naiman
publishedcode/data similarity 0.60
Abstract · 5 main pages, 12 pages total, accepted to The 4th Annual Workshop for Artificial Intelligence for Scientific Publications (WASP) 2026, Inter…

Figures are essential for scientific communication, and the field of Visual Question Answering (VQA) aims to test the figure understanding capabilities of large multimodal models (LMMs). Here, we investigate the accuracy of several low-cost LMMs on domain-specific VQA tasks for astronomical images of the sky. As instrument for this task we build a domain-specific VQA dataset comprising 2001 partially-synthetic figures divided evenly between contour plots, real telescope images drawn from NASA's SkyView, and "fake" sky images which share the axes and angular scales of the real images but draw their colors from a gaussian mixture model, alongside 47357 questions. Each figure is released as pristine and with realistic page-aging degradations mimicking the historical astronomical corpus. Testing three low-cost proprietary LMMs, we find models frequently fail to answer domain-specific questions at high accuracy. An initial comparison with 104 human-verified, astrometrically tagged figures from historical journal publications shows similar median errors in angular extent. All code is available on GitHub: https://github.com/jnaiman/SkyImagesWASP2026 and dataset is available on HuggingFace: https://huggingface.co/datasets/ReadingTimeMachine/astro_sky_image_vqa.

0.62

COSMIO: A Benchmark for Cross-Survey Modality Imputation

cs.CV astro-ph.IM ·2026-11-03 ·Dichang Zhang, Yixuan Shao, Jiaqi Deng, Jiali Cui, Haotian Yin +6 more
top venuebenchmarkcode/data similarity 0.60
Abstract · Accepted at NeurIPS 2026. 34 pages, 8 figures

Large-scale astronomical surveys define a heterogeneous, partially observed multi-survey learning setting: each object is observed by only a subset of surveys, while survey observations differ in resolutions, wavelength coverage, noise characteristics, and footprints. This setting gives rise to a natural machine learning problem: given observations of an object in some surveys, how to infer its appearance in other surveys. We formalize this as the problem of cross-survey modality imputation and introduce COSMIO, a multi-survey benchmark of aligned observations obtained by forward-rendered shared astrophysical scenes across LSST, Euclid, and Roman under survey-specific instrument characteristics. The benchmark enables systematic evaluation of imputation across source-target survey combinations and missing-survey patterns. We evaluate a broad range of machine learning methods for imputation, which reveal a clear domain gap: methods developed for existing imputation settings transfer poorly to the survey setting. We identify rare-event imputation as a central challenge. This regime arises in the early stages of new surveys, when rare but scientifically valuable phenomena may be available only from prior surveys. We use strong gravitational lensing, where light from a background galaxy is gravitationally deflected by a foreground galaxy, as a representative rare-event setting. We evaluate model performance on zero-shot imputation of gravitational lenses and on downstream scientific tasks using the imputed lens observations. On our constructed evaluation set of galaxy-galaxy strong lenses, performance degrades under rare-event distribution shift, and this degradation further propagates to the downstream lens-detection task. Code: https://github.com/fw-ic/COSMIO.

0.63

NEO-BENCH: A New Multi-Source Benchmark for Generalizable Astronomical Streak Detection

cs.CV astro-ph.IM ·2026-09-06 ·Jiayou He, Jessica Yao
benchmarkcode/data similarity 0.59
Abstract · The code and data used in NEO-BENCH are publicly available on GitHub at https://github.com/he-jiayou/NEOBench and on Hugging Face at https:/…

Near-Earth Objects (NEOs) can appear as faint streaks in long-exposure astronomical images. Detecting these streaks across diverse observatories requires methods that remain reliable despite differences in image quality, orientation, sky background, and noise. However, existing detectors are commonly evaluated using data from only one source, providing limited evidence of cross-source generalization. We introduce NEO-Bench, a multi-source benchmark containing 8,376 images from five astronomical-image datasets. The sources include the Hubble Space Telescope, a Stellina smart telescope, the United Arab Emirates Meteor Monitoring Network, a TETRA1 telescope using a Celestron C14 with Fastar, and the Roboflow Asteroid dataset. We converted the data to a common YOLO format, audited a sample of labels, and defined within-source and leave-one-source-out evaluation protocols. We evaluated four approaches: Hough, Radon, Gaussian PSF, and YOLO26L. Leave-one-source-out F1 decreased in 14 of 20 image-level method-source pairs and 13 of 20 IoU@0.50 localization pairs. Across the datasets categorized as medium or hard, F1 decreased in 11 of 12 image-level pairs and 9 of 12 localization pairs. These results show that cross-source performance remains inconsistent and that reliable generalization across astronomical imaging sources remains an open challenge. The benchmark, code, and data are publicly available.

0.30

Gestalt: a meta-foundation model for astronomy

cs.LG astro-ph.IM ·2026-09-29 ·Michael J. Smith, Shashwat Sourav
published similarity 0.59
Abstract · 12 pages, 3 figures, 6 tables, accepted to the Interpretability for Discovery workshop at NeurIPS 2026

The Platonic Representation Hypothesis predicts that sufficiently scaled foundation models converge on a shared representation of the world. As each non-converged model gives a noisy view of a common structure when passed the same input, we ask whether we can combine models into a representation that outperforms its individual components. We test this on galaxies: we embed images via a basket of 22 frozen foundation models from eight families, whiten each view, and take a randomised SVD of the embedding concatenation. The resulting 1024-dimensional embedding outperforms every basket member on 19/21 of our tested metrics for physical property and galaxy morphology estimation for HSC, JWST, and DESI Legacy Survey imagery. We find that performance rises with basket size and basket architectural diversity, and that the meta-foundation model's performance transfers across astronomical surveys. We conclude that a useful astronomical foundation model can be assembled from existing generalist models with no training required beyond a single unsupervised projection. By leveraging the community's already-spent work, we save a lot of compute: a fresh pre-train of a comparable single-domain model would cost $\mathcal{O}(10^{4}$--$10^{5})$ A100 GPU hours (emitting several tonnes of CO$_2$eq.), whereas assembling Gestalt requires minutes on a single machine.

0.30

Feature-Based and Deep Learning Methods for Astronomical Light Curve Classification

cs.LG astro-ph.IM ·2026-11-03 ·V. Baymakanov, Kh. Khamitov, A. Raimova, A. Isaeva, R. Taran +5 more
hot direction similarity 0.58
Abstract · 14 pages, 5 figures. Submitted to Moscow University Physics Bulletin (DLCP-2026 special issue)

We compare feature-based machine learning and end-to-end deep learning for variable star classification using the Zwicky Transient Facility (ZTF) StarEmbed dataset, which contains about 40,000 objects across seven imbalanced classes. Classical models trained on physically motivated features consistently outperform deep networks trained directly on raw light curves. We find that limited labeled data contributes to the performance gap, while the multivariate, irregularly sampled, and morphologically diverse nature of light curves poses additional challenges for deep learning models. While deep learning holds promise for learning representations directly from observations, our results indicate that current models fail to generalize with the available training set size. These findings motivate further investigation of the benefits of scaling labeled datasets and self-supervised pretraining on large unlabeled surveys. This work establishes a quantitative baseline for future research and highlights the need for further innovations to unlock the potential of deep learning in astronomical time series analysis.

0.28

Amplifying the imaging power of digital sky surveys with space telescopes data and generative AI

cs.AI astro-ph.IMastro-ph.GAcs.LG ·2026-08-21 ·Sai Teja Erukude, Lior Shamir
publishedcode/data similarity 0.58
Abstract · MNRAS, accepted

While Digital sky surveys provide excellent throughput of image data and can cover a large footprint, their imaging power is normally inferior to that of space-based telescopes. Space-based telescopes, on the other hand, provide excellent imaging power and can image the deep Universe, but cannot provide the same throughput as advanced ground-based sky surveys. Here, we utilize generative AI to elevate the quality of galaxy images taken by ground-based telescopes to the level of details enabled by space telescopes. The solution is based on the nature of galaxy shapes, allowing generative AI trained on space-based images to convert weak signal into detailed and clear galaxy images. The method allows for combining the high throughput of ground-based sky surveys with the image quality of space-based telescopes. The source code for the method is available, as well as paired training data and a catalog of 63,202 galaxy images enhanced by the proposed method. We also provide a software tool that encapsulates the entire pipeline and the custom generative AI model to generate galaxy images with enhanced quality.

0.58