CS Paper Radar

Contrastive Astromer: Two-Stage Contrastive Representation Learning for Astronomical Light Curve Classification

Justin Minggan Wei, Pavlos Protopapas, Guillermo Cabrera-Vives

cs.CV astro-ph.IM ·submitted 2026-10-30 02:33 UTC · v1

Abstract

Large astronomical surveys collect abundant light curves, but reliable class labels remain scarce. ASTROMER 2 learns from unlabeled light curves by reconstructing masked observations, yet this objective does not explicitly encourage class separation in a compact representation of each light curve. We introduce Contrastive Astromer, a two-stage approach that makes these representations more useful for classification without changing the ASTROMER 2 encoder architecture. First, unsupervised contrastive learning on single-band MACHO light curves brings together representations of augmented views of the same object. Supervised contrastive learning, supported by an auxiliary classification loss, then uses a separate labeled Alcock sample to strengthen class discrimination. We evaluate the resulting representations by freezing the encoder and training a separate classifier. On a held-out set of 600 objects, this classifier achieves a macro-F1 of 0.7943 with 500 labeled examples per class, exceeding our reproduction of the ASTROMER 2 finetune-and-classify pipeline by 5.10 percentage points. With downstream classifier training budgets of 20 and 100 examples per class, macro-F1 reaches 0.7078 and 0.7704, respectively, also surpassing this baseline. Embedding-space analyses further show that supervised adaptation improves local class consistency and global class separation. These results show that contrastive adaptation makes pretrained light-curve representations more effective for classification and more informative about the structure of variability classes.

Closest papers

Nearest neighbours by meaning across the whole index.

Feature-Based and Deep Learning Methods for Astronomical Light Curve Classification

cs.LG astro-ph.IM ·2026-11-03 ·V. Baymakanov, Kh. Khamitov, A. Raimova, A. Isaeva, R. Taran +5 more
hot direction similarity 0.72
Abstract · 14 pages, 5 figures. Submitted to Moscow University Physics Bulletin (DLCP-2026 special issue)

We compare feature-based machine learning and end-to-end deep learning for variable star classification using the Zwicky Transient Facility (ZTF) StarEmbed dataset, which contains about 40,000 objects across seven imbalanced classes. Classical models trained on physically motivated features consistently outperform deep networks trained directly on raw light curves. We find that limited labeled data contributes to the performance gap, while the multivariate, irregularly sampled, and morphologically diverse nature of light curves poses additional challenges for deep learning models. While deep learning holds promise for learning representations directly from observations, our results indicate that current models fail to generalize with the available training set size. These findings motivate further investigation of the benefits of scaling labeled datasets and self-supervised pretraining on large unlabeled surveys. This work establishes a quantitative baseline for future research and highlights the need for further innovations to unlock the potential of deep learning in astronomical time series analysis.

0.28

Object-independent evaluation of small-data astronomical image classifiers: architecture, representation, and acquisition shortcuts

cs.CV astro-ph.IM ·2026-11-04 ·Eraldo Pereira Marinho
similarity 0.65
Abstract · 17 pages and 3 figures

Reported performance in small astronomical image datasets can reflect object reuse, field-of-view differences, acquisition settings, and catalogue-derived properties as well as astronomical morphology. We designed \dataset{} to measure these effects in a four-class task comprising galaxies, globular clusters, nebulae, and open clusters. The dataset contains 370 catalogue-supported physical objects, each represented by three Digitised Sky Survey FITS bands, and uses a group-constrained split of 222 training, 75 validation, and 73 test objects. We compared a custom CNN--Transformer trained from scratch with ImageNet-pretrained ResNet-50 and ViT-B/16 under an adaptive full-field representation, a common angular field, and two central-region-masked controls, using three fixed seeds. In the one-pass object-independent test, mean balanced accuracy on the adaptive view was $0.773\pm0.014$, $0.874\pm0.054$, and $0.866\pm0.023$ for the custom model, ResNet-50, and ViT-B/16, respectively. No primary architecture contrast rejected its null after Holm correction; ResNet-50 and ViT-B/16 differed by 0.007 balanced-accuracy points (95\% cluster-bootstrap interval $[-0.048,0.062]$). Controls based on acquisition and catalogue metadata reached 0.721--0.735, and central-region-masked adaptive views reached 0.731--0.800. These results demonstrate generalisation to unseen physical objects, but also show that image context, acquisition settings, and catalogue-derived object properties carry substantial class information. In this regime, high accuracy cannot be interpreted as evidence that a classifier learned target morphology, and the comparison does not isolate architecture from pretraining.

0.20

What AstroPT knows about galaxies, and what that can teach us about LLMs

cs.LG astro-ph.IM ·2026-08-23 ·UniverseTBD, :, Kshitij Duraphe, Aman Kumar, Michael J. Smith +1 more
publishedcode/data similarity 0.59
Abstract · 13 pages, 12 figures, code at https://github.com/Smith42/astroPT, accepted at Sci-FM@COLM 2026

Interpretability research increasingly asks when concepts emerge during training and whether linear probes recover real structure, but in language models these claims are hard to validate because language offers little ground-truth ordering of concepts or relationships among them. We propose the use of astronomical ground truth through AstroPT, a transformer trained on millions of galaxy images, as a calibration testbed. AstroPT is an LLM-like model trained within a domain where the difficulty ordering of concepts and the relations among them are known in advance. Probing frozen representations across checkpoints, layers, model sizes, and objective choices, we find that galaxy properties emerge in a fixed order that tracks their known difficulty---quantities written almost directly into the pixels (band magnitude) become decodable early in training and shallow in the network, while multiband/spectra based and inferred quantities (such as redshift and specific star formation rate) emerge later and deeper. This order is invariant to our tested training objectives, and scales in magnitude but not in sequence with capacity. Our linear probe directions further recover the known physical structure among galaxy properties. Our findings suggest that astronomy offers a controlled sandbox for calibrating mechanistic interpretability methods we otherwise apply to LLMs blind.

0.59

UnEZ: A Fully Unsupervised Self-Templating Neural Network for Redshift and Line Strength Estimation

cs.LG astro-ph.IMastro-ph.GA ·2026-10-28 ·Asha Abd, Karl Glazebrook, Colin Jacobs
benchmark similarity 0.59
Abstract · 22 pages, 17 figures. Submitted to MNRAS

As spectroscopic surveys grow, estimating galaxy properties such as metallicity, star formation rate, and redshift becomes computationally cumbersome. Methods of estimating such properties rely on template spectra or curated redshift catalogs, which fail to represent the diversity of objects seen in large-scale surveys. We present UnEZ, a fully unsupervised, physics-informed machine learning model that can predict spectroscopic redshifts and line strengths with no prior redshift or line strength labels. An encoder predicts emission and absorption line amplitudes/widths, and a fixed deterministic decoder constructs a unique template, estimated redshift, redshift confidence curve, and spectral model. UnEZ was tested on the Sloan Digital Sky Survey (SDSS) Data Release 18 due to its large dataset and thoroughly vetted redshift catalog which serves as an independent benchmark. Despite the lack of templates, line strength labels, or redshift labels, UnEZ's performance competes with modern redshift estimation algorithms. At SNR > 10, UnEZ achieves an outlier fraction of 0.459% and a normalized median absolute deviation (NMAD) of $1.237 \times 10^{-4}$. At SNR > 3, UnEZ achieves an outlier fraction of 3.89% and an NMAD of $1.471 \times 10^{-4}$. Furthermore, we demonstrate how UnEZ's line strength and redshift predictions can be utilized for various purposes such as constructing template spectra required in traditional redshift estimation algorithms. UnEZ illustrates how a fully unsupervised neural network can allow redshift, among other galaxy properties, to emerge naturally from the spectra themselves, providing a template-free, label-free approach that scales to the demands of future surveys.

0.33

Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference

cs.LG astro-ph.COastro-ph.IMphysics.data-an ·2026-08-26 ·Yannic Pietschke, Caroline Heneka, Ayodele Ore, Romain Meriot
similarity 0.58
Abstract · 12 pages, 5 figures, prepared for submission to A&A

Simulation-based inference (SBI) for parameter estimation is vulnerable to model misspecification: neural summaries and density estimators trained on a specific forward model typically fail when applied to data drawn from another model, or from real observations, and no training simulator can capture the full observational pipeline of a real measurement exactly. We show that a self-supervised Vision Transformer (ViT), pretrained label-free on a fast approximate simulator, produces transferable data summaries that generalize across simulators. Without retraining, it can be reused as a frozen encoder to infer astrophysical parameters from a completely different simulator that resolves the radiative transfer explicitly, on which it has never seen either data or parameters. As a concrete use case in 21cm cosmology, SKATR, a ViT pretrained with a Joint Embedding Predictive Architecture (JEPA), serves as a foundation model for reionization inference from upcoming SKA measurements: SKATR is pretrained once on 67k low-cost, noiseless semi-numerical 21cmFAST lightcones, then frozen and applied to hydrodynamical Loreli II lightcones, where a lightweight conditional flow matching head infers five astrophysical parameters; the encoder is never shown Loreli data, its parameters, or any noise. In our comparison, SKATR yields the most precise and best-calibrated posteriors across all five parameters, matching the accuracy of the fully-supervised in-domain baseline while requiring 2.6x fewer radiative-transfer simulations. Under realistic SKA AA* noise, only SKATR remains simultaneously accurate, informative, and calibrated, outperforming even a supervised baseline retrained from scratch on noisy data. Self-supervised pretraining on computationally efficient semi-numerical simulations is therefore a viable route to calibrated, simulator- and noise-agnostic reionization inference for the SKA-era.

0.35

Bayesian classification of astronomical spectra with class uncertainties

cs.LG astro-ph.IMphysics.data-an ·2026-09-18 ·Simon Barton, Martin Sahlén, Andreas Korn, Christian Glaser
published similarity 0.57
Abstract

Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.

0.41

Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra

cs.LG physics.space-ph ·2026-09-25 ·Matthieu Le Lain, Gaël Cessateur, Sébastien Lefèvre
benchmarkunusual similarity 0.56
Abstract · 5 pages, 1 figure, 3 tables. Submitted to IEEE ICASSP 2027

Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.

0.28

Improving Faint Object Detection for Space Situational Awareness with Variational Autoencoders

cs.CV cs.AIcs.LG ·2026-09-10 ·Angela Cratere, Luca Ghilardi, Vishnu Reddy, Francesco Dell'Olio, Charalampos S. Kouzinopoulos +1 more
published similarity 0.56
Abstract · Accepted at SPAICE 2026: the 3rd European Space Agency Conference on AI in and for Space

We present a deep-learning pipeline for enhancing the detection of faint moving objects in optical space situational awareness (SSA) imagery through automated star removal and background reconstruction. Detecting low signal-to-noise ratio (SNR) objects remains extremely challenging in optical observations, particularly in the cislunar (X-GEO) environment, where structured sky backgrounds, dense stellar fields, and scattered moonlight significantly degrade the performance of classical detection algorithms. To address this problem, the proposed pipeline combines a lightweight segmentation network (Tiny-U-Net) to generate stellar masks with a partial-convolution variational autoencoder (astro-VAE), designed to learn the statistical distribution of astronomical backgrounds and perform context-aware inpainting of masked regions. The reconstructed background maps can then be used as a preprocessing step to suppress fixed sources and background inhomogeneities prior to detection. As a proof of concept, the approach is integrated with a shift-and-stack scheme and evaluated on real ground-based telescope observations targeting the X-GEO region. Results demonstrate that the method reconstructs star-free backgrounds with high fidelity, while preserving moving targets and significantly enhancing detectability, thereby providing an effective data-driven preprocessing strategy for faint moving-object detection in optical SSA scenarios.

0.38