CS Paper Radar

Numerical Embeddings for CTR Prediction: A Taxonomy, Unified Benchmark, and Open-Source FuxiCTR Extension

Heng Yao, Hanwei Zhu, Tianying Liu, Yulou Shu, Siyun Hou

cs.IR ·submitted 2026-10-31 16:24 UTC · v1
benchmarkcode/data

Abstract

Numerical embeddings map scalar features into vectors before a click-through rate (CTR) backbone combines the fields. To our knowledge, NumEmbBench is the first open benchmark dedicated to comparing numerical embeddings for CTR prediction. We classify 18 configurations into six mechanism-based families and evaluate them through one FuxiCTR interface. After equal-budget validation tuning, we compare frozen configurations on three datasets, nine backbones, and five seeds under a shared train-only z-score protocol. DEER has the best mean rank, but six methods lead at least one dataset--backbone setting. A matched native-scale test changes the winner in 11 of 27 settings, while paired latency measurements show that accuracy and inference cost need not move together. Code and per-seed results: https://github.com/YH-learning/NumEmbBench

Comment
5 pages

Closest papers

Nearest neighbours by meaning across the whole index.

ScalarLens: Numerical Embeddings with Stable Coordinates and Contextual Responses for CTR Prediction

cs.IR ·2026-09-24 ·Heng Yao, Tianying Liu, Yulou Shu, Yong He, Chuan Yuan +4 more
similarity 0.66
Abstract · 12 pages, 5 figures

Numerical embeddings for click-through rate (CTR) prediction are built on a convenient but restrictive premise: a scalar has one representation. This premise conflates where a value lies with what it means for the current sample. On the Criteo validation split, the same numerical interval carries residual click evidence with opposite signs across categorical and numerical contexts, even after additive main effects are removed. Production pipelines compound this mismatch because externally normalized features require transformations and statistics to remain synchronized between training and serving. We introduce ScalarLens, a numerical embedding that preserves what a value is while adapting how it should be interpreted. A monotone local mesh constructs a stable coordinate from the focal scalar alone; bounded low-rank dynamics then produce a contextual response without moving that coordinate or replacing categorical tokens and the CTR backbone. In a 1,539-run primary evaluation covering 19 representations, three datasets, nine backbones, and three seeds, ScalarLens ranks first in 25 of 27 settings on original numerical scales and second in the remaining two. Matched ablations show that scale correction, additional local capacity, and generic conditioning do not reproduce the gain. A controlled study further recovers categorical, numerical, and mixed response mechanisms under context shift while the focal coordinate remains exactly invariant. A complete rerun under shared standardization retains significant advantages over DEER, DAES, and NaryDis, showing that the result is not explained by tolerance to raw scales alone. ScalarLens therefore recasts numerical embedding as a measurement problem: coordinates belong to values, while predictive responses belong to values in context.

0.35

What DNA Foundation Models Learn Beyond Sequence Composition

cs.LG cs.AI ·2026-10-11 ·Vincenzo Yuto Civale, Andrew David Bagdanov, Alberto Magi
top venue similarity 0.57
Abstract · Accepted to NeurIPS 2026 (Evaluations & Datasets Track). 9 pages of main text, 5 main figures, with references and appendices

Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA FMs against normalized k-mer frequencies across 57 genomic classification datasets and two exploratory gene-expression regression tasks. Simple k-mer features remain competitive with most models, whereas NTv3 shows a consistent advantage; the ranking is robust to globally fixed and multi-scale k-mer baselines and alternative downstream probes. To test incremental predictive utility directly, we compare global 6-mer features with their concatenation with each FM embedding under a matched Random Forest protocol. NTv3 improves 46 of 57 datasets (mean $Δ$MCC = +0.072), while Caduceus provides smaller gains and the other models do not improve mean MCC. We separately decompose embeddings into a component linearly predictable from 6-mer frequencies and a ridge residual, finding that linear recoverability does not reliably indicate downstream utility. Exact 6-mer-preserving shuffles further reveal task-relevant sequence-order sensitivity, most pronounced for NTv3, although order sensitivity alone does not guarantee predictive improvements. Our results establish compositional features as essential controls and offer complementary diagnostics for evaluating the added value of frozen genomic representations, without making claims about fine-tuning capacity.

0.47

Pairwise Learning Increases Data Efficiency in Deep Neural Networks

cs.LG cs.AIcs.CV ·2026-10-20 ·Robert Bain, Marjan Rashidi, Elizabeth Chrastil
similarity 0.57
Abstract

Deep neural networks often fail at small-sample regression, collapsing to low-variance predictions that miss individual differences. Here, we introduce a pairwise comparison framework that reframes regression as aggregated relative rankings. This method improves data efficiency across model architectures and dataset sizes. On the UTKFace dataset (23k images), pairwise learning generates over one billion training comparisons without additional supervision. The paired ranking allows a lightweight ResNet10 trained from scratch on home hardware to achieve a 3.64-year MAE-surpassing previous models, including those with large-scale pretraining. For classification into categories, pairwise supervision consistently outperforms an otherwise identical 10-way crossentropy model on STL10 with 500-2000 labeled examples, improving accuracy by up to 5.2%. Overall, pairwise supervision enhances data efficiency and enables strong regression and classification performance when labeled data are limited, particularly in clinical settings.

0.49

AMLworld-Compact: Importance-Weighted Downsampling for Cost-Effective LLM Evaluation and Error Diagnosis

cs.CL ·2026-10-09 ·Natapong Nitarach, Phume Ngampornsukswadi, Pittawat Taveekitworachai, Surapon Nonesung, Warit Sirichotedumrong +1 more
code/data similarity 0.57
Abstract · 28 pages, 5 figures, 27 tables. Under review

Evaluating large language models (LLMs) on AMLworld's smallest test splits is costly because each of over a million targets requires a separate graph prompt. Sampling reduces this cost, but small balanced subsets change unweighted precision and F1. We introduce AMLworld-Compact, an evaluation set that retains every illicit test transaction and samples benign transactions from supervised-score strata. Its HT-Coreset protocol reduces evaluation targets by 271x and 611x on HI-Small and LI-Small. Horvitz-Thompson weights give unbiased full-test confusion-count estimates for fixed predictions; recall is exact, while precision and F1 are ratio estimates. In a held-out study using Graph Feature Preprocessor (GFP) features, one boosted-tree family defines strata for evaluating the other over 500 draws per configuration. At the released sizes, F1 RMSE is 0.00-2.58 percentage points versus 4.44-7.41 for uniform sampling with HT weights; conservative intervals remain broad. Under the evaluated graph prompts, seven open-weight LLMs have weighted precision near the predict-all-illicit baseline. In an outcome-stratified audit of 1,000 HI-Small few-shot outputs from one run per model, lexical checks pass for 84.5% of correct and 56.2% of incorrect binary verdicts, so these markers do not establish correctness. We release the evaluation sets, construction code, and audit tools at https://huggingface.co/datasets/typhoon-ai/AMLworldCompactEval and https://github.com/nat-nischw/AMLworld-Compact.

0.46

VX-One: Typed, Calibrated Decision Heads for Real-Time Perimeter Triage

cs.CR cs.CLcs.LGcs.NI ·2026-10-21 ·Juan S. Santillana
benchmark similarity 0.57
Abstract · 14 pages, 7 tables, 1 figure

Perimeter triage faces a latency/semantics trade-off: signature engines are fast and format-safe but blind to unseen attacks, while generative models are semantic but slow and unparseable. We present VX-One, a small multi-head network (100K--10M parameters) with calibrated typed outputs: a binary gate, a closed-set choice with an explicit unknown option where the benchmark supports it (and abstention elsewhere), and an ordinal score. Heads are temperature-calibrated, with 0.7--1.7 ms single-event CPU p50 latency. We evaluate UNSW-NB15, CIC-IDS2017, and EMBER2024 against calibrated XGBoost, Random Forest, MLP, and Kitsune baselines, and measure a Suricata-to-VX-One cascade over five days of CIC-IDS2017. Results are deliberately double-sided: on UNSW the gate ties the best baseline (0.9837+/-0.0002 versus MLP 0.9835+/-0.0013 across three seeds), while VX-One wins temporal CIC detection (0.734+/-0.022 versus 0.639+/-0.027) but loses multi-class attribution to XGBoost. Under leave-one-family-out, the gate detects all nine UNSW families but drops below chance on CIC recon; CIC trees transfer better. Suricata resolves only about 0.1% of attacks on flow reconstructions and 0.2--2.2% on payload-bearing PCAPs; the strictly by-day Phase 2 recovers only the in-scope DoS/DDoS share on the wave day (recall 0.34, precision 0.97). Calibration collapses under temporal drift (ECE 0.04->0.40) for every method while ranking partly survives. We contribute a measured account of where typed calibrated decisions help perimeter triage, and where they fail.

0.27

OlmoEarth v1.3: Embeddings

cs.CV ·2026-10-23 ·Gabriel Tseng, Favyen Bastani, Patrick Alan Johnson, Yawen Zhang, Piper Wolters +11 more
similarity 0.56
Abstract

We make some OlmoEarth embeddings. To do this, we train a new model: OlmoEarth v1.3. OlmoEarth v1.3 is designed to condense multimodal, multitemporal remote sensing data into a single digestible 64 or 128-dimensional vector. OlmoEarth v1.3 produces state-of-the-art embeddings. In addition, OlmoEarth v1.3 is a strong encoder backbone -- competitive with OlmoEarth v1 and v1.2 -- while requiring 18% and 53% of the GPU hours to train, respectively. We use OlmoEarth v1.3 to compute global, annualized 10m embeddings, which we release under the OlmoEarth artifact license.

0.55

When Are Predictions Enough? An Evaluation Protocol for Frozen Expert Composition

cs.LG cs.AI ·2026-11-09 ·Alessandro Pereira, Keith J Ransom, Lewis Mitchell
publishedbenchmark similarity 0.56
Abstract · Accepted, NeurIPS 2026, E&D track, poster

Practitioners often combine independently trained models while keeping their weights frozen. These experts are commonly combined through their final predictions, although their internal features may retain useful information that those predictions omit. We ask when combining features improves on combining predictions, and how to make that comparison fair. We introduce a reusable evaluation protocol, apply it to 21 frozen deepfake detectors from DF40 (Yan et al., 2024), and audit how evaluation choices change the conclusions. The protocol uses strong prediction baselines, compares simple and larger feature combiners, varies architecture and training data separately, and reports results by held-out generator with train/test overlap controls. Under leave-one-generator-out evaluation, logistic regression on the features of six architecture-diverse experts improves macro AUROC by 0.081 over the best tested prediction combiner in each fold, with gains on seven of eight generators. The larger network shows no clear improvement over the linear model (approximately 3.5M versus 10K parameters), suggesting that access to richer information matters more than additional combiner capacity on this pool. Features may help when they retain information that the predictions omit and that transfers to new generators. Predictions may be enough when the strongest prediction baseline already performs well, while neither approach reliably compensates for poor expert transfer. Weakening prediction baselines on the overlap-affected standard split inflates the apparent feature advantage by up to 0.172 AUROC. Overlap-aware evaluation also changes the estimated gain, though the contrast does not isolate overlap. Results by generator and expert pool show where the advantage holds, weakens, or reverses. Without these controls, the same benchmark can support different conclusions.

0.30

Data Curation and Model Souping: Unlocking Qwen3-0.6B for MS MARCO Passage Ranking

cs.IR ·2026-10-10 ·Shuguang Han
benchmarkcode/data similarity 0.56
Abstract · 12 pages, 9 tables

This report studies passage retrieval and re-ranking on the MS MARCO benchmark using lightweight Qwen3-0.6B models. Across three experimental settings --- BM25-then-reranking task, corpus-wide dense retrieval task, and dense retrieval then re-ranking task --- the results demonstrate that competitive ranking performance can be achieved without complex architectural modifications, custom loss functions, or cross-encoder distillation. Instead, substantial gains stem from data-centric strategies: corpus-targeted continued pre-training, rigorous data curation, hard-negative mining, training--inference distribution alignment, and weight merging via Model Soup. In particular, Model Soup delivers marked performance gains while preserving the inference efficiency of a single model. These findings highlight how data curation and weight merging can push sub-billion-parameter models to state-of-the-art performance. My models can be downloaded from Hugging Face: https://huggingface.co/shuguangh/models

0.40