CS Paper Radar

Where Do LLMs Learn to Recommend AI?

Benaya Trabelsi, Jonathan Shaki, Sarit Kraus

cs.CL cs.AIcs.CYcs.LG ·submitted 2026-10-21 14:13 UTC · v1

Abstract

Asked what to study, which career to pursue, or what product to build, large language models (LLMs) often recommend AI-related options. We ask where this tendency comes from by comparing models before and after fine-tuning and by varying how questions are asked. First, pretrained models already include explicit AI/ML terms in the recommendations of 27% of their responses, and their released instruction-tuned versions do so in 59%. Second, fine-tuning models ourselves shows that this rise is not inevitable: general instruction tuning leaves the rate unchanged, but removing the 7% of instruction examples that concern AI cuts it by 14 percentage points, below the level of the pretrained models it starts from. Data rewarding helpfulness raise it by 9 points, while data ranked by the preferences of diverse simulated users lower it by 9. Finally, with models held fixed, asking for the "best" option or for what to do "right now" raises the rate by 8 and 18 points, whereas telling the model it is an AI advisor adds 4. Overall, the tendency starts in pretraining, and what models learn afterward, together with how a question is phrased, decides how strongly it appears.

Comment
18 pages, 4 figures. Companion paper to arXiv:2601.13749

Closest papers

Nearest neighbours by meaning across the whole index.

What Should an LLM Know? Knowledge-Guided Warm Starts for Software Optimization

cs.SE ·2026-11-16 ·Srinath Srinivasan, Tim Menzies
similarity 0.67
Abstract

Large language models (LLMs) process gigabytes in minutes; human experts work far more slowly. Prior studies that explored combinations of human/AI optimizers ignore this gap. Those studies asked systems or humans to label too many examples (e.g., SynthCore needs over four dozen per trial). Here, we test human and AI knowledge under a budget where evaluation is practical (a few dozen labels or fewer). We compare five baselines (random search, RRP, EZR, few-shot prompting, and a budget-matched SynthCore variant) against three ways to give an LLM domain knowledge before it proposes its first configurations: (1) AMP lists the domain's rules, then checks each candidate against them; (2) HKMA adds scouted data or documentation to the prompt; and (3) H-DKP has human experts correct the LLM's rules. We find that domain knowledge improves the "start-up" phase of LLM optimization, but only when that knowledge is checked, by the LLM itself or by a human expert. Across 49 datasets, unchecked knowledge did no better than few-shot prompting or EZR. On the other hand, AMP's self-checking variant (AMP-4) reached the best statistical rank in 44 of 49 datasets (budget-matched SynthCore: 23 of 49), and H-DKP did so in 7 of 8 expert studies. These checks cost little: half a cent of LLM calls per trial, or 6 to 10 expert contacts. Hence we say LLMs may not need more data, but rather their knowledge needs checking (and checking is cheap).

0.33

AI Research Preference Models

cs.AI ·2026-08-14 ·Thomas Simon Foster, Bassel Al Omari, Tingchen Fu, Thomas Mann, Carl Domond +28 more
benchmarkhighly cited similarity 0.64
Abstract · 34 pages, 17 figures, 6 tables

AI research agents (AIRA) can now carry machine learning experiments from proposal through implementation and evaluation. Yet progress on frontier tasks is throttled by the cost of evaluations that can consume days of GPU time. When an agent can propose far more candidates than it can afford to run, progress depends on its research preference: how it allocates a fixed execution budget across many candidates. We introduce AI Research Preference Models (RPMs) that predict which candidate solution is most promising, without paying the cost of running them all. We build RPMs from frozen pretrained language models in two variants: an inference-only model that reasons over candidate plans, code, and previously executed solutions, and an agentic model that additionally runs small-scale pilot experiments. Integrated into the AIRA-dojo research agent and evaluated on the machine learning research benchmark AIRS-Bench, the two variants increase the average normalized score from 0.684 to 0.711 and 0.729, respectively. Both reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget, and together yield new state-of-the-art results on two AIRS-Bench tasks.

0.50

Large Language Models: A Plain-Language Overview for Engineers

cs.CL cs.AIcs.LG ·2026-10-25 ·Bahaeddin Türkoğlu
similarity 0.64
Abstract · 11 pages, 2 figures, 3 tables, 16 references. Author's preprint; to be submitted to a peer-reviewed journal

Large language models (LLMs) are neural networks trained on huge text collections to generate text by predicting what comes next. Engineers using ChatGPT and other generative artificial intelligence (AI) tools need a clear picture of what training changes and prompts add. The article starts with the Transformer, which connects words in context, and pre-training, where models learn to predict text. It then explains how fine-tuning adapts models to requests, with reinforcement learning from human feedback using human answer rankings. Next, retrieval-augmented generation (RAG) supplies documents, while chain-of-thought prompting encourages intermediate steps and agents use tools. FlashAttention lowers training and running costs by reducing data movement during attention; quantized low-rank adaptation (QLoRA) saves fine-tuning memory. Readers get a life cycle figure and tables comparing methods and their limits. A short checklist for judging LLM evaluation claims flags test material in training (data contamination) and model-based grading (LLM-as-a-judge). It also covers very long inputs (long context) and hallucination, meaning false or unsupported claims. Check factual accuracy separately from whether an answer sounds helpful.

0.28

DualEvolve: Dual-Level Self-Evolution of Mechanism and Orchestration for Recommendation

cs.IR ·2026-11-03 ·Tianqi Gao, Chenxu Wang, Wenxuan Yang, Yueqi Mou, Biao Tang +2 more
code/data similarity 0.64
Abstract · 12 pages, 4 figures, 15 tables. Tianqi Gao and Chenxu Wang contributed equally. Code: https://github.com/ToAdventure/dualevolve

Recommender systems are widely used in e-commerce and content platforms, where improvement requires repeated model design, training optimization, and experimental validation. Large language model (LLM) agents increasingly automate this development cycle. Sustained autonomous improvement requires agents to improve both recommendation models and their development workflows. However, task-performance-driven approaches select updates mainly by candidates' final scores, without a separate optimization signal for experimentation quality. We therefore present DualEvolve, a dual-level framework that co-evolves recommendation mechanisms and agent orchestration toward recursive self-improvement. Inner evolution revises model and training code using task feedback to improve recommendation performance. Outer evolution scores the inner-loop experimental process, updates harness and skill rules using these scores and execution traces, and adjusts orchestration choices based on next-round scores to improve the quality of subsequent experiments. Experiments on MovieLens-1M and three Amazon Reviews 2023 subsets cover four recommendation methods and 16 settings. DualEvolve achieves higher test means than Base, OpenEvolve, and DeepEvolve in all 64 setting-metric comparisons, with macro-averaged relative gains over DeepEvolve of 7.6% (HR@5) and 9.0% (NDCG@5).

0.30

"High-Quality" Data for Early-Stage Instruction Tuning: An Empirical Study of Data Quality Metrics

cs.CL ·2026-10-25 ·Vladimir Necula, David Gvadzabia, Sofia Serrano
similarity 0.63
Abstract

"Data quality" is widely invoked in instruction tuning, but the term is often underspecified. We study it as an operational property by asking which simple example-level metrics, drawn from prior work on data selection, text quality, and instruction tuning, best predict training utility under a fixed finetuning budget. For each metric, we sort instruction-response examples by score, partition them into low, middle, and high bins with roughly equal token mass, finetune separately on each bin, and compare validation perplexity and LLM-as-judge scores for the models' generated answers as a function of cumulative training tokens. Our results suggest that answer-dense supervision, as captured by response ratio, is a stronger and more portable predictor of instruction-tuning utility than commonly used alternative proxies.

0.20

StudentBench: AI and human tutoring yield equivalent GRE learning gains

cs.AI cs.CY ·2026-09-23 ·Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner +1 more
code/data similarity 0.63
Abstract · 47 pages, including references and appendices. Data: https://huggingface.co/datasets/handshake-ai-research/studentbench Code: https://github…

Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). To support future research, we open-source the de-identified data collected in our studies.

0.32

Don't Blame The Preferences: Examining The Emergence of Sycophancy in LLM Training

cs.AI ·2026-11-05 ·Chao Peter Yang, Catherine Zehr, Pranav Bijith, Brandon Fain
benchmark similarity 0.63
Abstract

AI sycophancy, the tendency of large language models (LLMs) to flatter users and affirm their beliefs, can undermine accuracy and contribute to misinformation, dependence, and overconfidence. Although prior work has benchmarked sycophancy in released models and proposed mitigations, its origins during training remain poorly understood. Preference alignment methods such as RLHF and DPO are often blamed, but their effects are rarely compared with earlier training stages. We investigate sycophancy across the training pipelines of three model families and three benchmarks. We find that sycophancy is largely established by the end of supervised fine-tuning (SFT), rather than introduced by preference alignment. Across all nine model benchmark combinations, SFT checkpoints attain at least 92% of preference aligned models' sycophancy scores and at least 85% of final models' scores; on average, they are about as sycophantic as final models. Preference alignment increases sycophancy in only four of nine cases, by an average of 4.2 percentage points when it does. To probe the underlying causes, we isolate model representations associated with sycophancy using mechanistic interpretability and measure their activation on training datasets from each stage. SFT datasets show roughly the same average activation as other training data but contribute substantially more examples, and the highest-scoring individual datasets all come from SFT. Our findings suggest that addressing sycophancy requires interventions throughout the training pipeline, particularly in data selection and augmentation for pre-training and SFT.

0.10

AutoRSE: Evaluating LLM Agents as Recommendation Engineers for Self-Evolving Recommender Systems

cs.IR ·2026-10-27 ·Rui Tian, Yi Fang, Yejin Hua, Du Wang, Yuzhe Wang +6 more
code/data similarity 0.63
Abstract · 12 pages, 7 figures. Code: https://github.com/Ray-Tiann/AutoRSE

Recommender Systems Engineer Agents (RSE-Agents), which use LLM-based agents to autonomously improve recommendation algorithms, are emerging as a promising paradigm. However, their capabilities and limitations remain poorly understood due to the lack of a unified, standardized evaluation framework and systematic trajectory-level analysis. To address this gap, we introduce AutoRSE, an evaluation framework spanning six representative industrial-grade recommendation tasks, and conduct a large-scale empirical study of three frontier LLMs across three widely used agent harnesses. Our results show that RSE-Agents consistently enhance the performance of backbones on five of the six tasks, with an average relative improvement of 16.1%, indicating that they can already autonomously improve recommendation performance. However, trajectory-level analysis also reveals that current RSE-Agents function primarily as engineering optimizers rather than algorithmic improvers: their gains are largely driven by hyperparameter tuning, training strategies, and model ensembling, while genuine algorithmic innovations are rare. Since the largest gains in recommender systems often stem from algorithmic advances, this behavior underscores a fundamental limitation of current RSE-Agents. To probe this limitation, we perform idea-injection experiments and show that agents equipped with innovative ideas from frontier research or human experts achieve substantially larger improvements than those relying on self-generated ideas. These results suggest that idea generation is a central bottleneck of current RSE-Agents, highlights a key direction for the future recommendation system research. Our implementation is publicly available on GitHub: https://github.com/Ray-Tiann/AutoRSE.

0.32