CS Paper Radar

Spotlights: Discovering Improvement Opportunities in Software Repositories

Udi Barzelay, Ophir Azulai, Idan Friedman, Inbar Shapira, Foad Abo Dahood, Yevgeny Burshtein, Orit Prince, Michael Soloveitchik, Oshri Naparstek, Roi Pony, Tal Drory, Michael Factor

cs.SE ·submitted 2026-09-17 14:16 UTC · v1

Abstract

Coding agents and evolutionary code-search systems can improve implementations once a target and evaluation criterion have been specified. Applying these methods to an existing software repository raises an earlier question: which implementation choices are worth investigating for a high-level engineering objective? We introduce \emph{optimization-opportunity discovery}, the repository-level task of identifying candidate source regions, explaining how they relate to the objective, and proposing possible changes. The task takes as input a repository, an engineering objective, and optional runtime evidence such as offline telemetry observations or profiles. It does not require the user to specify a defect, bottleneck, or code location. We present \emph{Spotlights}, a system that performs this task through logical repository mapping, successive agent reviews, and optional research linking candidates to relevant techniques. We evaluate Spotlights across model serving, document retrieval, blockchain ordering, and document processing. Across three cases, it recovers seven of nine expert-selected targets. In the reliability study, 70\% of the top ten candidates meet the stated correctness and severity thresholds. Across five repeated retrieval runs, 73.6\% of candidate occurrences have a matching source region in all five runs. Spotlights also rediscovers the target of a withheld retrieval optimization and connects it to a relevant tiling technique. In an implementation study, a discovered change reduces end-to-end page-processing runtime by 10.6\% while preserving measured output quality. These results establish optimization-opportunity discovery as a distinct and empirically evaluable step between a broad engineering objective and subsequent implementation and validation.

Closest papers

Nearest neighbours by meaning across the whole index.

How Should Agents See Code? Rethinking Code Representations for Agentic Software Engineering

cs.SE cs.AI ·2026-11-05 ·Melika Sepidband, Jiho Shin, Song Wang, Hadi Hemmati
similarity 0.73
Abstract

LLM agents are increasingly used for repository-level software engineering tasks. However, most agents still access repositories through raw files, lexical search, and shell commands, treating code primarily as text. This paper investigates map-based code-representation workflows as a design variable in agentic software engineering, examining how different repository views change agents' retrieval behavior, task success, token use, and latency. In this paper, we study five code-representation workflows: abstract syntax trees, call graphs, module dependencies, documentation-enriched search, and a hybrid workflow that combines them. Each workflow exposes a repository map that the agent can query during a task. We evaluate Claude Code with Haiku 4.5 against a raw-tool baseline on four repository-level tasks: fault localization with LinuxFLBench, repository-level code generation and repair with SWE-bench Pro, test generation with SWT-bench, and security patching with SecBench. The results show that representation workflows are most useful as token-saving tools when localization is the bottleneck, while their success gains narrow when the decisive step is patch or test authoring. On LinuxFLBench, all five workflows improve recall@10 over the baseline (0.757-0.829 vs. 0.717) and reduce token usage by 25-39%. On SWE-bench Pro, resolution remains close to the baseline (44.3-50.6% vs. 49.0%) while token usage drops by 17-26%. On SecBench, Callgraph has the highest patch-resolution rate in this sample (34.3% vs. 28.9%) and uses fewer tokens, but all workflows increase latency. On SWT-bench, Hybrid and Docs have the highest reproducing-test rates in this sample (47.8% and 47.2% vs. 44.7%), but all workflows use more tokens and take longer. Overall, map-based workflows change both what the agent inspects and how much it spends, but their benefits are task-dependent rather than universal.

0.33

Building a research-software catalog with a coding agent: from hackathon prototype to public deployment

cs.SE cs.AIcs.CYphysics.ed-ph ·2026-09-04 ·Kazuyoshi Yoshimi, Satoshi Terasaki, Gotai Yamada
similarity 0.73
Abstract · 23 pages, 6 figures, 2 tables

Generative AI and coding agents can accelerate research software development, but they also increase the need for efficient software discovery and maintenance. We developed a repository catalog during a three-day hackathon and subsequently examined the engineering required to make it suitable for public deployment, including adversarial review, data-quality checks, browser-level validation, and publication safeguards. We then explored whether the lessons learned from this prototype could be transferred to a much larger, human-curated portal, through a retrieval agent under development for MateriApps that combines curated portal metadata, external documentation, vector search, and local language-model generation. Implementation with coding agents was rapid, but achieving reliable operation required substantial additional engineering: the most consequential problems were not crashes but silent failures that produced plausible yet incomplete or incorrect outputs, arising from incomplete data acquisition, misleading assessments, and retrieval or preprocessing failures. These observations suggest that AI-assisted software portals require explicit validation, monitoring, and repeated review, and that curated metadata and maintained documentation remain essential. The MateriApps work is exploratory and remains under active development, so the observations reported for it are preliminary; a comparable combination of curated metadata, automatically collected documentation, and retrieval-based assistance may nevertheless be useful for extending other research-software portals.

0.21

When Relevant Code Is Wrong: Runtime-Aware Context for AI Coding Agents

cs.SE ·2026-10-28 ·Wesley K. G. Assunção, Ramesh Periyathambi
similarity 0.71
Abstract

AI coding agents increasingly rely on retrieved code and development history to guide software changes. However, highly relevant artifacts may describe implementations that are inactive or incompatible with the target deployment. Neither semantic similarity nor recency establishes whether retrieved information remains applicable under feature flags, architectural migrations, and deployment-specific configurations. We challenge relevance-oriented retrieval by introducing operational context validity, a task-, deployment-, and time-dependent criterion for determining whether software evidence can legitimately guide implementation decisions. We instantiate this new retrieval principle in CASE, an agent-agnostic framework that connects repository artifacts with runtime configurations and architectural evolution, classifies evidence as valid, invalid, or unknown, and generates context packages distinguishing actionable guidance from historical rationale and unresolved constraints. We investigate its feasibility through a reproducible preliminary evaluation involving six development tasks, 15 artifacts, and three deployments. Conventional retrieval exposes 16 incompatible artifact occurrences as actionable, whereas CASE excludes them while retaining all applicable artifacts under the prototype's reference rules. Recency filtering reduces valid recall to 0.43, while removing feature-flag evidence from CASE reduces it to 0.13. Although these results demonstrate feasibility rather than improvements in coding-agent correctness, they reveal limitations of relevance-based retrieval and motivate a new research direction: enabling coding agents to reason not only about which information is relevant, but also whether it remains operationally valid.

0.09

Semantic Navigation for Issue Localization in Code Repository

cs.AI ·2026-09-25 ·Yunxiang Wei, Zhenyu Lei, Jundong Li
similarity 0.70
Abstract

Repository-level issue localization aims to identify and rank the files and functions relevant to resolving a reported issue. LLM agents approach this task iteratively: they identify a set of potentially relevant locations, inspect the corresponding code, and revise their judgments about these candidates as new evidence is acquired. Existing environments, however, provide limited support for this loop: agents must search for unresolved relation targets, reconstruct entity semantics from raw source code, and revise candidates without evidential basis. To address these limitations, we present SemNav, a framework that leverages deterministic retrieval to seed a broad candidate set and an LLM agent to continually refine that set, thereby combining initial coverage with evidence-guided revision. SemNav supports this process through three key components. A Semantic Navigation Graph resolves program relations on demand through a language server, enabling direct navigation to related entities across files. Issue-conditioned Semantic Cards provide compact, source-grounded interpretations of each entity's role and relevance to the issue. A persistent Candidate Workspace records each candidate together with its evidential basis, enabling grounded verification, revision, and ranking. Across SWE-bench Lite and PLocBench, SemNav outperforms existing baselines, improving File Hit@10 from 68.33\% to 82.67\% with Gemma 4B. Component ablations and trajectory analysis support the complementary roles of all three components, while Semantic Cards reduce working-context load by 48.2\% relative to full-source reading. SemNav further ranks first on all seven evidence-quality metrics on SWE-Explore and improves downstream issue resolution from 44.00\% to 52.33\%.

0.29

AI-Research Agents in the Wild. From GitHub and arXiv to Regularities and Gaps

cs.DL ·2026-09-01 ·Aleksey Komissarov, Andrey Ustyuzhanin
similarity 0.68
Abstract · 56 pages, 1 figure

AI-research agents, or autoresearch systems, combine language models with tools, search, evaluation, and iterative modification of research artifacts. Their public software ecology is hard to compare because repositories, papers, benchmarks, libraries, and companion artifacts are often counted as one population. We connect two agent-drafted, human-adjudicated registries frozen on 10 June 2026: 139 canonical public repositories and 101 papers on AI-assisted research systems, each record carrying an evidence card anchored to its primary sources. Nine promoted design lineages contain 59 memberships among 52 repositories, seven of them assigned to two lineages. Against this evidence we examine six time-stamped candidate design regularities. A prospective audit of 25 newly ingested repositories observed zero of six specified trigger events, supplying bounded support for R1-R4; broader corpus evidence narrowed the original scope of R5, and a failed directional prediction leaves R6 exploratory. The paper-repository graph contains 212 curator-assigned relatedness edges: 64 of 101 papers carry at least one edge, the edges reach 23 of 139 repositories, and the six most-linked repositories carry 120 of the 212 edges (56.6%). A full-text check of seven papers shows the relation measures our curation, not the papers' citation behavior. An identity audit adjudicated against commit histories and public contributor records confirms at least 18 paper authors who also author a canonical registry repository (20 of 63 candidate author slots in 13 of 101 papers); a byline-matched tier reaches 58 identities but is reported separately and more weakly. The study contributes a source-grounded theory under prospective test, a curator-assigned map of paper-repository relatedness, and explicit limits on claims about public visibility and community structure.

0.18

SWE-Time: Benchmarking Temporal Repository Understanding

cs.SE ·2026-11-09 ·Shijun Xiao, Xinrong Chen, Hao Liu, Wensheng Zhang, Yongqiang Tang
benchmark similarity 0.68
Abstract

Repository-level code-understanding benchmarks have progressively expanded the spatial scope of reasoning, from local code to entire repositories, while largely treating each repository as a static snapshot. However, how models reason about repository evolution over time remains largely unexplored. We introduce SWE-Time, a benchmark for temporal repository understanding that makes repository evolution itself the reasoning target. SWE-Time evaluates two complementary temporal directions: retrospective understanding of repository evolution through historical regression analysis and prospective understanding of intended changes from pre-change repository states. To enable realistic yet reliable evaluation in these settings, we introduce the RealWorld-to-Verifiable Construction Framework, which transforms real-world developer queries and historical evidence into verifiable evaluation tasks. We evaluate 15 coding agents on 1,661 cases from 65 GitHub repositories. Within Claude Code, Claude Opus 4.8 achieves the highest SWE-Time Avg. of 61.9, leaving substantial headroom. Controlled analyses highlight the importance of historical code evidence and developer intent, while consistent reasoning across repository states remains challenging. These results highlight repository evolution as a distinct and challenging dimension of code understanding.

0.20

Shortcutting the Fix: Identifying and Categorizing Agentic Exploits in Software Engineering Benchmarks

cs.SE ·2026-09-06 ·Nikolai Ludwig, Wasi Uddin Ahmad, Somshubra Majumdar, Boris Ginsburg
benchmark similarity 0.68
Abstract · Preprint

High software engineering benchmark scores often mask exploitative shortcuts, such as retrieving upstream solutions, exploiting local Git histories, or recalling memorized code. We formalize this phenomenon as agentic shortcutting and audit three state-of-the-art open LLMs on SWE-bench Multilingual and DeepSWE using trajectory-level LLM judges. Standard evaluations trigger shortcutting attempts in 60.7%-90.1% and 55.5%-89.1% of trajectories, respectively, with up to 72.1% of tasks resolved with the aid of a successful exploit; disabling network access fails to curb these high attempt rates. Prompting agents to construct solutions independently suppresses attempts to 9.4%-14.2% and 0.9%-10.3%, respectively, while virtually eliminating shortcut-dependent resolutions and preserving core task performance. Our findings demonstrate the necessity of exploit-aware evaluation frameworks to measure repository-level problem solving.

0.42

RepoNav: From Snippet Retrieval to File-Centered Repository Navigation for Code Agents

cs.SE cs.CL ·2026-09-08 ·Hongzheng Chai, Jiakun Li, Hongyue Yu, Yuan Yuan
top venuebenchmark similarity 0.68
Abstract · Accepted to EMNLP 2026

Solving repository-level code tasks requires LLM-based agents to use code search tools to navigate large codebases and identify a small set of relevant files and functions. However, current retrieval tools typically return flat lists of isolated code snippets: such lists can surface relevant files, but provide insufficient structure for agents to distinguish the target function from semantically similar alternatives in the same file. We introduce RepoNav, a lightweight post-retrieval interface that reorganizes retrieved snippets into a file-centered navigation scaffold. By presenting compact structural cues and candidate targets, this scaffold guides on-demand file-structure browsing, helping agents compare sibling symbols before selecting a target function. Across diverse models on LocBench, RepoNav improves function-level localization and narrows the file-to-function gap. Controlled ablations demonstrate that these gains come from structured evidence organization rather than simply exposing additional file structure, and the approach also improves performance on a repository-level question-answering benchmark.

0.59