CS Paper Radar

Detecting Argument-Swap Bugs Using Context-Enhanced Code Representations

Subrata Das, Ali Aman, Muhammad Asaduzzaman, Kawser Wazed Nafi, Salimur Choudhury

cs.SE ·submitted 2026-09-15 21:03 UTC · v1
publishedbenchmarkdataset

Abstract

Names of source code elements convey rich semantic information and have been widely used in software engineering tasks such as bug detection, code completion, type prediction, and code classification. Prior studies exploit lexical similarity between method arguments and formal parameter names to detect bugs caused by incorrectly ordered arguments, typically relying on establishing mappings between method calls and their corresponding definitions. However, such mappings are often difficult to obtain in dynamically typed languages like Python. In this paper, we present BugProbe, a learning-based approach for detecting incorrectly ordered arguments in Python method calls that does not require call-to-definition mappings. Our approach leverages multiple sources of contextual information, including local context and argument usage context, and combines name-based similarity with machine learning to construct expressive representations of method arguments. We collect a new dataset of 132,739 Python source files from the top-1,000 starred GitHub repositories, yielding 3,371,244 synthetic training examples, and contribute a curated benchmark of 55 real-world argument-swap bugs manually verified from commit histories. We evaluate our approach on this dataset and show that it achieves high accuracy and consistently outperforms a state-of-the-art baseline across standard evaluation metrics. These results demonstrate that effective detection of argument-ordering bugs is possible without relying on explicit call-to-definition resolution, making the approach well suited for dynamically typed language settings.

Comment
Accepted in the 26th IEEE International Conference on Source Code Analysis and Manipulation (SCAM 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

A Comprehensive Study of Native Code Bugs in Python Applications

cs.SE cs.CR ·2026-08-30 ·Haoran Yang, Haipeng Cai
similarity 0.70
Abstract

The impact of Python applications has been evidenced by their widespread presence in some of the most impactful software domains, such as machine learning frameworks and scientific computing platforms. These applications often integrate native code components written in a lower-level programming language like C. This multilingual construction brings various benefits such as greater performance efficiency and easier interoperability with diverse runtime environments. However, bugs in the native code (i.e., native code bugs), which are usually stealthy, also constitute a major additional challenge to the quality of the Python applications as a whole. Yet despite existing relevant studies, there remains a lack of comprehensive understanding of native code bugs in Python applications. In this paper, we aim to mitigate this knowledge gap through the first in-depth study of such bugs, dissecting their common symptoms, introducing locations, manifestation characteristics, root causes, and fixes. Based on our extensive automated and manual analyses of 216 native code bugs in real-world Python projects on GitHub, we obtained novel findings about and new insights into the occurrence mechanisms and resolution strategies of those bugs.

0.09

From Codebase to Culprit (C2C): Reducing the Search Space for Bugs with Semantic Retrieval and Hierarchical Reinforcement Learning

cs.SE cs.LG ·2026-09-29 ·Ankur Garg, Corey Yang-Smith, Rishav Rishav, Ahmad Abdellatif, Samira Ebrahimi Kahou
similarity 0.68
Abstract

We introduce C2C (From Codebase to Culprit), a framework for precise bug localization that progressively reduces the debugging search space across multiple levels of granularity: files, functions, and lines of code. To mirror developer's natural top-down debugging workflows, C2C integrates semantic retrieval and Hierarchical Reinforcement Learning (HRL) in a two-stage process. First, it performs recall-oriented retrieval of buggy candidates via semantic vector similarity search using bug-report text, including available stack-trace information, against a database of embeddings, where the embeddings are fine-tuned via contrastive learning with CodeBERT. Building on this reduced search space, the HRL framework incrementally localizes bugs, reasoning from files to functions and ultimately to individual lines of code. Unlike prior approaches which operate at a single granularity, C2C enables multi-resolution localization while maintaining contextual consistency across decisions. Experiments on real-world Java and Python datasets demonstrate that C2C improves retrieval precision and localization accuracy. Ablation studies further highlight the contributions of hierarchical decomposition, structured learning signals, and reward shaping in advancing multi-level bug localization.

0.15

RepGen: Reproducing Deep Learning Bugs through Context-Aware Code Generation and Verification

cs.SE ·2026-10-30 ·Mehil B Shah, Mohammad Masudur Rahman, Foutse Khomh
code/data similarity 0.67
Abstract · Under Review - ICSE 2027 Tool Demonstration and Data Showcase Track

Reproducing deep learning (DL) bugs is challenging because failures can involve source code, models, data, dependencies, and execution environments. Incomplete bug reports further complicate reproduction by omitting essential details. We present RepGen, an open-source tool for reproducing DL bugs through context-aware code generation and iterative verification. Given a bug report and repository, RepGen explores the codebase to build a learning-enhanced context, develop a reproduction plan, and generate candidate scripts. It refines candidates using static checks and LLM feedback to improve their correctness and relevance. When enabled, the tool prepares the execution environment, runs the script, and assesses its output against the reported failure. RepGen provides an interactive command-line interface supporting local and hosted LLMs, configurable execution budgets, and inspectable artifacts, along with a GitHub Action for automated issue processing. The underlying approach reproduced 85 of 106 real-world DL bugs (80.19%). A controlled study with 27 developers showed a 23.35% improvement in reproduction success and a 56.8% reduction in reproduction time.

0.33

Labelling Bug-Fixing Commits with Local Open-Weight Language Models

cs.SE ·2026-09-18 ·Philip König, Georg Goldenits, Caroline König, Sebastian Raubitzek, Fabian Obermann +4 more
similarity 0.67
Abstract · 19 pages, 6 figures

Defect prediction depends on knowing which commits fix bugs, yet the labels that encode this are produced by routes that each introduce noise. Reused benchmarks carry documented data-quality problems, issue-tracker links are biased and the underlying reports are frequently mistyped, and matching keywords in commit messages is a coarse heuristic. This paper examines whether commits can be labelled as bug fixes from their content alone, using open-weight language models that run locally and therefore keep the process reproducible, inexpensive at corpus scale, usable on proprietary code, and independent of any issue tracker. Against datasets of manually validated and curated bug fixes spanning Java, Python, and JavaScript, we compare a keyword baseline with a set of open-weight models of varying size, prompting each with the commit message and the code diff. On the manually validated corpus the keyword baseline recovers fewer than half of the fixes, whereas the open-weight models recover the large majority and outperform it repository by repository with statistical significance, and larger models do not consistently outperform smaller ones. We further show that evaluation corpora without negative examples cannot support a precision-aware comparison of such classifiers. We release the labelling pipeline together with a labelled, multi-language corpus produced by the recommended configuration, as a reproducible silver-standard resource for building current, project-specific datasets.

0.22

SWE-at-Scale: Scalable Issue Synthesis from Historical Bug Fixes

cs.SE ·2026-10-15 ·Silin Chen, Zihan Zhang, Xiaodong Gu, Jiale Huang, Yuling Shi +5 more
code/datacommunity pick similarity 0.65
Abstract · Our code and models are available at https://github.com/cslsolow/SWE-at-scale and https://huggingface.co/SWE-at-Scale

As coding agents become increasingly capable, robust evaluation requires a broader and more diverse supply of verifiable software engineering (SWE) tasks. However, real-world tasks are constrained by the natural frequency of issue reporting and the substantial cost of manual curation. Furthermore, existing synthetic approaches rely primarily on the pretrained knowledge of large models coupled with heuristic prompting, often failing to capture the authentic, complex mechanics of software failures. To exponentially scale SWE task production, we introduce SWE-at-Scale, a framework that automatically synthesizes realistic issues by reverse-engineering real-world software fixes. Our key insight is that historical fixes encapsulate deep mechanistic knowledge of bug formation: inverting a fix transforms repair supervision into bug-induction supervision grounded in actual development scenarios. SWE-at-Scale transfers these learned bug patterns to novel code contexts, repeatedly sampling diverse bug typologies to construct executable SWE tasks. Extensive experiments show that SWE-at-Scale generates more diverse and realistic SWE tasks than current synthesis baselines while posing greater challenges to state-of-the-art coding agents. Notably, SWE-at-Scale can generate distinct bug types from a single code context, with bug diversity scaling robustly even up to 200 generations per file. Overall, SWE-at-Scale turns finite real-world fixes into reusable supervision for scalable synthesis of realistic, diverse, and challenging SWE tasks.

0.59

Standards-Guided and Dependency-Aware Code-Level Documentation Generation

cs.SE ·2026-10-20 ·Tingxu Han, Wei Song, Ziming Li, Yuchen Chen, Weisong Sun +4 more
positioncode/data similarity 0.65
Abstract

Automatically generating high-quality documentation for software repositories remains challenging: existing approaches largely treat documentation as unconstrained text generation, lacking explicit quality specifications, depending heavily on long-context prompts over raw code, and offering limited support for reproducible evaluation. We argue that code-level documentation generation should instead be a standards-guided software engineering task, in which documentation is generated and assessed against explicit, reusable quality requirements derived from authoritative standards. We define code documentation as natural-language descriptions of repository functions and classes, organized by source-code path. We present CODEDOC, a standards-guided and dependency-aware framework that identifies dependencies between code entities via static call-graph and AST analysis and constructs dependency-aware structured prompts combining local code context, repository paths, and documentation criteria drawn from authoritative standards and style guides. We further introduce an interactive LLM-as-a-judge protocol that supplies criterion-aligned feedback to refine generated documentation along fine-grained dimensions from formal and practice-oriented standards. Experiments on ten open-source Python repositories show that CODEDOC produces higher-quality documentation than RepoAgent and a long-context baseline under standards-aligned evaluation, with relative improvements of 11.6%-17.0% and 21.4%-29.6%, respectively, while consuming roughly an order of magnitude fewer tokens per documentation entity than the long-context baseline. A human study confirms substantial alignment between LLM-based and human judgments (Spearman's rho = 0.72), and the relative findings hold across different judge LLMs. Code is available at https://github.com/ControllableModel/CodeDoc.

0.35

Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks

cs.SE ·2026-09-09 ·Dongdong Zhao, Jian Chen, Guancheng Lin, Jianwen Xiang, Jacky Wai Keung +1 more
benchmark similarity 0.65
Abstract

Code generation benchmarks are widely used to evaluate Large Language Models (LLMs), but benchmark data leakage into training sets can inflate performance and undermine evaluation validity. DetectLeak, a method specifically designed for code generation benchmark leakage detection, relies on perplexity scores to identify likely leaked samples. However, perplexity mainly reflects general familiarity with code patterns and may perform poorly on complex or rare samples. It also overlooks other useful signals, such as code similarity, functional correctness, and semantic representations. To address these limitations, we propose CGMIA (Code-Generation-specific Membership Inference Attack), a method for detecting leakage in code generation benchmarks. CGMIA fine-tunes a shadow model on a subset of benchmark samples to construct labeled member and non-member data. For each sample, it collects the input prompt, generated code, and reference solution, and extracts expert features, including CodeBLEU, edit distance, test pass rate, and perplexity, together with semantic features from CodeBERT embeddings. An integrated learning module combines these features to capture both surface-level memorization signals and deeper behavioral patterns, enabling a classifier to predict whether a sample was included in the target model's training set. Experiments on eight code generation benchmarks show that CGMIA outperforms eight existing membership inference methods in most cases. It also effectively detects known leaked APPS samples in StarCoder-7B's training data.

0.19

PyReprism: A Shared Lexical Foundation for Polyglot Code Analysis

cs.SE ·2026-10-29 ·Daniel Ogenrwot, Jorge Gonzalo Delgado Cervantes, Richard Sserunjogi, John Businge
code/data similarity 0.65
Abstract · 5 pages, 2 figures, Submitted to SANER 2027 Tool Demo Track

Empirical studies of polyglot repositories apply the same analyses to every language of a corpus, such as separating code from comments, computing metrics, normalizing identifiers, and screening for duplicated code. Assembling them from specialized tools introduces differences in language coverage, measurement conventions, and workflows. We present PYREPRISM, an open-source Python library and command-line tool that separates language-specific lexical processing from reusable analyses: each language declares its lexical surface, which yields a lossless stream of typed tokens, and every analysis is implemented once on that stream. Across 187 built-in languages, language-specific declarations have a median length of 16 lines, and no language overrides a shared analysis. Against a reference parser, comment and string classification reach F1 scores of 0.994 and 0.958 on 1,412 files in 12 languages that were never used to develop the tokenizer. In a study of a polyglot repository, one tool replaces three; with matched inputs and analysis operations, it takes 2.9s instead of 8.7s on one core. We also characterize where lexical analysis falls short, in complexity estimation and clone screening. Tool: https://github.com/unlv-evol/PyReprism Video: https://youtu.be/IXc-NP5t4GU

0.38