CS Paper Radar

STORY2DIFF: A Dataset Linking Real-World User Stories to Code Changes

Benedetta Donato, Daniele Franchini, Alessandro Greco, Oliviero Riganelli, Daniela Micucci, Leonardo Mariani

cs.SE ·submitted 2026-10-29 11:46 UTC · v2
datasetcode/data

Abstract

Research on requirements-to-code tasks needs examples in which a genuine requirement is paired with the change that implemented it. Existing user-story collections, however, largely contain isolated or synthetic text. To address this limitation, we present STORY2DIFF, a dataset of 7,785 English user stories linked to merged pull requests from January 2017 to July 2026. The dataset is mined from a set of 982 public Java and Python repositories. Each record in the dataset combines the original issue, extracted Connextra elements, the associated pull request and commit metadata, a file-level diff capturing the code changes introduced by the associated pull request, and the affected files before and after the change. The tool we built to mine STORY2DIFF can be used to repeat the automatic extraction of the records, using different parameters (e.g., a different timeframe). The dataset can support a variety of tasks, including requirement discovery, traceability, change- impact estimation, and requirements-aware code generation. Dataset: https://github.com/riganelli/user-story-mining-pipeline/ releases/tag/V4.0.1. Video: https://youtu.be/qOhXUeyeiTQ.

Comment
Submitted to the ICSE 2027 Demonstrations and Data Showcase track

Closest papers

Nearest neighbours by meaning across the whole index.

The Construction of an Empirical Dataset of Incomplete Software Changes from Open Source Projects

cs.SE ·2026-09-28 ·Savira Ramadhanty, Profir-Petru Pârţachi, Yoshiya Ishida, Takashi Kobayashi
datasetcode/data similarity 0.66
Abstract

During software development, a modification to a software component may propagate across the system, requiring precise identification and correct revision of all affected components. This is a complex task, and developers often (45.7%) miss related changes. To address this, several methods have been developed to extract co-change rules from files that are frequently changed together in the revision history. However, previous research evaluated the methods using artificially created incomplete changes, which may not be representative of real-world data. To solve this problem, we construct a dataset by mining incomplete changes from a collection of open-source software, using information about induced bugs and their respective fixes from an issue tracking platform. We also analyze the characteristics of incomplete changes using this constructed dataset and found that 89.4% of missed changes involved five or fewer files. Finally, we re-evaluate LCExtractor, an existing co-change rule extraction method, on our constructed dataset, and we identify the optimal sorting criterion and the impact of the number of used commits.

0.19

GitCF: Reducing Incomplete Changes by Exploiting Multiple Similarities Among Commits

cs.SE ·2026-09-29 ·Yuto Kitabayashi, Takashi Kobayashi
code/data similarity 0.66
Abstract

Software developers often struggle to identify all locations that their changes affect, and incomplete changes frequently result from these omissions. To mitigate this problem, several approaches have been proposed to recommend additional locations that should be modified together with a developer's current change. A large class of these techniques mines co-change rules from repository revision histories, but because they rely solely on which elements have been modified together, they cannot recommend elements that have rarely co-changed, and they disregard the textual information that accompanies commits. A second class of techniques does exploit textual information, but it derives it from the source code or from a change request rather than from commit history, and it does not use the developer's current change as its input. To address these limitations, we propose GitCF, a change recommendation method that compares a developer's current change with past changes in the revision history and recommends additional change locations by combining two sources of similarity: the set of modified elements and the textual content of commits, including commit messages, code diffs, and linked issue descriptions. We evaluate GitCF on 543 incomplete changes from 17 open-source projects, where an incomplete change denotes a commit in which an omitted modification is supplied in a later commit, comparing it against three co-change-rule-based techniques. GitCF outperforms the strongest baseline on all five metrics (MAP, Recall@10, Recall@20, Hit@10, and Hit@20); in particular, it raises Hit@10 from 0.322 to 0.403, and the improvement in average precision is statistically significant.

0.16

ComBeautify: Leveraging Code Change Summaries for Conventional Commit Generation

cs.SE cs.AI ·2026-10-25 ·Rishi Raj, Aashray Tandon, Nikolay Matyunin, Jibesh Patra
code/data similarity 0.63
Abstract

Modern software development relies on version control systems such as Git to track changes and facilitate collaboration. While commit messages are crucial for conveying developer intent, they do not affect program functionality; as a result, developers often lack motivation to write meaningful ones. The Conventional Commits Specification (CCS) addresses this with a structured format and guidelines that help developers express the intent of a change, improving clarity and enabling automation. Yet, developers frequently struggle to express intent while adhering to the format. In particular, they may misclassify commits due to the large number of possible categories or the lack of project-specific guidance. We introduce ComBeautify, an approach that automatically generates CCS-compliant commit messages. Our hybrid approach leverages context-preserving summarization of Git diffs with large language models (LLMs) to improve commit classification accuracy under CCS. Rather than relying on manually curated category definitions, we learn them from historical commits. We also propose a method for automatically generating the CCS description component. Our empirical evaluation across 13 languages, 6,500 open-source repositories, and over 20 million commits provides long-term insights into CCS adoption, revealing increasing compliance. Our evaluation on twenty popular repositories reveals the limitations of approaches guided by generic category definitions, and shows that ComBeautify, when trained on project-specific data, outperforms baselines by up to 3x. Additionally, using a dataset of 2,000 manually curated commits, we validate our summarization approach, achieving up to 95.25% accuracy and surpassing baselines. These findings emphasize the need for adaptable, context-aware AI methods for commit classification and CCS-compliant commit message generation.

0.25

Hybrid Source Code Change Impact Analysis

cs.SE cs.AI ·2026-11-07 ·Katharina Stengg, Christian Macho, Martin Pinzger
code/data similarity 0.63
Abstract

Change impact analysis (CIA) helps developers identify code entities impacted by their initial changes. While several approaches for CIA were proposed, they still struggle with either low precision or low recall, thereby increasing manual effort for developers to verify the results. Furthermore, limitations in the used datasets and experimental setups can bias the results reported by many CIA approaches. In this paper, we first address the dataset limitations by manually labeling 1,931 changed code entities stemming from 263 commits of 22 open-source Java projects. Next, we introduce two variants of katIA, our code change impact analysis approach that combines static impact analysis with the reasoning capabilities of large language models (LLMs). The first variant uses a single prompt (katIA-SP) for refining an initial static impact set while the second variant uses three specialized agents (katIA-MA). The evaluation with our curated dataset shows that katIA-SP with Gemini 2.5 Flash achieved an F1-score of 22.4% when matching impacted entities by their fully qualified names (FQN) and 27.0% when matching them by their class and entity names (CE). The latter shows an improvement of 2.0 percentage points over the state-of-the-art. This score is further improved by katIA-MA which, using Gemini 2.5 Flash, yields an FQN-matching F1-score of 29.1% and a CE-matching F1-score of 33.3%. While these improvements come at the cost of increased median generation time from 0.570 s to 11.214 s and median token usage from 1,235 to 6,462 tokens, the improved performance justifies the additional cost. The comparison of three LLMs using katIA-SP finds that Gemini 2.5 Flash is the most consistent model. These results indicate that combining static impact analysis with LLM-based refinement, particularly with the multi-agent variant, can improve code change impact analysis.

0.26

PR4Code: A Pull Requests Dataset for AI Code Generation

cs.SE ·2026-10-29 ·Benedetta Donato, Leonardo Mariani, Daniela Micucci, Oliviero Riganelli
publisheddataset similarity 0.63
Abstract · Published in IEEE Access

AI-driven code generation is evolving from handling small, localized code-completion tasks toward supporting the implementation of full software features. To support this transition, this paper introduces PR4Code, a dataset of 4,508 Java and 8,831 Python curated Pull Requests (PRs) collected from GitHub, each enriched with metadata, commit histories, and detailed code changes. Unlike resources focused on isolated code fragments, PR4Code captures the feature implementation process, delivering a large and diverse collection of real-world development tasks that can be used to assess AI-driven solutions. Our analysis reveals substantial variability in PR structure, commit granularity, and textual content. In addition, we release scripts to regenerate and update the dataset, ensuring reproducibility and maintainability.

0.63

Semantic Navigation for Issue Localization in Code Repository

cs.AI ·2026-09-25 ·Yunxiang Wei, Zhenyu Lei, Jundong Li
similarity 0.62
Abstract

Repository-level issue localization aims to identify and rank the files and functions relevant to resolving a reported issue. LLM agents approach this task iteratively: they identify a set of potentially relevant locations, inspect the corresponding code, and revise their judgments about these candidates as new evidence is acquired. Existing environments, however, provide limited support for this loop: agents must search for unresolved relation targets, reconstruct entity semantics from raw source code, and revise candidates without evidential basis. To address these limitations, we present SemNav, a framework that leverages deterministic retrieval to seed a broad candidate set and an LLM agent to continually refine that set, thereby combining initial coverage with evidence-guided revision. SemNav supports this process through three key components. A Semantic Navigation Graph resolves program relations on demand through a language server, enabling direct navigation to related entities across files. Issue-conditioned Semantic Cards provide compact, source-grounded interpretations of each entity's role and relevance to the issue. A persistent Candidate Workspace records each candidate together with its evidential basis, enabling grounded verification, revision, and ranking. Across SWE-bench Lite and PLocBench, SemNav outperforms existing baselines, improving File Hit@10 from 68.33\% to 82.67\% with Gemma 4B. Component ablations and trajectory analysis support the complementary roles of all three components, while Semantic Cards reduce working-context load by 48.2\% relative to full-source reading. SemNav further ranks first on all seven evidence-quality metrics on SWE-Explore and improves downstream issue resolution from 44.00\% to 52.33\%.

0.30

The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation

cs.CL ·2026-09-13 ·Advait Deshmukh, Nora Benedict, Melanie Walsh, Maria Antoniak
published similarity 0.61
Abstract · Accepted at COLM 2026

Large language models (LLMs) have changed the way people engage with stories. Drawing on public chatbot logs, we can see that when users generate stories, they iteratively edit their prompts to explore narrative possibilities, adjusting characters, redirecting plots, and swapping fictional universes. As aggregated data, these prompts represent rich traces of creative preference at scale. Yet story generation evaluation benchmarks rely on static, one-shot prompts that cannot capture this exploratory behavior. In this work, we study how users revise consecutive story prompts in the wild. Using a dataset of naturally occurring user-chatbot conversations, we construct WildStories, a sample of 275,635 story generation prompts (labeled with story format, prompt components, and explicitness), and WildEdits, a collection of 24,291 edit trees that model how users iteratively edit base story prompts and explore branching story possibilities. From these trees we develop a framework of edit types crossing four directions (adding, removing, changing, and extending) with fourteen targets (e.g., plot, character, genre). We then use our datasets and this framework to analyze user behavior in navigating narrative space via LLMs. Finally, we show how automated permutations based on the framework can be used for story generation benchmarking. Content Warning: This paper works with "wild" chatbot logs, which often include toxic and sexually explicit themes.

0.32

Git2KG: Mining socio-technical relationships in GitHub repositories as temporal knowledge graphs

cs.SE ·2026-11-11 ·Lennart Hessenauer, Lisi Qarkaxhija, Ingo Scholtes
code/data similarity 0.60
Abstract

The widespread use of Git and GitHub in open source software (OSS) projects has produced large volumes of data on collaborative software development. Existing approaches to mine those data often use graph models to (i) capture communication between developers in the issue tracker, or (ii) represent developer-developer co-authorship or developer-file code ownership networks from the editing history of code artifacts. Integrating these data in a single knowledge graph (KG) is complicated by the need to disambiguate authors and match entities from multiple sources. Moreover, most works using network analysis have taken a time-aggregated perspective that neglects temporal dynamics of collaborations. To address these issues, we present Git2KG, an extensible Python library that mines temporal knowledge graphs (TKGs) from GitHub repositories. Git2KG extracts fine-grained co-editing relations and code ownership from the Git log. It further disambiguates developer identities to align the resulting graphs with management and communication networks extracted from GitHub issue trackers. Git2KG supports temporal analysis by slicing development events into fixed-size, non-overlapping time windows, yielding a series of knowledge graph snapshots per repository. It further supports computing node features and labels for downstream machine learning tasks. Snapshots can then be exported to PyTorch Geometric, PathPyG, and NetworkX for temporal network analysis, visualization, and graph learning.

0.53