CS Paper Radar

CoCoRerank: Towards Conventional Commit Message Generation by Component and Candidate Consistency Reranking

Shaopeng Jia, Yali Du, Ming Li

cs.SE ·submitted 2026-09-25 08:04 UTC · v1
benchmarkcode/data

Abstract

Commit messages are essential for understanding software changes, yet automatic commit message generation typically treats a message as an unstructured text sequence. This limits its ability to support standardized development workflows, where commit messages are often expected to follow the Conventional Commits Specification (CCS) in the form type (scope): subject. In this paper, we study conventional commit message generation under the complete CCS format. We construct a new benchmark of 86,688 high-quality commits collected from open-source GitHub repositories, with each message normalized into type, scope, and subject through structural normalization and semantic quality filtering. Based on this benchmark, we propose a two-dimensional consistency-based reranking framework named CoCoRerank for LLM-based generation. CoCoRerank exploits horizontal consistency among the code change, type, scope, and subject, as well as vertical consensus across multiple generated candidates. Experiments with representative CMG baselines, LLM generators, reranking strategies, and ablation variants show that CoCoRerank improves both structural component prediction and subject generation quality. The results demonstrate that complete CCS supervision and multidimensional consistency modeling provide an effective foundation for accurate and standardized commit message generation. The artifact is publicly released at https://github.com/bluewhalebug/CoCoRerank.

Comment
14 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

ComBeautify: Leveraging Code Change Summaries for Conventional Commit Generation

cs.SE cs.AI ·2026-10-25 ·Rishi Raj, Aashray Tandon, Nikolay Matyunin, Jibesh Patra
code/data similarity 0.77
Abstract

Modern software development relies on version control systems such as Git to track changes and facilitate collaboration. While commit messages are crucial for conveying developer intent, they do not affect program functionality; as a result, developers often lack motivation to write meaningful ones. The Conventional Commits Specification (CCS) addresses this with a structured format and guidelines that help developers express the intent of a change, improving clarity and enabling automation. Yet, developers frequently struggle to express intent while adhering to the format. In particular, they may misclassify commits due to the large number of possible categories or the lack of project-specific guidance. We introduce ComBeautify, an approach that automatically generates CCS-compliant commit messages. Our hybrid approach leverages context-preserving summarization of Git diffs with large language models (LLMs) to improve commit classification accuracy under CCS. Rather than relying on manually curated category definitions, we learn them from historical commits. We also propose a method for automatically generating the CCS description component. Our empirical evaluation across 13 languages, 6,500 open-source repositories, and over 20 million commits provides long-term insights into CCS adoption, revealing increasing compliance. Our evaluation on twenty popular repositories reveals the limitations of approaches guided by generic category definitions, and shows that ComBeautify, when trained on project-specific data, outperforms baselines by up to 3x. Additionally, using a dataset of 2,000 manually curated commits, we validate our summarization approach, achieving up to 95.25% accuracy and surpassing baselines. These findings emphasize the need for adaptable, context-aware AI methods for commit classification and CCS-compliant commit message generation.

0.25

LLM-Enhanced Commit Message Generation via Issue Information: An Exploratory Study

cs.SE ·2026-08-22 ·Zongen Ren, Wei Shi, Bo Xiong, Chong Wang, Peng Liang
dataset similarity 0.72
Abstract · 28 pages, 7 images, 11 tables, Manuscript submitted to a journal (2026)

Commit messages help developers understand code changes, support collaboration, and improve long-term maintenance. However, the use of issue information alone as the external context for LLM-based CMG has not been systematically studied. We propose an ISsue-Augmented framework for Commit message generation (ISAC) by combining code diffs with issue information as LLM input. To support the evaluation, we construct ApacheCM-Issue, a commit-issue aligned dataset built upon ApacheCM by linking commits with issues from GitHub and Apache Jira. Using samples from Scala, Java, and C++ projects, we evaluate four input configurations using two representative LLMs, GPT-5.5 and DeepSeek-V4-Flash in different reasoning configurations. The results show that incorporating issue information consistently improves LLM-based CMG across all evaluated model configurations and metrics, with the largest gains observed for CIDEr. Incorporating a similar historical commit further improves automatic metric scores, while replacing full issue information with a structured issue summary decreases them. ISAC also outperforms the four reproduced state-of-the-art (SOTA) CMG baselines across all five automatic metrics on the experimental dataset. The human evaluation further shows that structured issue summaries may improve perceived completeness, although replacing the original issue information can sacrifice contextual details and lead to worse results on automatic metrics.

0.26

CAMGen: A Core Change-Aware Multi-View Framework for Commit Message Generation

cs.SE ·2026-10-22 ·Bo Xiong, Weisheng Chen, Zongen Ren, Yifeng Zhu, Xutian Li +3 more
similarity 0.71
Abstract · 32 pages, 5 figures, 8 tables, Manuscript submitted to a Journal (2026)

Commit messages help developers understand code changes and support software maintenance. Recent approaches use repository context to improve commit message generation (CMG) with large language models (LLMs). However, existing CMG methods provide limited support for understanding how local edits affect functionality jointly implemented by multiple code entities. This functional understanding is essential for generating commit messages that explain the main change and its impact. To address this problem, we propose CAMGen, a core change-aware multi-view commit message generation framework that incorporates functional context into generation. CAMGen identifies core entities that represent the primary change intent. It then forms candidate functional units through repository-wide semantic clustering and uses those containing the core entities to construct a Functional Semantic View. This view supplies semantic context for interpreting the affected functionality. CAMGen also uses context provided by the Structural Dependency View and the Historical Exemplar View. The context from all three views is then provided to the LLM together with the code diff to guide CMG. The experimental results show that CAMGen outperforms representative CMG baselines across three programming languages. Human evaluation further shows that CAMGen generates more complete commit messages with clearer explanations of change rationale and impact. These results demonstrate the value of functional context and complementary repository information for CMG.

0.10

Beyond Similarity-Based Generation: Guiding Commit Message Generation with Issue-Commit Traceability

cs.SE ·2026-10-18 ·Chunying Zhou, Gong Chen, Xiaoyuan Xie, Yi Song, Yuxuan Yang +1 more
code/data similarity 0.70
Abstract · 16 pages, 5 figures

Commit messages provide records of code changes and help developers understand and maintain software. To reduce the manual effort involved in writing them, commit message generation (CMG) approaches automatically generate commit messages that describe code changes in natural language. Existing CMG approaches generally treat CMG as a standalone task and optimize generated messages primarily for similarity to developer-written references. However, this objective may not properly reflect the practical usefulness of generated commit messages. This limitation matters in current AI-assisted software engineering workflows, where commit messages are used as shared information to serve different software engineering activities. To fill this gap, we propose TraceMsg, which uses issue-commit traceability, a representative software engineering activity, to guide commit message generation. TraceMsg is a collaborative multi-agent framework that coordinates message generation, traceability reasoning, and feedback-guided refinement, allowing generated messages to serve traceability while using reasoning feedback to guide issue-conditioned refinement. We evaluate TraceMsg on six open-source projects. The experimental results show that TraceMsg achieves an average Hit@1 of 0.893 and an MRR of 0.911, improving Hit@1 and MRR by 13.16% and 38.75%, respectively, over the best baseline for each metric.

0.35

Labelling Bug-Fixing Commits with Local Open-Weight Language Models

cs.SE ·2026-09-18 ·Philip König, Georg Goldenits, Caroline König, Sebastian Raubitzek, Fabian Obermann +4 more
similarity 0.68
Abstract · 19 pages, 6 figures

Defect prediction depends on knowing which commits fix bugs, yet the labels that encode this are produced by routes that each introduce noise. Reused benchmarks carry documented data-quality problems, issue-tracker links are biased and the underlying reports are frequently mistyped, and matching keywords in commit messages is a coarse heuristic. This paper examines whether commits can be labelled as bug fixes from their content alone, using open-weight language models that run locally and therefore keep the process reproducible, inexpensive at corpus scale, usable on proprietary code, and independent of any issue tracker. Against datasets of manually validated and curated bug fixes spanning Java, Python, and JavaScript, we compare a keyword baseline with a set of open-weight models of varying size, prompting each with the commit message and the code diff. On the manually validated corpus the keyword baseline recovers fewer than half of the fixes, whereas the open-weight models recover the large majority and outperform it repository by repository with statistical significance, and larger models do not consistently outperform smaller ones. We further show that evaluation corpora without negative examples cannot support a precision-aware comparison of such classifiers. We release the labelling pipeline together with a labelled, multi-language corpus produced by the recommended configuration, as a reproducible silver-standard resource for building current, project-specific datasets.

0.22

KG-Commit: A Dynamic Knowledge Graph for Online Just-in-Time Software Defect Prediction

cs.SE ·2026-09-05 ·Mohsen Hesamolhokama, Mohammad Sina Beyrami Aghbash, Behnam Rohani, Mohammadamin Fazli, Jafar Habibi
hot direction similarity 0.63
Abstract

Just-in-time software defect prediction (JIT-SDP) aims to identify risky commits as they arrive and provide developers with timely feedback. This need for low latency has led most approaches to rely on commit-level information and overlook the broader project context in which a change occurs. Incorporating this context is challenging because it requires both efficient retrieval for incoming commits and continual maintenance as the repository evolves. We introduce KG-Commit, a dynamic knowledge graph that incrementally maintains repository history, within-file code structure, and commit semantics as the project evolves. It also uses an AST-delta mechanism to track structural changes between file edits and relies on lightweight graph inference running entirely on CPU. Our evaluation on 11 Apache software projects against six baselines shows that KG-Commit achieves the highest aggregate Macro-F1 (0.704), G-Mean (0.706), and AUC (0.809) using our selected inference pipeline. Under a realistic online protocol, it outperforms LR, HGB, RF, and DeepJIT on all 11 projects, LApredict on 10, and JITLine on 9 projects in Macro-F1, with the aggregate paired difference significant in every case. KG-Commit processes each commit in approximately 1.33~s, with a cost that remains stable as the graph grows and is compatible with commit rates observed in real-world projects. These findings show that rich project context can be efficiently maintained and exploited for online JIT-SDP.

0.27

GitCF: Reducing Incomplete Changes by Exploiting Multiple Similarities Among Commits

cs.SE ·2026-09-29 ·Yuto Kitabayashi, Takashi Kobayashi
code/data similarity 0.63
Abstract

Software developers often struggle to identify all locations that their changes affect, and incomplete changes frequently result from these omissions. To mitigate this problem, several approaches have been proposed to recommend additional locations that should be modified together with a developer's current change. A large class of these techniques mines co-change rules from repository revision histories, but because they rely solely on which elements have been modified together, they cannot recommend elements that have rarely co-changed, and they disregard the textual information that accompanies commits. A second class of techniques does exploit textual information, but it derives it from the source code or from a change request rather than from commit history, and it does not use the developer's current change as its input. To address these limitations, we propose GitCF, a change recommendation method that compares a developer's current change with past changes in the revision history and recommends additional change locations by combining two sources of similarity: the set of modified elements and the textual content of commits, including commit messages, code diffs, and linked issue descriptions. We evaluate GitCF on 543 incomplete changes from 17 open-source projects, where an incomplete change denotes a commit in which an omitted modification is supplied in a later commit, comparing it against three co-change-rule-based techniques. GitCF outperforms the strongest baseline on all five metrics (MAP, Recall@10, Recall@20, Hit@10, and Hit@20); in particular, it raises Hit@10 from 0.322 to 0.403, and the improvement in average precision is statistically significant.

0.16

Semantic Navigation for Issue Localization in Code Repository

cs.AI ·2026-09-25 ·Yunxiang Wei, Zhenyu Lei, Jundong Li
similarity 0.63
Abstract

Repository-level issue localization aims to identify and rank the files and functions relevant to resolving a reported issue. LLM agents approach this task iteratively: they identify a set of potentially relevant locations, inspect the corresponding code, and revise their judgments about these candidates as new evidence is acquired. Existing environments, however, provide limited support for this loop: agents must search for unresolved relation targets, reconstruct entity semantics from raw source code, and revise candidates without evidential basis. To address these limitations, we present SemNav, a framework that leverages deterministic retrieval to seed a broad candidate set and an LLM agent to continually refine that set, thereby combining initial coverage with evidence-guided revision. SemNav supports this process through three key components. A Semantic Navigation Graph resolves program relations on demand through a language server, enabling direct navigation to related entities across files. Issue-conditioned Semantic Cards provide compact, source-grounded interpretations of each entity's role and relevance to the issue. A persistent Candidate Workspace records each candidate together with its evidential basis, enabling grounded verification, revision, and ranking. Across SWE-bench Lite and PLocBench, SemNav outperforms existing baselines, improving File Hit@10 from 68.33\% to 82.67\% with Gemma 4B. Component ablations and trajectory analysis support the complementary roles of all three components, while Semantic Cards reduce working-context load by 48.2\% relative to full-source reading. SemNav further ranks first on all seven evidence-quality metrics on SWE-Explore and improves downstream issue resolution from 44.00\% to 52.33\%.

0.29