CS Paper Radar

What Ethereum Developers Ask For, and Whether They Get It: Ten Years of Ethereum Stack Exchange

Tamer Abdelaziz

cs.SE ·submitted 2026-10-22 06:59 UTC · v1
published

Abstract

The Ethereum community learns where developers struggle through surveys and interviews, which record what developers say when asked about a chosen theme. Ethereum Stack Exchange (ESE) records what they ask for unprompted, and whether anyone resolves it. We study every question posted to ESE from Jan 2016 to Dec 2025, 54,637 in all, and label each with a prompt-based topic pipeline seeded with eight Ethereum-specific topics and open to new ones (22 topics in all). A three-provider committee of language models screens the labels, and manual adjudication of every disputed case puts their accuracy at 87.9 to 93.5%. Because the corpus is the complete record, no rate carries sampling error. We ask what developers ask for help with, whether the venue still resolves it, and, where it does not, whether the questions changed or their resolution did. Demand is concentrated on writing, deploying, and debugging contracts; the toolchain is the second-largest topic and growing, while security, consensus, and scaling are thin. Supply has weakened, and unevenly: answers still arrive, closure falls further. Across five estimators of the decade, the share of questions drawing any answer fell by 9.5 to 18.6 percentage points, but among answered questions the share reaching an accepted answer fell by 16.2 to 24.5. Rebuilding the later acceptance rate at the earlier topic mix moves the fall only from 9.40 to 9.32 points, so the topic mix does not explain it. Who asks separates outcomes far more sharply than what is asked: acceptance runs from 19.8% for askers below 10 reputation to 66.8% at 10k and above, a 47- point spread with a gradient in every year, against 17 across topics, and a shift toward newer askers accounts for up to 59% of the acceptance fall. The largest of seven properties an asker controls accompanies 5 points. The record places the support gap in the toolchain and among newcomers.

Comment
12 pages, Mining Software Repositories
DOI
10.5281/zenodo.23551228

Closest papers

Nearest neighbours by meaning across the whole index.

The Uneven Decline of Collective Knowledge Production: Evidence from Stack Overflow After Generative AI

cs.SE cs.AIcs.DL ·2026-09-28 ·Myokyung Han, Taegyoon Kim, Jinhyuk Yun, Lanu Kim
similarity 0.58
Abstract

Generative AI (Gen AI) is reshaping how individuals learn and work, but its consequences for collective knowledge, the shared body of knowledge that online communities produce together, remain poorly understood. Prior work has documented an aggregate decline in participation on knowledge-sharing platforms, but it remains unclear which specific kinds of knowledge are being lost first. We study this question using Stack Overflow, one of the largest online communities for software engineering, treating the release of ChatGPT-3.5 as a natural shock. Analyzing over two million questions posted between 2020 and 2025, we track how two dimensions of collective knowledge, difficulty and data availability, change following Gen AI's release. Using diverse methods and robust checks, we find consistent patterns. Easy questions decline sharply while difficult questions become more common, a pattern corroborated by rising code complexity. Data-rich topics and tags lose share of questions, while data-scarce ones gain ground. The two dimensions also interact: the decline in easy questions is concentrated specifically within data-rich domains, while difficult questions increase regardless of data availability. This pattern extends beyond Python across programming languages, with more prevalent languages showing sharper shifts. Together, our findings reveal that Gen AI's impact on collective knowledge is uneven, eroding easy, accessible knowledge first while more complex, less common knowledge persists.

0.21

The Infrastructure Divide: Analyzing Community Stance on Data Center Development via Reddit

cs.AI ·2026-11-06 ·Yoko Koyama, Gongfan Chen, Mansi Saxena
code/data similarity 0.57
Abstract · 42 pages, 7 figures

The rapid expansion of AI-driven data center construction has outpaced public engagement: an estimated $64 billion in U.S. data center projects were blocked or delayed by local opposition between May 2024 and March 2025, yet the concerns driving this resistance remain poorly characterized. Prior engagement methods such as surveys and hearings are introduced late in the planning process, and no study has systematically mined social media to measure community stance at scale. To address this gap, this study presents an LLM-powered opinion-mining framework and applies it to 798 Reddit posts and 102,328 associated comments spanning a decade. An open-source large language model, guided by a Chain-of-Stance prompting framework and three domain-specific labeling rules, classified stance at 83% of the human inter-reviewer agreement ceiling. Opposition dominated discourse volume rather than disapproval: against posts outnumbered favor posts 2.6:1 and generated 8.5 times more comments, and this gap widened significantly over 86 weeks. Topic modeling of opposition arguments resolved into four reproducible clusters, resource consumption and scarcity (42.8%), contested siting processes and secrecy (34.2%), political opposition to data centers (13%), and political and corporate accountability (8%). Engagement analysis revealed that posts with emotional or provocative political framing attracted the most engagement, while key technical concerns (e.g., water, power) were dispersed across lower-engagement comment threads. These findings give local leaders, developers, and policy makers a quantified, continuously updatable basis for prioritizing concerns and policy levers, supporting a shift from opposition to negotiated, verifiable partnership for data center development.

0.32

IssueLighter: An Issue Type-Aware LLM Agent for Issue Report Validity Assessment

cs.SE cs.AI ·2026-11-02 ·Jiawei Ran, Yuxia Zhang, Ziyang Zhao, Hui Liu
top venuecode/data similarity 0.57
Abstract · Accepted to ICSE 2027. 12 pages, 2 figures

Invalid issue reports are prevalent in open source software (OSS) projects, imposing substantial burdens on maintainers who already face a high risk of burnout. Prior research has primarily focused on identifying invalid reports for specific issue categories (e.g., bug and wontfix), leaving the broader landscape of OSS issue types largely unexplored and limiting the generalizability of existing approaches. Moreover, existing methods provide little support for explaining or validating their predictions, reducing their practical usefulness for maintainers. With the rise of LLM-based agents, it remains unclear whether a comprehensive understanding of the causes of invalidity across all issue types can improve validity assessment performance while producing well-grounded rationales. To this end, we first manually annotated the validity status of 520 issue reports from three OSS projects, i.e., pandas, scikit-learn, and mypy, resulting in 257 invalid reports and 263 valid ones. Then, we conducted a thematic analysis on these invalid reports and identified 13 causes of invalidity that are closely specific to issue types. Based on these insights, we propose IssueLighter, an agent-based framework with a type-aware validity assessment strategy that dynamically executes commands. Experimental results show that IssueLighter consistently and substantially outperforms all baseline methods, with an average F1 improvement of over 20 percentage points. Ablation studies confirm that both the type-aware strategy and the taxonomy of causes of invalidity contribute to the performance gains. Qualitative analysis further demonstrates that IssueLighter generates rationales grounded in accurately identified causes of invalidity and supported by evidence. Moreover, leveraging these rationales, we challenged maintainers' original decisions in four disputed cases, and all received approval.

0.53

Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation

cs.CL cs.AIcs.HCcs.LG ·2026-09-22 ·Zeyu He, Zhuqian Zhou, Kirk Vanacore, Rene F. Kizilcec, Ting-Hao 'Kenneth' Huang
similarity 0.57
Abstract

Large-scale text annotation brings expert insight to millions of documents, often through a codebook that AI annotators follow. Developing a robust codebook, however, takes months. Large language models (LLMs) could speed this process by applying an early codebook to the data, surfacing cases with strong LLM disagreement, and eliciting expert feedback to address them. We examined three ways experts can provide feedback for LLM codebook revision: (i) editing LLM-generated revisions driven by cross-LLM disagreement (Codebook Verifying), (ii) answering questions about LLM disagreements (Question Answering), and (iii) labeling disagreement cases with rationales (Rationale Labeling). Experiments on thousands of tutoring-session transcripts show that Rationale Labeling yielded the highest LLM-labeling accuracy (64.9%) against expert labels, outperforming the expert-revised codebook (57.8%). The best Question Answering setting also outperformed it (60.5%). Our work shows that LLMs can be used to strategically target expert attention, shortening months of codebook revision to days without sacrificing labeling performance.

0.43

How AI Models Manage Epistemic Authority: A Taxonomy and Comparative Analysis of Responses to User Disagreement

cs.CL cs.AIcs.HC ·2026-09-07 ·Riyadh Alnasser, Yusuf Mücahit Çetinkaya, Sumin Zhao, Tuğrulcan Elmas
top venuebenchmarkdataset similarity 0.56
Abstract · Accepted to EMNLP 2026

Large language models are increasingly used as sources of advice and information, including in high-stakes settings, yet little is known about how they respond to user disagreement. We study how a model manages its epistemic authority, referring here to its claim to knowledge, competence, or the right to advise, once a user challenges its answer. Building on Conversation Analysis, we introduce a taxonomy of six challenge types and a four-layer framework for analysing each response: whether the original claim is maintained or changed, where authority is located, how the disagreement is socially managed, and what kind of evidential support is offered. We construct a new dataset of 2,310 controlled challenge scenarios and 32,340 corresponding responses from 14 models, and analyse them using our framework with an LLM-as-judge pipeline, providing a vocabulary which future evaluation and benchmark design can build on. We find that models show conflicting behaviour: they validate users in 85% of responses but maintain their original claim in 65%. They explicitly apologise in 33% of responses, yet 59% of those apologies accompany maintenance of the original claim. They transfer authority most often in advice tasks, doing so in 28% of responses and reaching 57% in health advice and 49% in legal advice, compared with 6% in fact and 3% in explanation tasks. Abandonment of the original claim ranges from 0.8% for GPT-5.2 to 40% for DeepSeek 7B, while complete replacement of the original claim is rare overall at 1.5%.

0.56

You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come From

cs.CL ·2026-09-21 ·Calvin Zhou, Vincent McCloskey, Krishna Srinivasan
code/dataunusual similarity 0.56
Abstract · Accepted to the 13th Web as Corpus Workshop (WaC-13) at EMNLP 2026. 14 pages, 4 figures. Code and data: https://github.com/bodhiumlabs/tell-…

Questions scraped from the web are used across academia and industry as a proxy for what people want to know. Across QA training data, retrieval benchmarks, and content strategy, questions on a page are assumed to reflect human intent. We test this assumption at scale by extracting 13.4B question occurrences across 110 FineWeb snapshots (2013-2025), and report three findings. First, you can tell who is asking: provenance (the host/page of questions) leaves a signal in question form, and a logistic model can separate genuine user questions from templated/manufactured ones at AUC 0.725 via length and surrounding context rather than question type, though only 0.554 against commerce FAQ writing. Second, question frequency does not measure demand: the most-frequent questions are boilerplate/templated (over 70% of the top thousand), so occurrence counts measure how often a string was published and not how often it was asked. Third, over twelve years the genuine share of occurrences fell by 79% (42-56% after controlling for crawl composition), with question length and context decreasing. We present the first diachronic, occurrence-level measurement of web question provenance, and find the crawlable web's questions have shifted from being asked by humans toward manufactured for machines to read.

0.49

Big-Big-SycoBench: A Benchmark to Detect Sycophancy Through Model Responses to Valid vs. Invalid Arguments

cs.AI ·2026-10-11 ·Marthe Ballon, Arne Vanhoyweghen, Seorin Kim, Francesca Carlon, Lynn Delcon +9 more
benchmark similarity 0.56
Abstract

Sycophancy in large language models risks amplifying unsupported claims or beliefs across the knowledge ecosystem as these systems increasingly contribute to research synthesis, evaluation, and decision-making. A reliable model should revise its judgement when a user identifies a genuine flaw, but should resist pushback based on fallacies, emotion, or irrelevant appeals to authority. While sycophancy is usually measured on questions with a verifiable answer, the evaluative tasks in which it is most difficult to detect have no such ground truth. We introduce Big-Big-SycoBench, which measures sycophancy through argument discernment: the ability to revise an evaluation after valid criticism while resisting invalid pressure. The benchmark consists of 450 text artefacts with 5,400 arguments for raising or lowering their quality scores, each reviewed by one or more of twelve human annotators. An LLM scores each artefact twenty times to establish its baseline score and uncertainty. The model is then given a valid or invalid argument and asked to update its score. We only count a revision if it exceeds the model's own baseline variability. We use the resulting score delta to distinguish responsiveness to valid arguments from compliance with invalid ones. Across ten models, responsiveness is 88-95\% for seven, 76-77\% for Kimi K3 and GLM-5.3, and 32\% for Opus 5.5. Compliance ranges from 0.2\% for GPT-6 Astra to 95\% for GPT-4o. Big-Big-SycoBench is intended to guide the development of models whose judgements can be corrected by sound arguments, but not displaced by unsupported pressure.

0.14

Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers

cs.CY cs.AIcs.SE ·2026-09-08 ·Louis Yiven Zhu
positioncode/data similarity 0.55
Abstract · 31 pages, 3 figures, 10 tables. v2: coding checked by three re-runs, a blind re-coding and a second coding; strict rate revised from 67% to …

Frontier AI developers publish safety frameworks that commit them to evidencing whether their models are dangerous. The European Union and California now treat these documents as instruments of accountability and impose duties on their revision. Neither requires the developer's account to identify each change to a commitment. We introduce the silent revision rate, the share of material changes to a framework's commitments that the developer's published account does not identify, and we release the versioned, hash-pinned corpus needed to compute it. The corpus holds every public version of the safety frameworks of the twelve developers that have published one, with each provider's changelog, redline or announcement. We trace 710 commitment instances across twelve consecutive version pairs, code them against a frozen codebook, and check the coding with three independent re-runs, a blind re-coding of every disputed row and a second coding of a stratified sample. First, 64% of material changes (95% CI 59 to 69) are silent under a strict standard and 30% under a lenient one. Second, the form of the account matters less than its granularity. Narrative and itemised accounts differ by eight points that eight pairs cannot separate from chance, whereas the rate falls with the number of discrete items a provider publishes. Third, 75% of traced changes weaken or remove a commitment, and weakenings are silent more often than strengthenings, at 70% against 52%. A justification explains why a framework changed and an enumeration states what changed, and only the latter makes revision auditable. We argue that publication duties should carry an enumeration duty, which one provider already meets, voluntarily and incompletely.

0.32