CS Paper Radar

Disentangling Threads: Exploring the Potential of LLM-Supported Discussion Forum Analysis for Community Insight

Tony W. Li, Zhiqing Wang, Thanh-Nha Tran, Yu-Chun Grace Yen, Steven P. Dow

cs.HC ·submitted 2026-08-20 22:00 UTC · v1
published

Abstract

Online discussion forums enable people from diverse backgrounds to share ideas, feedback, and perspectives. These organic discussions can help researchers understand communities' collective viewpoints, but insights are often difficult to uncover given their freeform reply structure. Large language models (LLMs) support qualitative text analysis but can misalign with researchers' analytical intent and miss key insights. To inform design considerations for forum sensemaking tools, we manually analyzed a forum discussion, synthesized an exploratory analysis framework from relevant literature, built a design probe, and interviewed 21 researchers to uncover perceived opportunities and barriers with LLM representations of collective discussions. We provide recommendations for community sensemaking tools to support flexible analytical goals grounded in raw user data and enable follow-up research processes, while balancing anonymous free expression with the desire for contextual information on commenters.

Comment
Accepted to ACM Collective Intelligence Conference, 2026
DOI
10.1145/3834581.3838629

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond One User and One Model: Rethinking LLMs as Mediators in Collaborative Sensemaking and Ideation

cs.HC ·2026-08-20 ·Pengqi Wang, Emily Kuang
publishedposition similarity 0.63
Abstract · Accepted by 2026 Co-Data : Cultivating Effective Human-LLM Collaboration for Collaborative Data Processing Workshop at ACM CHI

As LLMs enter qualitative data workflows, they are predominantly designed for individual researchers rather than collaborative teams, which risks disrupting the collective negotiation essential to sensemaking, driving premature consensus, and potentially erasing marginalized voices. In this position paper, we argue that LLMs must be reconceived as active team mediators. Grounded in empirical studies of human-LLM collaborative analysis, we propose three design directions: quote-level traceability to preserve accountability of human and LLM contributions, controllable delegation boundaries to maintain researcher agency across workflow stages, and structured dissent mechanisms to surface interpretive conflicts that teams might otherwise suppress. We ground these directions in accessibility research as a domain where methodological rigor and representational equity are obligatory.

0.41

How Much Can AI Understand? Toward AI-Assisted Sensemaking of Collaborative Discussion in Groups with Shared History

cs.HC cs.AI ·2026-08-28 ·Soobin Cho, Mark Zachry, David W. McDonald
position similarity 0.63
Abstract

AI tools that support collaborative discussion typically treat the discussion as a standalone task, focusing only on its content and setting aside the social context of the group having it. But it is groups with a shared history, with their own norms, hierarchies, and relationships, where the most tangled and complex discussions tend to arise. These discussions cannot be understood apart from that context, and AI that overlooks it risks failing to convey what a discussion means, or even misrepresenting it. Drawing on two studies of how experienced Wikipedia editors read and make sense of discussions, we propose an AI-Assisted Sensemaking Model for Collaborative Discussions, which captures not only a discussion's arguments but also the norms and participants behind it, along with the context that gives each meaning. In this model, the system supports the early stages of the sensemaking process, and the degree to which it performs interpretive work can range from low to high. We argue that higher interpretive work reduces the burden on users but increases their reliance on the system's judgment. We then discuss the risks of an insufficiently intelligible system, what it would take to make one more intelligible, and the safeguards it still requires.

0.26

CrossWeave: Bridging Perspectives Across Online Communities with a Dual-Pane Design

cs.SI cs.HCcs.IR ·2026-10-07 ·Fei Fang, Reva Hirave, William Jurayj, Yuqi Li, Brian Lu +17 more
published similarity 0.62
Abstract · CSCW 2026 + small improvements

Social media systems typically display conversations among already familiar contributors, which can be predictable and one-sided. In civic discourse, this design narrows discussion, reinforces divides, and distorts the perception of public opinion. To encourage cross-community engagement, we present CrossWeave, an AI-powered bridging system that augments the standard social media feed. As the user reads a post, CrossWeave surfaces diverse relevant posts from other threads in a side pane and highlights the connections. Users are invited to venture out of their echo chamber, explore a broader range of views and arguments, and ``click across'' to engage with their authors. When they do, CrossWeave facilitates constructive posting, not only by showcasing relevant past content but also by simulating possible reactions as the user drafts a post.

0.41

Using LLMs to Mimic the Conversational Dynamics of Reddit Communities

cs.HC ·2026-08-29 ·Vedaant Jain, Yoshee Jain, Ishq Gupta, Aditi Shrivastava, Koustuv Saha +1 more
similarity 0.62
Abstract · 5 pages, 2 tables, 1 figure

Online communities face a constant battle against toxicity and misinformation. While human moderators struggle to keep pace with the volume of content, LLMs offer a promising solution for automatically generating constructive responses and shaping online interactions. This paper preliminarily investigates if LLMs can mimic the communication styles of Reddit users using their comment history as context. We evaluate two prompting approaches: predicting a target comment and filling in masked comments. We find that LLMs outperform expectations at replicating comment structure and formality, but struggle to accurately capture nuanced emotions, e.g. understating joy and overstating anger. These findings highlight a promising direction for LLMs in guiding online conversations towards prosociality influencing emergent communication patterns and norms within the community. The results of our study inspire future work with more rigorous methods of evaluation to explore the LLMs' effectiveness across diverse online communities to better understand their broader societal impact.

0.25

LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads?

cs.CL ·2026-10-02 ·Xinyi Liu, Rinat Khaziev, Dilek Hakkani-Tür, Tarek F. Abdelzaher
top venuebenchmark similarity 0.62
Abstract · Accepted to EMNLP 2026 Main Conference

Long-context LLMs can now ingest entire online discussion threads, but understanding their social discourse requires more than reading a long document: models must track parent-reply relations, turning points, scoped subtrees, cross-branch contrasts, and participant trajectories. To test this structure-aware social reasoning, we introduce LongSocialBench, a benchmark of 1,462 verified human-authored multiple-choice items drawn from 94 complete Hacker News, Stack Exchange, and Reddit r/ChangeMyView episodes, with a median length of approximately 73K tokens. Each item pairs a complete serialized discussion and reply structure with a four-option question, requiring models to recover structured social evidence. Released items are verified for answerability, option uniqueness, and evidence grounding. Across 18 models and 29 evaluation settings, current long-context workflows remain far below human performance. The best individual result comes from Claude-Opus-4.7, which reaches 63.0% when prompted to eliminate incorrect options before answering, compared with 72.4% for independent human readers. Averaged across all 18 models, the full-context Baseline scores 43.9%. Supplying the gold evidence scope raises this to 55.0%, showing that substantial errors remain even after the relevant thread region is identified. LongSocialBench shows that the missing capability is not context access or prompting alone, but social understanding over structured reply trees.

0.42

What Makes an Initial Reaction Ready for Discussion?: Multi-Persona AI Support for Stance Reflection and Writing

cs.HC ·2026-08-24 ·Sky Shih-Kai Hong, Mu-Tien Kuo, Wei-Ji Chen, Dennis Wang
published similarity 0.61
Abstract · 5 pages, 3 figures, 2 tables. Accepted to TAICHI 2026 (https://taichi2026.taiwanchi.org/program)

An initial reaction to a social or community issue can feel meaningful before it is ready to become a message: people still need to clarify the claim, anticipate audience risks, and decide how much reasoning should become visible to others. We present StanceLab, a prototype for preparing a stance before entering a discussion. The prototype compares a three-persona mode, where an Interviewer, Mentor, and Opponent respond in parallel to help users diagnose and revise a stance, with a standalone LLM mode. In a formative within-subject pilot with six participants and 12 task sessions, every session produced a short final message in the notepad. The pilot revealed two design requirements: persona roles should diagnose useful blind spots or objections, and parallel responses need coordination support. We propose a future diagnosis-and-writing workflow that turns persona-based reflection into selective, audience-aware final messages.

0.38

The Infrastructure Divide: Analyzing Community Stance on Data Center Development via Reddit

cs.AI ·2026-11-06 ·Yoko Koyama, Gongfan Chen, Mansi Saxena
code/data similarity 0.59
Abstract · 42 pages, 7 figures

The rapid expansion of AI-driven data center construction has outpaced public engagement: an estimated $64 billion in U.S. data center projects were blocked or delayed by local opposition between May 2024 and March 2025, yet the concerns driving this resistance remain poorly characterized. Prior engagement methods such as surveys and hearings are introduced late in the planning process, and no study has systematically mined social media to measure community stance at scale. To address this gap, this study presents an LLM-powered opinion-mining framework and applies it to 798 Reddit posts and 102,328 associated comments spanning a decade. An open-source large language model, guided by a Chain-of-Stance prompting framework and three domain-specific labeling rules, classified stance at 83% of the human inter-reviewer agreement ceiling. Opposition dominated discourse volume rather than disapproval: against posts outnumbered favor posts 2.6:1 and generated 8.5 times more comments, and this gap widened significantly over 86 weeks. Topic modeling of opposition arguments resolved into four reproducible clusters, resource consumption and scarcity (42.8%), contested siting processes and secrecy (34.2%), political opposition to data centers (13%), and political and corporate accountability (8%). Engagement analysis revealed that posts with emotional or provocative political framing attracted the most engagement, while key technical concerns (e.g., water, power) were dispersed across lower-engagement comment threads. These findings give local leaders, developers, and policy makers a quantified, continuously updatable basis for prioritizing concerns and policy levers, supporting a shift from opposition to negotiated, verifiable partnership for data center development.

0.32

TractorBeam: Personalized AI Sensemaking Support via Collaborative Machine Annotation

cs.HC ·2026-08-19 ·Sireesh Gururaja, Jordan Taylor, Emma Strubell
published similarity 0.59
Abstract · UIST Poster Extended Abstract

Language model-based systems which allow asking questions of documents have become popular tools for sensemaking. Despite their implied capability, these systems still suffer from issues of factuality and provenance, while encouraging confirmatory, rather than exploratory, research. We present TractorBeam, a browser extension-based mixed-initiative system that uses collaborative annotation as an interface metaphor for sensemaking, re-framing language model (LM) outputs as suggested highlights in a process that we call collaborative machine annotation. This metaphor allows us to present LM results in-context on PDF documents, directly addressing concerns of provenance and factuality, while allowing users to iteratively construct mental schemas and queries for language models directly in the context of a document. In a preliminary user study, all of our participants felt that TractorBeam enabled them evaluate and iteratively improve the model's reflection of their intended highlighting, and several found suggestions that made them reconsider their original schema. TractorBeam suggests that systems that facilitate exploratory research on individual documents may lead to verifiable sensemaking for users and complement tools that work across broader corpora.

0.22