CS Paper Radar

Human and LLM Facilitation in Collaborative Design Meetings: A Computational Behavioral Comparison

Mohammed Azeez Khan, Aaron Dsouza, Ashutosh Mishra, Ummul Faiz Z. Bibi, Neha K. Nair, Arshaan Khan

cs.LG ·submitted 2026-10-09 06:36 UTC · v2

Abstract

Large language models (LLMs) are increasingly being explored as facilitators of group discussion and collaborative work, but the behavioral differences between LLM-generated and human facilitation remain insufficiently characterized. We compare human Project Manager facilitation turns from scenario-based collaborative design meetings with Llama 3.1 8B Instruct responses across 199 paired contexts from the AMI Meeting Corpus. The analysis covers five confirmatory dimensions: contextual semantic distance, a lexical elaboration proxy, empathyand-hedge lexical density, topic divergence, and phase-vocabulary alignment. An exploratory directiveness measure failed domain validation and is not interpreted. The sampled human Project Manager turns were higher on contextual semantic distance (r = 0.644) and on empathyand-hedge lexical density (r = 0.203). The second contrast is carried mainly by hedge terms, so we read it as lexical hedging rather than psychological empathy. Llama-generated facilitation was higher on topic divergence (r = 0.703), the lexical elaboration proxy (r = 0.288; lengthconfounded), and instruction-conditioned phase-vocabulary alignment (r = 0.382). A secondary no-phase check left the Llama-higher phase contrast in place. The system prompt still requested a facilitation move, so that check is not evidence of spontaneous phase awareness. These differences should not be interpreted as a ranking of facilitation quality. The resulting behavioral profile provides a reference point for later comparisons of LLM facilitators, human facilitators, prompts, models, and collaborative settings.

Comment
17 pages, 7 figures, 6 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Evaluating Context Sensitivity of Collaboration Behavior in Multi Party Simulation Frameworks

cs.MA cs.AI ·2026-10-22 ·Anvesh Rao Vijjini, Sheshera Mysore, Pegah Jandaghi, Ashish Sharma, Snigdha Chaturvedi +1 more
similarity 0.63
Abstract

Group Social Behavior is increasingly used as a lens to study Large language model (LLM) simulations to aid model training and evaluation. However, little prior work has examined how contextual factors such as team size and goal type manifest in long running simulations. To overcome this gap, we conduct a controlled, full-factorial study examining how contextual factors affect collaboration behavior across multi-party simulation frameworks. Using well established social science taxonomies, we quantify context sensitivity of collaboration behavior in simulated teams. Our analyses reveal significant effects of contextual factors such as simulators consistently favor smooth collaboration over disagreement, negotiation pressure, and coordination breakdown. These findings provide the first systematic evidence of how contextual factors influence emergent collaboration in LLM based multi-party simulation frameworks, offering both guidance and empirical results for the use of such simulations.

0.24

Semantic Variability of Replies Across LLMs: Implications for Designing Conversation-Based Assessment

cs.CL cs.AIcs.HC ·2026-08-13 ·Jiangang Hao
published similarity 0.61
Abstract · 9 pages, 4 figures, two tables. Accepted to the AI in Measurement and Education Conference 2026

This study examines whether LLM-generated replies remain semantically consistent when the underlying LLM changes. Using messages from real collaborative conversations, we compared the semantic similarity of generated replies across LLMs under two conditions: with and without preceding chat history. Results show that model choice and conversational context both affect response similarity and alignment with human replies. These findings indicate that prompting and conversational context alone may not be sufficient to preserve response consistency across LLMs, highlighting the need for infrastructure and design strategies that can maintain stable and comparable responses amid the rapid and continuous evolution of LLMs.

0.25

CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation

cs.HC ·2026-08-13 ·Jiin Choi, Kyung Hoon Hyun
top venue similarity 0.59
Abstract

LLM-based chat systems have become valuable tools for design practice, enabling rapid ideation and flexible task support. Yet these systems process designer utterances as generic sequences, maintaining context through recency rather than through any model of how the speaker organizes knowledge. In design conversation, this gap compounds as relational context decays between turns, identical words go unresolved across designers, and the conversation loops or restarts rather than deepens. We present CogChat, a real-time chat framework that grounds conversational AI in a personal heterogeneous knowledge graph constructed from each designer's input. The system extracts typed entities and relations into a heterogeneous graph, then applies a HGT (Heterogeneous Graph Transformer) to select structurally relevant nodes for response generation and to generate both intentional and exploratory probing questions. Technical evaluation shows that HGT-based entity selection outperforms both ungrounded LLM interaction and naive KG augmentation, which introduces noise that degrades response quality. A within-subjects study with nine professional designers indicates that grounding conversation in a relationally structured, designer-specific semantic context improves context retention, personalized intent interpretation, and conversational depth while reducing cognitive load. These findings suggest that structuring a designer's expressed concepts and relations as a dynamic knowledge graph can preserve relational context that fades across turns, pointing toward a graph-grounded approach to long-term context management in LLM-based interaction.

0.34

On the Prospects of Dynamic LLM Conversations in Software Development

cs.SE cs.AI ·2026-08-31 ·Annemarie Wittig, Alina Mailach, Janet Siegmund, Norbert Siegmund
similarity 0.59
Abstract

Large language models (LLMs) have become an essential tool for assisting developers, yet we still lack knowledge on ways to effectively support their interactions during development activities. That is, the quality of interactions with a chat-based LLM still strongly depends on how developers phrase prompts and which information they include. Our goal is to evaluate whether interventions into these interactions with LLMs have an effect on software developers---be it harmful or beneficial. To this end, we conducted a four-month longitudinal study with third-semester computer science students working on a full-stack Web development project using chat-based LLMs under three conditions: (1) a \emph{context}-aware group received intent-based conversation augmentation, (2) a \emph{proactive} group received follow-up suggestions and tailored advice, and (3) a \emph{control} group without intervention. Our augmentations are minimal: (i) to reduce confounding factors and (ii) to isolate treatment effects. Analyzing interaction logs and user surveys revealed no major differences in interaction patterns, indicating no detectable harmful effects in the measured outcomes when intervening in interactions. Moreover, we observed trends of increased satisfaction with the \emph{proactive} treatment. The results indicate that even with minimal interventions, dynamic guidance mechanisms for developer-LLM interactions show observable effects, such that more severe augmentations may have the potential to substantially improve developer satisfaction.

0.22

Mapping the Emerging Social Science of Large Language Models

cs.CY cs.AIcs.CL ·2026-09-07 ·Zhanzhan Zhao, Yi Yang, Xiao Jia, Zeyun Dong, Chenzhang Wang
similarity 0.59
Abstract

As large language models (LLMs) blur the boundary between cognition and computation, they reshape knowledge production and collective life. We combine a curated review of 198 papers with computational mapping of 47,719 formally published papers from five databases. Text embeddings, clustering, and topic modeling organize the curated literature into 13 subcategories across three domains. LLM as Social Minds examines mind-like reasoning and socially grounded behavior in individual models. LLM Societies explores how interactions among agents produce collective behavior and social organization. LLM-Human Interactions examines how people perceive and engage with LLMs, and how interaction reshapes human thinking, creativity, and relationships. Structural topic modeling of the larger corpus traces how research attention shifts over time and differs across publication venues. We present a reproducible framework that identifies the field's underlying structure and enables its continuous refinement as new studies emerge. The review also identifies a gap in current understanding: the mechanisms underlying many of LLMs' observed cognitive and social behaviors remain unclear. Human-like performance can then be mistaken for understanding, obscuring the norms, biases, and structures of power that LLMs reproduce. We call for shared benchmarks and robustness checks across the three domains to build a reliable evidence base for understanding LLMs' social behaviors and informing their design and use.

0.13

Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams

cs.AI cs.CLcs.MA ·2026-10-11 ·Yirui Qi, Xiaoming Zhang, Jaehong Yoon
code/data similarity 0.58
Abstract · Project Page: https://lululu896.github.io/Agent-Sociodrama

Large language model (LLM) agents are increasingly deployed as teams, yet they fail in characteristically social ways: information holders stay silent, dominant speakers anchor groups on wrong answers, and norm violations go unsanctioned. We present Agent Sociodrama, a framework for modeling and validating evolving social dynamics in LLM agent teams. Under fixed personas, agents' emotional states and directed, asymmetric relationships are updated every round by a unified external LLM judge and shape their subsequent interactions. Across six LLM backbones and five social scenarios, we assess social fidelity through classical social-theory patterns and agreement with human judgments. The resulting trajectories exhibit structured social patterns and model-specific differences. We then introduce Trajectory-Guided Persona Recasting (TPR), which diagnoses group-dynamic failures from interaction transcripts and social trajectories and recasts agent personas before rerunning the task. Across four collaborative benchmarks, TPR improves performance from both heterogeneous and randomly initialized agent persona compositions, yielding up to 83.3%p of accuracy improvement over the initial casting. On representative distributed-information cooperation and mixed-motive bargaining tasks, we outperform SOTOPIA and AgentVerse in three of four comparisons, by up to 37.5%p in Pareto rate. These results show that validated social trajectories provide actionable signals for repairing LLM agent teams.

0.26

From Process Loss to Assembly Bonus: Human-Grounded Diagnosis of Multi-Agent LLM Collaboration

cs.MA cs.AIcs.CL ·2026-09-06 ·Ala N. Tak, Teruhisa Misu, Kumar Akash, Zhaobo K. Zheng, Kevin H. Joo +1 more
top venuehot direction similarity 0.57
Abstract · Accepted at EMNLP 2026. Camera-ready version

LLM agents are increasingly used for collaborative problem solving and human-group simulation. This makes outcome-only evaluation insufficient: if LLM groups are used as models of human groups, we need to know whether they succeed or fail through human-like deliberative mechanisms. We compare human group chats with matched LLM deliberation traces on Wason-style deductive reasoning, then test whether the same process signatures generalize to analogical, abductive, and analytical tasks. Humans and LLMs show the same assembly bonus asymmetry: discussion improves the average member more often than the best initial member. Initial-answer diversity accounts for the effect of model heterogeneity, increasing movement in both corrective and destructive directions. The main differences are process-level. Compared with humans, LLM groups follow majorities more often, surface less unique information, and converge earlier; correct minority signals succeed mainly when re-expressed early. Interventions motivated by human group-decision research yield modest improvements in collective outcomes, but do not remove the coordination bottleneck. Together, these results suggest that LLM groups can reproduce some outcome-level patterns of human deliberation while diverging in the mechanisms that generate assembly bonus and process loss, with implications for group simulation and human-AI collaboration.

0.50

Beyond One User and One Model: Rethinking LLMs as Mediators in Collaborative Sensemaking and Ideation

cs.HC ·2026-08-20 ·Pengqi Wang, Emily Kuang
publishedposition similarity 0.57
Abstract · Accepted by 2026 Co-Data : Cultivating Effective Human-LLM Collaboration for Collaborative Data Processing Workshop at ACM CHI

As LLMs enter qualitative data workflows, they are predominantly designed for individual researchers rather than collaborative teams, which risks disrupting the collective negotiation essential to sensemaking, driving premature consensus, and potentially erasing marginalized voices. In this position paper, we argue that LLMs must be reconceived as active team mediators. Grounded in empirical studies of human-LLM collaborative analysis, we propose three design directions: quote-level traceability to preserve accountability of human and LLM contributions, controllable delegation boundaries to maintain researcher agency across workflow stages, and structured dissent mechanisms to surface interpretive conflicts that teams might otherwise suppress. We ground these directions in accessibility research as a domain where methodological rigor and representational equity are obligatory.

0.41