CS Paper Radar

Multi-agent discussion gains less when dissent is withheld

Chand Sahil Mansuri, Xin Wang, Mengying Li, Bryan Acton, Rory Eckardt, Dhaval Patel, Sadamori Kojaku

cs.CL physics.soc-phcs.MA ·submitted 2026-09-29 18:00 UTC · v1
benchmark

Abstract

Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on whether discussion improves accuracy or leads to an incorrect consensus. Here, we introduce a parsimonious model that explains when discussion improves accuracy and when it ends in an incorrect consensus, built from four behaviors repeatedly observed in LLM agents: (1) withholding dissent, (2) internalizing a stated answer, (3) reconsidering after seeing dissent, and (4) correcting toward the correct answer. The model shows that discussion can overturn an incorrect initial majority only when the withholding rate $c$ is below a critical rate $c^* = γ/(γ+ a)$, set by the net correction rate $γ$ and the internalization rate $a$. We estimate these rates from conversation logs with a Bayesian method and place LLM teams relative to $c^*$. As the model predicts, the gain from discussion shrinks as withholding rises, across LLMs and on a hidden profile benchmark, HiddenBench, and MedEInst. Instructing agents not to withhold dissent increases this gain. Turning reasoning off also increases the gain, because reasoning raises the internalization rate $a$ and keeps agents from reconsidering a minority answer. These findings reconcile the conflicting reports and identify when discussion outperforms majority voting.

Closest papers

Nearest neighbours by meaning across the whole index.

Believed but Not Said: Measuring Hidden Belief Collapse in Multi-Agent LLM Discussions

cs.AI cs.CLcs.MA ·2026-10-15 ·Liangjie Huang, Henry Peng Zou, Wei-Chieh Huang, Wooseong Yang, Yuwei Han +1 more
similarity 0.70
Abstract · 14 pages, 6 figures

LLM systems increasingly rely on groups of agents that discuss, critique, and check one another, assuming that peers will catch individual errors. We test this assumption against a minimal adversary. In a ten-agent factual discussion, one or three adversarial agents repeatedly assert a plausible false answer. After each round we probe each benign agent's private answer through a separate call whose response is hidden from its peers, and we separately label its public stance toward the claim. Across six models and three factual-QA datasets, groups remain stable without an attacker: only 0.6% of benign agents finish with the false answer. A single adversarial agent increases this rate to 44.8% and three to 60.5%. Studies of conformity typically report public agreement without private change; we find the reverse. With three adversarial agents present, benign agents hold the false answer in 51.6% of exposed turns but publicly endorse it in only 19.7%. Much of the influence therefore remains invisible in the discussion, and the false answers persist when agents are asked for their own views. Agents that retain the true answer often fail to defend it; once adopted, the false answer spreads among benign agents, and baseline knowledge does not predict robustness.

0.41

Bayesian Belief Layer for Controllable Opinion Dynamics in LLM Agents

cs.MA cs.AI ·2026-09-18 ·Hafsa Akbar, Daniel Platnick, Marjan Alirezaie, Hossein Rahnama, Alex 'Sandy' Pentland
similarity 0.69
Abstract · Accepted to The 2nd Workshop for Research on Agent Language Models (REALM) at EMNLP 2026

LLM agents in social simulation revise their opinions implicitly, in context: how open an agent is to persuasion can neither be specified nor verified, and collective outcomes inherit the model's training prior. We introduce Bayesian Chronicle Agents (BCA), a minimal belief layer separating what an agent believes from how it speaks. Each stance is a probability, updated by one Bayesian step per utterance heard. A single prior-strength parameter $κ$ encodes stubbornness, modeled after its role in Friedkin--Johnsen (FJ) opinion dynamics. We then sweep this parameter to yield three canonical regimes of opinion dynamics on demand (consensus, persistent disagreement, committed-minority influence), with persistent disagreement matching the FJ closed-form fixed points at $R^2\!=\!0.93$--$0.99$. We further show that prescribed $κ$ remains recoverable after the language round-trip, with perfect rank-order recovery across all four models. Explicit belief also makes simulation auditable: the layer surfaces systematic per-model stance biases that end-to-end simulation would silently absorb.

0.31

Reinforcement Learning of Communication in a Mesh of Small Language Models

cs.LG ·2026-09-24 ·Mehmet Kerem Turkcan
similarity 0.69
Abstract

Language models gain accuracy from more compute at test time, but majority voting over independent samples saturates: as samples grow, the vote converges to the model's most frequent answer. Communication can add what sampling cannot: an agent that solves a problem can pass the key step to the others. We present TalkMesh, a decentralized mesh of small language model agents that learns when and what to communicate. Each agent samples a proposal and scores it with a trained confidence head. The most confident agent broadcasts a hint; agents below a confidence threshold revise, keeping each revision that outscores its proposal. Gossip consensus approximates the vote weighted by confidence without a coordinator. A talk policy, trained with group relative policy optimization on the change in correctness after revision, writes hints and revisions. With three agents, which together generate at most six outputs, the mesh reaches the accuracy of majority voting over 32 samples with each of three models. Trained with at most 8 agents and evaluated with 32, it raises accuracy from 0.568 under self-consistency to 0.705 (Qwen3.5-0.8B, GSM8K) and from 0.492 to 0.722 (SmolLM3-3B, MATH-500). When 4 of 8 agents collude on a wrong answer with fabricated confidence and poisoned hints, majority vote accuracy falls to 0.000 (Qwen3.5-0.8B, GSM8K). A defended mesh, whose agents rescore solutions with their own confidence heads, retains 0.507. Across reasoning, embodied coordination, and traffic signal control, messages improve a decision when the acting agent cannot observe the information it requires and another agent can send it.

0.25

What Does Multi-Agent Debate Actually Change?

cs.AI ·2026-09-07 ·Chen Qian
code/data similarity 0.68
Abstract · Code: https://github.com/chenmoneygithub/llm-committee

Multi-agent debate, in which several LLMs exchange arguments before producing an answer, raises a basic question: does expressed disagreement reflect changes in the members' own positions? No single signal can settle this question, so we organize the analysis around five questions: (A) does the debater say it disagrees; (B) does its reply text actually argue; (C) does its own position change after each debate turn; (D) how much, quantitatively, does the position change; and (E) how do members' final positions compare with their initial ones? We evaluate two- and three-member committees on 50 curated opinion questions from GlobalOpinionQA, using same-model, same-family, and mixed-family configurations with friendly, neutral, and hostile instructions assigned at each turn. (A) Tone changes reported agreement: the share of replies reporting strong agreement is 70.7-96.3% under friendly instructions, compared with 9.5-19.3% under hostile ones, varying with model choice. (B) Self-reports broadly align with text judgments, but consistency varies by agreement level and model choice. (C) Position changes depend on the interaction: after a peer's leaning-disagree reply, members reporting strong agreement switch options more often than those reporting leaning disagreement. (D) For open-weight members, the probability of the option a member already holds stays near saturation, even after a peer's pushback, while endorsement of its earlier position text drops after a peer's argument relative to neutral filler, more for Qwen3.8-27B than Inkling. (E) The selected option is unchanged from members' initial to final positions in over 90% of comparisons in every configuration. Taken together, expressed disagreement need not translate into position revision, either within individual exchanges or over a complete debate.

0.27

Phantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-Making

cs.MA cs.AIcs.CL ·2026-11-06 ·Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta
publishedbenchmark similarity 0.68
Abstract · Accepted at NeurIPS 2026, SocialAgent Workshop

Multi-agent LLM systems increasingly aggregate judgments across model instances, but reasoning about other agents can itself introduce failure. We study a setting in which LLMs shift their public decisions after observing ambiguous social signals while reporting inaccurate estimates of what other agents privately judged, even when the group's recorded private judgments do not support the apparent consensus. We call this failure phantom consensus and study it as a pluralistic-ignorance-style pattern in controlled LLM collectives. To measure this phenomenon, we introduce PHANTOM-Bench, a benchmark that separates isolated private judgments, public decisions after ambiguous social signals, elicited estimates of the group's private judgment, transparency-based recovery, and controls for neutral public text and explicit vote-following. Across nine closed- and open-weight models, ambiguous public signals consistently degrade decision quality and can turn correct private group judgments into incorrect public collective outcomes. Public accuracy drops by as much as 47.8 percentage points, and collective rationality failures reach 47.3% in the most susceptible setting. We observe substantial errors in reported group-private beliefs alongside public-stage failures, while control conditions reveal model-dependent differences in susceptibility to ambiguous and explicit peer signals. Revealing the agents' recorded private decisions repairs many failures without revealing the gold label, although recovery varies across models. Our results show that evaluating multi-agent LLM systems requires measuring not only final agreement, but whether that agreement reflects independently measured private judgments or an apparent consensus.

0.17

Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study

cs.AI cs.MA ·2026-10-14 ·Isabel Jones, Matthew Edwards, Janina A. Hoffmann
code/data similarity 0.67
Abstract

Large Language Models (LLMs) are increasingly deployed in multi-agent environments that produce unanimous agreement, both in intended interactions that need coordination and in unintended emergent interactions. Consequently, understanding how LLMs with competing incentives reach consensus is crucial to better interpret and design LLM collectives. We introduce an evaluation, based on the Network Consensus game, to assess the mechanisms driving this process. Here, a network of agents must agree on one of two colours to receive a payoff, but agents are offered different payoffs for the agreed colour. We vary components such as communication access and network structure and assess six open-source models and one closed model (ChatGPT 5.6-sol). We find that open-source LLMs reach consensus less often than humans in comparable experiments and only if messages are communicated across the global network. ChatGPT 5.6-sol, however, is able to reach consensus with communication limited to its neighbours and with no communication ability. This is achieved through lower stubbornness and higher majority-following tendencies, and by providing more information about its neighbours' states in messages. However, situating agents in a highly clustered graph reduces consensus, regardless of these traits, demonstrating that both system design and individual model traits drive collective outcomes.

0.32

Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation

cs.CL cs.AI ·2026-09-03 ·Xuanfa Jin, Zhijian Ma, Yongcheng Zeng, Xinyu Cui, Haifeng Zhang +1 more
top venue similarity 0.67
Abstract · EMNLP 2026 Findings, 24 pages, 4 figures

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R$^2$-MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R$^2$-MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R$^2$-MAD achieves consistent improvements over existing single-agent and MAD baselines.

0.60

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate

cs.CL ·2026-09-28 ·Xin Li, Mengbing Liu, Chau Yuen
top venuecode/data similarity 0.66
Abstract · Accepted at NeurIPS 2026 (Evaluations and Datasets Track). Project page: https://lixin.ai/DebateLedger. Code: https://github.com/LiXin97/Deb…

Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but a changed answer is not necessarily a better one: the same discussion can correct an initially wrong majority or collapse an initially correct one. Standard final-accuracy evaluations conflate these opposing transitions. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol records 253 collapses together with the corrections that offset them, and the correction side changes how interventions should be judged. Replay experiments reveal the central tradeoff: freezing every debate prevents 251 collapses but loses 804 corrections under equal weights, and a leave-one-model-out probe-gated freeze that is tuned for net utility never fires, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman $ρ$ = 0.893, exact two-sided p = 0.0123), but initial-majority accuracy is a close comparator ($ρ$ = -0.821; family partial $ρ$ = 0.767, p = 0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both collapses and corrections. Reasoning-mode models, including one model at two reasoning-effort levels, also show both collapses and corrections. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model rows can be compared under the same denominators and signed utility ledger.

0.52