CS Paper Radar

The Failure Happens Before the Drift: The Social Dynamics of Values in LLM Agent Societies

Farah Atif, Sougata Saha, Monojit Choudhury

cs.AI cs.MA ·submitted 2026-08-31 09:30 UTC · v1
publishedhot direction

Abstract

Large Language Model (LLM)-based agents are increasingly used as proxies for human participants in social science research, yet it remains unclear whether they can faithfully simulate diverse and conflicting human value systems. We present a World Values Survey (WVS)-grounded simulation framework where culturally diverse agents with different communication styles engage in longitudinal, value-laden discussions. Across approximately 4,000 conversations involving 1,200 personas, 15 topics, and three models (GPT-4o, Gemini-2.5-Flash, and Gemma-4-E4B), we evaluate value faithfulness, value drift, and conversational realism. We find that more than 50\% of personas fail to express their assigned WVS profiles from the outset, while 2-7\% drift after repeated conversations. Ablations removing demographic details improve faithfulness for some models but do not change the broader trend: simulated value distributions still systematically deviate from the assigned WVS profiles. Compared to human discussions, simulated dialogues show a different trade-off between stylistic consistency and semantic diversity, often producing content-wise varied but stylistically repetitive exchanges. These findings suggest that current LLM agents can generate plausible conversations, but remain limited proxies for representing and preserving diverse human value profiles over time.

Closest papers

Nearest neighbours by meaning across the whole index.

Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

cs.CL ·2026-09-05 ·Qishuai Zhong, Zongmin Li, Siqi Fan, Aixin Sun
top venue similarity 0.71
Abstract · EMNLP 2026 Findings

Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.

0.42

Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions

cs.CL cs.AI ·2026-10-06 ·Dennis Fucci, Andrea Bacciu, Dong Liu, Weronika Łajewska, Saab Mansour
dataset similarity 0.71
Abstract

LLMs are increasingly deployed as autonomous agents in social environments, making it critical to study their ability to faithfully simulate human interactions. Central to this is grounding agents in realistic user personas, yet existing datasets rely on fictional personas and are limited to a handful of languages, lacking the empirical grounding necessary to evaluate behavioral fidelity across diverse populations. We introduce Wiki-Talkie, a multilingual dataset of real-world conversations from Wikipedia Talk pages across five languages spanning two language families: Germanic (German, English) and Romance (Spanish, French, Italian), paired with personas derived from real user communities and encompassing sociodemographic attributes, self-descriptions, and behaviorally grounded interaction traits. Using Wiki-Talkie, we evaluate agent interactional behavior on a next-turn generation task across various persona conditioning strategies. Our evaluation assesses whether agents collectively reproduce the distributional behavioral patterns observed in human discussions. Results show that user's comment history exemplifying interaction behavior consistently outperforms explicit persona information. In addition, models systematically underproduce negative or extreme sentiments, while over producing references and suggestions, revealing biases toward agreeableness and positivity. Crucially, these patterns hold robustly across languages, with small cross-lingual differences.

0.13

From Simulated Citizens to Simulated Deliberation: Challenges in Representation and Interaction

cs.AI ·2026-09-07 ·Chaemin Jang, Junsik Min, Jaewoo Choi, Donggyu Lee, Haiin Lee +7 more
similarity 0.69
Abstract

Can LLM agents simulate how a real public would deliberate? For such simulations to be meaningful, agents must reflect the population's opinions, and their views must be shaped by what others say during discussion. We evaluate these requirements across six LLMs using national surveys, controlled deliberation experiments, and a real deliberative poll. Comparing census-grounded Korean personas against national surveys, we find that agents often produce overly concentrated opinions and fail to reliably reproduce demographic differences. During simulated deliberation, agents exchange arguments, respond to one another, and frequently change positions. Yet similar changes occur even when agents cannot see one another's contributions, while simply restating their starting positions sharply reduces movement. We further test whether these limitations persist in a real-world setting, using agents constructed from actual respondents in the 2017 Shin-Kori deliberative poll. Even with real respondent profiles, agents fail to reproduce the panel's initial opinion distribution or how its opinions change across survey waves. Our findings show that generating deliberative discussions, representing public opinion, and reproducing interaction-driven opinion change are distinct capabilities. These results suggest a different potential role for LLM deliberation: surfacing arguments and perspectives, rather than simulating a real public.

0.19

Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models

cs.AI ·2026-09-15 ·Keqing Zhang, Jingyu Chen, Yufan Liu, Yongqiang Zhu, Nai Ding +4 more
similarity 0.69
Abstract · Preprint. 9 authors

As Large Language Models (LLMs) increasingly handle complex subjective tasks, aligning their intentions and behaviors with human values has become a critical scientific challenge. However, current efforts are confounded by a striking behavioral paradox: they fluctuate unpredictably under minor wording changes ("swing"), yet stubbornly ignore explicit instructions to correct ingrained biases ("rigidity"). Resolving this duality is critical for reliable AI alignment. To systematically understand and safely steer these latent subjective preferences, our study is structured around three fundamental questions. First, do LLMs possess an intrinsic value system? By projecting responses from 106 LLMs (150,000 queries per model) and 95,000 human survey profiles into a shared sociological space, we empirically confirm that they do. However, they do not mirror human diversity, instead crystallizing into a highly concentrated, idealized value core. Second, how can these values be quantified? We propose the Prior-Environment-Cognition (PEC) framework. This model mathematically defines value expression as the joint outcome of inherent dispositions like parameter weights (Prior), external contexts such as user prompts (Environment), and internal reasoning processes like Chain-of-Thought (Cognition). Finally, how can LLMs' values be aligned toward a desired target? Using PEC diagnostics, we establish an adaptive "Alignment Prescription". Rather than blindly applying resource-intensive training, this method identifies the minimum effective intervention needed for each dimension, ranging from zero-cost prompts to targeted parameter updates. Extensive empirical validation confirms that our approach successfully verifies the presence of LLM values, accurately quantifies their shifts, and achieves more efficient and precise steering than conventional blind training, all without degrading general capabilities.

0.09

Measuring Model-Specific Alignment Drift in LLM-Based Opinion Dynamics

cs.AI physics.soc-ph ·2026-11-02 ·Nataša Djurdjevac Conrad, Sebastian Pokutta, Christof Schütte
similarity 0.68
Abstract

Large language model (LLM) agents are increasingly used as synthetic participants in opinion-dynamics experiments, often assuming that an agent's behaviour is fully determined by its prompt. We test this assumption with a framework in which LLM agents discuss a contested policy issue over repeated rounds while their stance is quantified by a fixed psychometric instrument independent of any dynamics model. Across six LLMs, committed personas are intrinsically stable: in each of the five models tested without peer input, every agent keeps its assigned side for 20 rounds. The observed loss of alignment under discussion is therefore interaction-induced, we call it alignment drift, and model-specific: gpt-4.1 keeps opinion clusters apart, claude-sonnet-5 holds them partially, and gpt-5.6-sol and claude-opus-5 abandon them. Under selective attention, one LLM maintains separated opinion clusters under stance-homophilous attention. Qualitative profiles persist at doubled population and horizon and on a second issue. Such heterogeneity makes the LLM choice a first-class validity variable for social simulation.

0.20

Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations

cs.CL cs.AI ·2026-09-24 ·Yeeun Chae, Yewon Choi, Seunghyun Lee, IL Im
published similarity 0.68
Abstract · Accepted to the EMNLP 2026 Workshop on Pluralistic AI & NLP (PANDORA)

Large language models (LLMs) are increasingly used as synthetic survey respondents to estimate population response distributions. In cross-cultural survey simulation, evaluations should assess not only distributional fidelity within countries but also whether differences across countries are preserved. However, existing distance-based metrics such as Jensen--Shannon divergence (JSD) do not directly capture such cross-country differences. To address this limitation, we introduce Cultural Divergence Preservation (CDP), a reference-light diagnostic based on a one-time human calibration. CDP identifies reduced cross-country divergence as cultural flattening and increased divergence as cultural caricature. To evaluate CDP, we conduct experiments across four LLM backbones, three persona-based prompting methods, and two survey domains, the World Values Survey (WVS) and the Big Five Personality Test. The results reveal a systematic discrepancy between conventional fidelity metrics and CDP. Controlled experiments show that CDP changes monotonically as cross-country divergence is attenuated or amplified, while the corresponding changes in JSD remain relatively small. In our audit of real LLM generations, DeepPersona-Inspired prompting is frequently favored by conventional fidelity metrics but exhibits the strongest flattening in every model--domain block. CDP thus complements fidelity metrics by directly quantifying the attenuation or amplification of cross-country divergence.

0.47

PERSONAWEAVER: Controllable Diversity Beyond Conventional Archetypes in Procedural Character Generation

cs.CL ·2026-09-22 ·Maan Qraitem, Kate Saenko, Bryan A. Plummer
publishedcode/data similarity 0.67
Abstract · Accepted at the 1st PANDORA Workshop: Pluralistic AI and NLP

Procedural character generation aims to populate games, simulations, and other virtual worlds with diverse characters. Large language models (LLMs) offer a promising foundation for scaling this task. However, LLM-based procedural character generation remains at an early stage: existing methods either generate characters directly or adapt profiles retrieved from persona banks. As we show, both approaches produce behaviorally homogeneous populations: characters overwhelmingly agree with positive moral norms and respond to questions with helpful, assistant-like reactions. To mitigate this homogenization, we introduce PersonaWeaver, which disentangles world building from behavioral specification and models behavior through setting general, diverse, manually curated banks of moral positions and conversational reactions. This design allows us to test how far LLM(s) can be pushed beyond their default behavioral patterns across settings. Across ten realistic and fantastical settings and three LLM(s), PersonaWeaver produces broader moral and interactional response distributions than prior work. Its guidance also diversifies interpersonal language, response length, and sentiment. It also produces less archetypal combinations of world attributes. Code is available at https://github.com/mqraitem/PersonaWeaver.

0.38

Value Faces: Surfacing How Self-Presentation Shifts Across Relationships

cs.HC cs.CY ·2026-09-17 ·Gabriel Koo, Rayhan Rashed, Farnaz Jahanbakhsh
similarity 0.66
Abstract

People present different aspects of themselves across relationships. Computational work has captured such variation in communication style. But this variation also extends to which principles people foreground or background in a particular relationship--i.e., in the values they express and how they balance them. We conceptualize these relationship-specific expressions of values as demonstrated values. To make demonstrated values visible, we introduce Value Faces, a system that analyzes a person's existing chat histories from their everyday messaging platforms using Schwartz's ten basic human values and produces separate value profiles for their different relationships. In a mixed-methods study(N=18), we find that the resulting value profiles distinguished participants' relational contexts with twice the odds of guessing, while system-inferred differences across relationships aligned with participants' perceptions of those differences. Participants used these profiles to articulate previously implicit differences in how they presented themselves, connect them to roles and changes over time, and reconsider their self-assessments.

0.22