CS Paper Radar

When Persona Attributes Improve Population Alignment in Large Language Models

Leon Fröhling, Jens Rupprecht, Markus Strohmaier, Claudia Wagner

cs.CL cs.CY ·submitted 2026-09-02 12:30 UTC · v1
survey

Abstract

Large Language Models (LLMs) are increasingly used to predict the responses of human participants in survey panels. Towards that goal, persona prompting has recently emerged as a technique to inform and align large pretrained language models. Persona prompting refers to the practice of using short textual descriptions of 'personas' in prompts to steer the LLM's generations. Personas describe individuals through different attributes such as their socio-demographics, attitudes, or behaviors, with the aim of aligning LLMs to produce responses that correlate with the corresponding human responses. Yet, recent work has produced mixed and partly conflicting results of persona prompting without clear patterns of success and failure. Among the few consistent findings is that the selection of persona attributes matters, and that using more attributes does not necessarily lead to better performance. It remains unclear how different attribute selection methods perform and how to choose among them. In this paper, we propose that observed human response variation of a survey question is a potential explanation for the mixed performance observed so far. In addition, we compare the performance of persona prompting associated with different methods for selecting persona attributes. We evaluate these methods on four different (general) social surveys across two countries, six LLMs, and twenty prediction tasks per survey. Our work helps to identify when persona prompting can be expected to be useful in survey prediction tasks, and provides new insights on the effectiveness of different attribute selection methods for LLM-based survey prediction using persona prompting.

Comment
45 pages, 15 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Data-Driven Personas for Survey Simulation: Insights into Simulation Alignment Across Data-Access Regimes

cs.AI ·2026-10-05 ·Dongryeol Lee, Weronika Łajewska, Leonardo Perelli, Saab Mansour
top venue similarity 0.76
Abstract · Work accepted at REALM EMNLP 2026

Large language models (LLMs) offer new opportunities for public opinion research by enabling early prediction of survey responses, potentially reducing the cost and time of traditional surveys. However, many existing steering approaches rely on target-domain human data for fine-tuning or prompting that is costly to collect and raises privacy concerns. In this paper, we study demographic group-level survey simulation, where personas induced from heterogeneous, anonymized public behavioral data condition agents that simulate responses of individuals from specific demographic groups. We examine whether representative personas can be induced from diverse sources and analyze how the domain, scale, and granularity of the source data affect survey simulation alignment. We find that personas induced from out-of-domain sources rarely outperform simulations conditioned only on basic demographic information, largely due to population mismatch. However, when personas are accurately assigned to the target demographic groups, alignment improves substantially. Finally, personas induced from target-domain survey data generalize better as more survey question history becomes available, suggesting that richer behavioral evidence enables more stable persona trait inference that transfers to better unseen questions simulation alignment.

0.42

Personalized Alignment of Language Models with Human Value Profiles

cs.HC cs.AI ·2026-09-01 ·Rashon Poole, Farnaz Jahanbakhsh
top venue similarity 0.73
Abstract · Accepted to EMNLP 2026 Main

Large language models are increasingly used for subjective tasks such as seeking advice, where there is often no single correct answer. Yet dominant alignment methods learn a single behavior policy that applies uniformly across users, limiting their ability to reflect individual value priorities. We propose personalized value alignment, a framework for adapting model responses to users' stable value profiles rather than to personas, demographics, or task-specific preferences used in prior personalization work. Our approach represents users through Schwartz value profiles and selects which of their values are relevant to each query before applying targeted prompting, activation steering, or LoRA-DPO. We create a dataset of 1,501 value-tension scenarios and evaluate over 152 diverse value profiles sampled from a nationally representative survey. Value-targeted methods outperform no-personalization, demographic, and full-profile prompting baselines, indicating that gains come from focused conditioning on query-specific value priorities rather than mere exposure. We further find that steering effects are not uniform across users: alignment gains vary by where a user's priorities fall in the value space. Combining targeted prompting with activation steering or LoRA-DPO yields the strongest personalization while keeping logical quality within adequate range.

0.40

Personas Differ from Native-Language Generation: Language Pathways Shape LLM Interpersonal Advice

cs.CL cs.AI ·2026-08-31 ·Jinhee Won, Xinlan Emily Hu
top venuehot direction similarity 0.72
Abstract · Accepted to EMNLP 2026 (Main Conference). 36 pages, 24 figures, 17 tables (9 pages main text; remainder references and appendices)

LLMs are increasingly used for interpersonal advice and as tools for studying social behavior across languages and cultures. A common shortcut for eliciting language- or culture-related variation is to ask a model to answer as a native speaker. We test whether this native-speaker persona reproduces the outputs obtained when models instead generate advice in the target language and translate the response back into English. Using 600 interpersonal advice questions across 13 languages and eight LLMs, we compare native-language generation followed by translation (NL) with native-speaker persona prompting (NP), measuring linguistic style, behavioral scaffolding, and forced-choice action recommendations. We find that NP and NL are not interchangeable. Compared to NL, NP often increases lexical social cues, including affiliation and positive tone, while reducing qualities such as concreteness and social attunement; NP also provides less actionable scaffolding in open-ended advice. In forced-choice scenarios, NP changes which action the model selects, favoring confrontation over redirection, with effect sizes varying across languages, topics, and models. Our results show that cross-lingual elicitation strategy is a consequential methodological choice that can change both how advice is framed and which actions models recommend.

0.55

Behavioral History Outperforms Descriptions of the Person for LLM Synthetic Personas

cs.AI cs.CLcs.CY ·2026-10-02 ·Khashayar Pourtaheri, Ahmad Zareei
benchmark similarity 0.70
Abstract

Large language models (LLMs) are increasingly used as synthetic personas representing survey respondents. Their validity as substitutes for particular respondents depends on whether they reproduce individuals' decisions. We examine what information helps synthetic respondents predict each individual's later choices, using five conditions that add progressively richer information: no personal information, demographics, personality traits, cognitive scores, and finally the respondent's earlier survey choices as behavioral history. We use a two-wave panel of 845 US adults who completed measures of 14 behavioral biases (spanning risk, time preferences, overconfidence, and reasoning), so each respondent's earlier answers provide a human test-retest benchmark; in the behavioral-history condition, all items that score the target bias are withheld. At the population level, the average number of biases per respondent in every condition is close to the human average (7.1-8.1 biases, against 7.1 for humans). This aggregate similarity masks differences in variance: persona descriptions recover only 53-67% of human between-person variation, whereas adding behavioral history restores it to approximately the human level. At the individual level, description-based personas achieve only 7-12% of the informedness observed in human test-retest responses, while adding behavioral history raises this to 28%. The condition including behavioral history has the highest estimated informedness in all 17 demographic groups, whereas description-based conditions provide little or no information for some groups. Synthetic responses also exhibit stronger education- and income-related differences than human responses. For LLM synthetic personas, a respondent's past answers add more to individual-level prediction than a description of who they are.

0.37

Population Fidelity: Evaluating Population Representativeness in LLMs

cs.CL cs.AIcs.CY ·2026-09-28 ·Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang +3 more
positioncode/data similarity 0.70
Abstract · 37 pages, 16 figures, 14 tables. Code and data: https://github.com/CriticalMaking/LLM-population-fidelity

Large language models (LLMs) show considerable potential in simulating human attitudes and preferences. Prior work finds that LLM-generated responses can compress the range of attitudes found within populations and misrepresent particular subgroups in ways that vary across models and topics. We introduce Population Fidelity, an evaluation framework that distinguishes key conditions required for a set of LLM-generated responses to represent a population. It incorporates three dimensions: group-level accuracy, the amount of between-group variation, and the structure of that variation. We demonstrate the framework's utility in two ways. First, we reproduce a prior study of "machine bias" in LLM survey responses and apply the framework to its models and more recent ones, showing that poor representation reflects not only insufficient between-group variation but also variation assigned to the wrong groups. Second, we evaluate one proposed approach to improving models' population representativeness: cultural fine-tuning. We find that cultural fine-tuning can improve alignment with the survey center without improving the representation of within-population differences, a distinction that measures of aggregate agreement do not capture. We argue that representing a population requires models to reproduce several features of human attitudinal variation simultaneously. Our framework organizes these features and provides reusable code, data, and trained models for evaluating population fidelity across substantive domains and assessing proposed alignment methods.

0.31

When Persona Simulations Are Informative: Graph-Structured Signals for Pluralistic Opinion Sensing

cs.AI cs.CY ·2026-08-23 ·Taehyeon An, Jaehyeong Park, Donghyuk Shin
top venueposition similarity 0.70
Abstract

Persona-conditioned large language models (LLMs) are increasingly used to simulate survey responses across diverse domains. However, apparent response variation can reflect unconditioned model priors or token sampling noise rather than systematic persona conditioning. We argue that persona-conditioned variation is informative when semantically similar personas exhibit concordant response shifts. To operationalize this principle, we introduce Persona-Conditioned Informativeness (PCI), an unsupervised diagnostic metric that measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines. By modeling personas as a similarity graph, PCI uses Local Moran's I to quantify local spatial coherence and extract compact persona subsets without using construct labels. To evaluate PCI without external human benchmarks, we test its ability to recover established latent value structure using the 57-item Portrait Values Questionnaire-Revised (PVQ-RR). Confirmatory factor analysis (CFA) shows that a PCI-selected 10% subset substantially improves overall construct recovery relative to response-stability and random selection. These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.

0.35

Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

cs.CL ·2026-09-05 ·Qishuai Zhong, Zongmin Li, Siqi Fan, Aixin Sun
top venue similarity 0.69
Abstract · EMNLP 2026 Findings

Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.

0.42

Measurement Validity in LLM Cultural Alignment

cs.AI physics.soc-ph ·2026-08-29 ·An Duy Nguyen, Muhammad Aurangzeb Ahmad
similarity 0.69
Abstract

Researchers increasingly treat LLM survey responses as a proxy for human cultural values. This includes projecting model outputs onto instruments like the Inglehart-Welzel Cultural Map and drawing conclusions about which cultures a model resembles. While a model's answer to a value-laden questions may be interpreted as a cultural signal, it also carries sampling noise and, can be quite sensitive to question framing. In this paper, we separate survey responses, sampling noise and question framing for multiple LLMs. We decompose response variance from these models into variation across random seeds, prompt rewordings. We employ noise-to-signal ratio (NSR) to test whether a model's apparent cultural position is distinguishable from noise. When applied across a dozen models from four geographic origins, calibrated against 88 Integrated Values Survey countries, the answer is often no. NSR exceeds 1.0 on 49 of 117 valid model-question pairs (42%), reaching 5.56 in the worst case. Two models even refuse to answer sufficient number of survey questions outright. Our results corroborate previous findings that LLMs cluster toward Western, English-speaking cultural positions. However, what does not hold up in this study is the precision with which anyone can currently interpret a specific model's coordinates: prompt tone alone can shift a model by 2.4 map units, comparable to the distance between actual countries in the Inglehart-Welzel Cultural Map. These findings suggest that cultural attribution from LLM survey responses requires establishing the reliability of the underlying measurements before interpreting model coordinates as evidence of cultural representation.

0.35