CS Paper Radar

Large Language Models Substantially Compress Well-Being Dispersion but Largely Preserve Its Socioeconomic Structure

Nattavudh Powdthavee

cs.AI ·submitted 2026-09-27 00:37 UTC · v2
unusual

Abstract

Research using large language models (LLMs) to generate synthetic populations shows that model outputs compress human variation. We ask whether this compression also distorts where dispersion is concentrated across social groups. Using 93,901 respondents from 66 countries and territories in Wave 7 of the World Values Survey, we compare observed life satisfaction with predictions from six LLMs. All six substantially understate overall dispersion. Yet after normalizing for these scale differences, they largely reproduce the human income gradient in well-being inequality: lower-income groups remain relatively more heterogeneous than higher-income groups. The pattern is robust to country fixed effects and alternative weighting schemes, and extends directionally to employment, education, and perceived control. In a separately preregistered mechanism experiment using Claude Sonnet 5, counterfactually increasing income strongly raises predicted mean life satisfaction but produces essentially no linear change in predicted dispersion. Removing income from the respondent profile leaves the negative dispersion gradient almost unchanged; the same pattern appears with Gemini and DeepSeek, although somewhat attenuated. These findings show that LLMs can substantially compress well-being dispersion while preserving much of its socioeconomic structure, and that this structure does not depend on the explicit income cue alone.

Comment
28 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

The Asymmetric Harms of LLM Compression

cs.CL ·2026-08-20 ·Yuan Wu, Mairui Li, Lesia Semenova, Chudi Zhong
similarity 0.53
Abstract

Large language models (LLMs) compression reduces deployment costs, but standard aggregate metrics like perplexity and accuracy often mask underlying behavioral shifts. In this work, we systematically evaluate 3 LLMs across 11 compression methods to investigate the effects of compression on knowledge retention, model confidence, and social bias. We find that compression disproportionately reduces the relative retention of head knowledge compared to tail knowledge. Furthermore, compressed models often remain substantially confident in their incorrect answers on newly lost knowledge. Finally, we demonstrate that stable aggregate bias scores can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups. Together, these findings reveal asymmetric behavioral changes that aggregate performance measures fail to capture, highlighting the need for granular evaluation of compressed models before deployment.

0.37

Online and Survey Representations of Well-Being: Coherence and Cross-Source Correspondence

cs.LG econ.EMphysics.soc-phstat.AP ·2026-09-29 ·Stefano M. Iacus, Giuseppe Porro, Haodong Qi
similarity 0.51
Abstract

Digital traces may complement population surveys in the measurement of well-being, but the two sources do not necessarily capture identical constructs. We compare the Human Flourishing Geographic Index (HFGI) with World Values Survey and European Social Survey measures across three domains using structural equation models. The results show strong within-source coherence and construct-specific cross-source correspondence. Cross-source convergence varies across domains; the observed pattern is consistent with stronger correspondence where the underlying indicators are more closely aligned conceptually. Near-perfect agreement would be unexpected, since surveys elicit standardized self-reports, whereas HFGI captures naturally occurring public expressions. HFGI should therefore be viewed as a distinct but empirically connected measurement system that complements survey-based evidence on well-being.

0.23

HouseholdBench: Evaluating Large Language Models as Predictors of Household Economic Behavior

cs.CL ·2026-10-06 ·Jin Huang, Diego Ferreras Garrucho, Yutong Xie, Walter M. Yuan, Qiaozhu Mei +2 more
benchmark similarity 0.50
Abstract

Large language models (LLMs) have the potential to meet a key goal in economics: a quantitative model of household decision making, across a variety of settings. Yet existing evaluations cover few surveys and outcomes, and do not study how households adjust to changing economic conditions. We introduce a new evaluation, HouseholdBench, which unites 6 U.S. household surveys and 32 prediction tasks spanning numeric, categorical and probabilistic outcomes, related to consumption, income, labor, expectations, and housing. Using past behavior, demographics and macroeconomic conditions, the tasks test whether LLMs predict behavior, including how households adjust to changes in various policies. We evaluate 13 proprietary and open-weight LLMs against a no-change baseline and a gradient-boosted tree model. Most LLMs outperform the no-change baseline, including for policy response tasks -- with the best model lowering error for numeric outcomes by 12.2%. Across most tasks, gradient-boosted trees rank first; leading proprietary LLMs approach their performance, but open-weight models lag. LLMs exhibit systematic over- and underprediction across different tasks. We identify methods that enable a 4 billion parameter open-weight model to match proprietary models' performance: fine-tuning and aggregating 16 predictions per observation. Improvements generalize to policy-response tasks, which are excluded from fine-tuning. We release our datasets, code, and leaderboard on our website: https://jn-huang.github.io/householdbench

0.31

Shared Autoregressive Context Can Distort Relationships in Synthetic Data

cs.LG cs.CL ·2026-09-26 ·Thomas S. Robinson
similarity 0.49
Abstract · 54 pages, 8 figures, including appendices

Large language models can generate several records within one autoregressive completion, making earlier answers available as context for later records. This paper shows that such shared-completion batching can distort relationships among variables in the resulting synthetic data, using controlled tests on synthetic survey respondents. In a matched experiment on 2,000 European Social Survey profiles, generating ten rather than one respondent per request increases mean absolute error in within-country correlations by 48-58% for Qwen3.8-27B and 114-127% for Llama-3.3-70B-Instruct across three seeds, holding profiles, examples, questions and decoding parameters fixed. The distortion primarily reflects exaggerated relationship strength, while retaining substantial agreement with the human ordering of correlations. Controlled interventions establish answer history as a causal channel: re-pairing the same preceding values, with profiles and marginal distributions fixed, changes correlations among subsequently generated responses. Hiding preceding answers reduces correlation error in the tested settings but worsens marginal accuracy. Exploratory corrections across social-attitude, health and economic data likewise show that lower correlation error can coexist with worse marginal distributions and regression estimates. Request construction is therefore part of the data-generating process, and synthetic-data validity must be evaluated against the analyses the generated data are intended to support.

0.29

Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations

cs.CL cs.AI ·2026-09-24 ·Yeeun Chae, Yewon Choi, Seunghyun Lee, IL Im
published similarity 0.49
Abstract · Accepted to the EMNLP 2026 Workshop on Pluralistic AI & NLP (PANDORA)

Large language models (LLMs) are increasingly used as synthetic survey respondents to estimate population response distributions. In cross-cultural survey simulation, evaluations should assess not only distributional fidelity within countries but also whether differences across countries are preserved. However, existing distance-based metrics such as Jensen--Shannon divergence (JSD) do not directly capture such cross-country differences. To address this limitation, we introduce Cultural Divergence Preservation (CDP), a reference-light diagnostic based on a one-time human calibration. CDP identifies reduced cross-country divergence as cultural flattening and increased divergence as cultural caricature. To evaluate CDP, we conduct experiments across four LLM backbones, three persona-based prompting methods, and two survey domains, the World Values Survey (WVS) and the Big Five Personality Test. The results reveal a systematic discrepancy between conventional fidelity metrics and CDP. Controlled experiments show that CDP changes monotonically as cross-country divergence is attenuated or amplified, while the corresponding changes in JSD remain relatively small. In our audit of real LLM generations, DeepPersona-Inspired prompting is frequently favored by conventional fidelity metrics but exhibits the strongest flattening in every model--domain block. CDP thus complements fidelity metrics by directly quantifying the attenuation or amplification of cross-country divergence.

0.47

Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

cs.CL ·2026-08-29 ·Andrew Aquilina, Xiang Lorraine Li, Yu-Ru Li
similarity 0.49
Abstract

Judgments about psychological distress are socially situated: what counts as concerning hinges on community norms around emotional expression, vulnerability, and help-seeking. Yet large language models (LLMs) used for distress detection are typically aligned to a single, undifferentiated standard. How well do these models capture the perspectives of the communities whose language they assess? We address this question through a perspectivist annotation study in which 321 participants provided 9,587 judgments on 1,198 Reddit posts spanning six identity-based communities, yielding community-specific labels. Raters in the contextualized in-group condition show a modest tendency to agree more with their community than uncontextualized out-group raters (OR = 1.18), an effect varying significantly across communities. We then evaluate nine open-weight LLM configurations and four frontier configurations against these labels. Open-weight LLMs systematically over-estimate distress: when communities perceive none-to-mild distress, these models achieve only 31-44% accuracy, predominantly producing false positives. GPT-5 and Gemini 2.5 Pro show the same none-to-mild inflation even when their full-sample over/under rates are mixed, while Claude Opus 4 is more conservative. This pattern does not simply mirror an outsider reading position: uncontextualized out-group human aggregates were nearly symmetric, with 18% over-estimation versus 19% under-estimation. Instead, the models that inflate none-to-mild cases exhibit a distress prior that exceeds both contextualized in-group and uncontextualized out-group human judgments. These findings have implications for equitable AI deployment in mental health contexts, where miscalibrated distress detection may unevenly affect the communities being assessed.

0.24

Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model

cs.CL ·2026-08-19 ·Fathin Difa Robbani
similarity 0.48
Abstract · v2: restructured into standard paper format (Method / Results I-II / Conclusion / lettered appendix); no changes to numbers or claims

Large language models are widely used to simulate survey respondents, yet their outputs are homogeneous and unfaithful to real inter-group differences, and whether this reflects what a model knows or uses has remained untested. Using representational similarity analysis against Pew American Trends Panel ground truth, we score demographic read-out locations in Mistral-7B and intervene causally across six attribute types. The internal geometry is faithful: attention-head read-outs dominate the standard residual read-out, reaching selection-corrected $ρ$ up to 0.63 -- about 70% of the measurement-reliability ceiling -- and one head, L11 H16, is significantly faithful across all six types, though race-based types stay weak and prompt-fragile, replicating in a second model family. Yet causal use does not track fidelity: the clearest causal pathway ($p=0.002$) sits in one of the least faithful types, the most faithful type shows no correction-surviving effect, and full identity swaps in the prompt move predictions by under 2% of their error. A 128-dimensional probe on that head lands 21-31% closer to survey truth than the model's answers, yet recovers almost none of the per-question group ordering. Readable, faithfully arranged, and causally used are three dissociable properties of the same model; treating them as one claim is what keeps the "can LLMs simulate populations" debate unresolved.

0.24

Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study

cs.AI ·2026-09-26 ·Grandee Lee, Wang Yue
benchmark similarity 0.48
Abstract

Large language models (LLMs) are increasingly used to generate synthetic survey respondents and digital twins of real people, but whether their output preserves real human statistical structure, rather than surface plausibility, remains unresolved, and most existing evidence comes from proprietary models rather than open-weight ones. We evaluate three open-weight LLM families on a cross-instrument calibration task: conditioning personas on real respondents' verbatim answers to one psychometric instrument and measuring them on a second, construct-distance-controlled instrument, checked against a 2,058-person human panel. Across a 139-pair grid, the simulated cross-instrument correlation tracks the real human correlation at r = 0.70 - 0.73 in every model, driven mainly by correct sign rather than precise magnitude and concentrated in pairs of moderate construct distance. A correlation of this magnitude, obtained from untuned open-weight models conditioned only on individual-level survey data, is a substantively encouraging result for LLM-based behavioral simulation and digital-twin applications: specific model families and releases already reproduce a meaningful share of real human cross-instrument structure without any fine-tuning. This capability does not, however, improve monotonically across model releases: on a matched panel, the newest of three tested Llama releases performs worst on two of three headline metrics, so realizing its promise in practice requires release-specific, distance-aware verification rather than a one-time benchmark.

0.17