CS Paper Radar

Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models

Axel Abels, Elias Fernández Domingos, Tom Lenaerts

cs.AI cs.CLcs.GT ·submitted 2026-10-24 13:06 UTC · v1

Abstract

Do preferences Large Language Models (LLMs) express in standard elicitation tasks predict their strategic behaviour? LLM agents interacting with humans and other AI systems face potentially conflicting objectives, such as individual welfare, group welfare, and environmental concerns. Prior work characterizes LLM preferences through behavioral-economic and psychological elicitation or strategic games, but rarely tests consistency across settings and prediction in other strategic environments. This requires modeling how competing objectives jointly shape behavior, whereas existing approaches typically measure preference dimensions in isolation. Drawing on utility theory, we introduce the Preference-Behavior Model (PBM), which combines a preference representation with mechanisms capturing imperfect utility maximization. We formulate estimation as an inverse-game problem, jointly inferring preferences and behavioral parameters from observed choices. We evaluate PBM in three social dilemmas that expose conflicting motives: a Public Goods Game, a Collective-Risk Dilemma, and a Common-Pool Resource dilemma. We find that while preferences estimated from established behavioral economic elicitation instruments do not consistently predict behavior in these dilemmas, models fitted in one dilemma retain predictive value in others, suggesting that inferred preferences capture meaningful aspects of LLM decision-making that elicitation instruments may miss. Finally, we assess alignment with persona instructions, finding that inferred preferences broadly reflect intended priorities with specific divergences across dimensions, LLMs, and games being identified by our approach. Our methodology provides a framework for studying how consistently LLM preferences explain behavior, assessing alignment with intended priorities, and revealing trade-offs and behavioral patterns that standard elicitation may miss.

Closest papers

Nearest neighbours by meaning across the whole index.

Exploit or conform? How norm utility prompting changes the way LLMs use community beliefs

cs.AI ·2026-11-02 ·Leon Klingborg, Theodor Cimpeanu, Alexander J. Stewart
similarity 0.67
Abstract

Large Language Models (LLMs) serve both as tools for aiding human decision-making in everyday life and as models of human agents in experimental simulation studies. Alignment on prototypical social decisions -- such as whether to cooperate on a collective action -- therefore has immediate practical consequences for researchers and for society at large. Here we systematically explore the impact of ``norm utility'' -- the non-material components of utility that account for an individual's desire to conform to social norms and to be internally consistent -- on the strategic decision-making of three open-weight LLMs in one-shot, two-player social dilemmas. We find that when a social dilemma is presented together with information about community beliefs -- but without any indication of how those beliefs should matter -- models tend to treat community belief as strategic information, so that in the Snowdrift game cooperation falls as community belief in cooperation increases. When we explicitly prompt the components of norm utility this relationship inverts, and cooperation increases with community belief, in broad agreement with the predictions of a simple norm utility model. We then show that prompting norm utility reduces the volatility of strategic decisions with respect to changes in the underlying social context, and generates more consistent reasoning habits across models. Taken together, our results suggest that prompting norm utility can help generate more human-like and consistent behavior across models and lower volatility within models.

0.25

Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation

cs.GT cs.AIcs.CLcs.MA ·2026-08-12 ·Akash Kundu, Emanuel Tewolde, Ratip Emin Berker, Samuel F. Brown, Vincent Conitzer
similarity 0.67
Abstract · 41 pages, 18 Figures, 4 Tables, 16 Listings

As LLM-based agents with user-instructed goals are becoming widely deployed, they increasingly encounter each other in strategic interactions, and face challenges of finding mutually beneficial outcomes. Prior literature has argued that cooperation problems such as the Prisoner's Dilemma are resolvable in settings where agents know they follow very similar decision making patterns, as for example in monocultural AI ecosystems. Following that line of work, this paper introduces the first framework for evaluating LLM decision making when agents are provided with graded similarity signals. Among our findings, we establish that different LLM models vary drastically in how they navigate similarity signals, with some modern models showing consistent behavior across cooperation problems, payoff structures, and prompt framing. Perhaps surprisingly, our experiments also show that the dataset based on which the similarity signal is computed has small to no impact on induced cooperation, and that LLM models systematically self-identify as highly similar when asked to evaluate another model's chain-of-thought reasoning by themselves. Finally, we develop an LLM-behavioral-game-theoretic model that captures some of their reasoning rationale, and show that it can support cooperative outcomes in equilibrium under sufficiently high similarity scores.

0.10

Adaptive Personalization Under Preference Epistemic Uncertainty

cs.CL ·2026-10-29 ·Priyanka Dey, Rajarshi Das, Vikas Bahirwani, Qi Zeng, Julia Kruk +1 more
benchmark similarity 0.66
Abstract

Human preferences change over time: an established preference may remain valid over time, become uncertain as context shifts, or be explicitly revised. Effective personalization requires conversational AI to adapt its response strategy to evolving preferences rather than relying on passive memory retrieval. We formalize this capability as adaptive personalization under epistemic uncertainty. A human preference study further confirms that people favor state-contingent response strategies as preference uncertainty changes. To evaluate whether language models mirror this adaptation, we create a multilingual benchmark spanning 250 personas, 5 languages, and 25K conversational turns that decouples strategy selection from response execution. Across eight closed- and open-weight models, we uncover a recognition-generation gap: models reliably identify appropriate strategies when selecting among candidate options, but collapse during open-ended generation, suffering a 35-46% drop in strategy calibration. Crucially, this failure is systematic: models default to exploit-first behavior, over-relying on existing preferences regardless of preference epistemic uncertainty. We show this failure is addressable: while prompting interventions fail, targeted training that supervises models' internal strategy representations toward state-conditional human preferences narrows the user acceptability generalization gap by a third on out-of-distribution personas and conversational styles.

0.12

Learning Heterogeneous Preferences

cs.AI cs.HCcs.LG ·2026-09-15 ·Shiwali Mohan, Matt Hong, Dule Shu, Aniek Fransen, Shabnam Hakimi +1 more
similarity 0.66
Abstract

Learning from human feedback has become a central paradigm for training modern AI systems, where models of human utility are used as reward models in policy learning. Existing methods typically assume a \emph{universal utility} function shared across a population and treat disagreement between annotators as stochastic variation. While suitable for objective tasks, this assumption breaks down in subjective domains where preferences vary systematically across individuals. We study the problem of subjective preference learning, in which observed choices arise from heterogeneous but internally consistent utility functions. Drawing upon rational choice theory, RCT \parencite{tversky1981framing}, we introduce \emph{individuated utility} functions conditioned on both the individual and their decision context, and propose a novel multi-stage architecture for estimating them from multi-modal data. We evaluate our framework on a newly collected dataset of more than $575{,}000$ pairwise aesthetic judgments from $2{,}398$ participants comparing automotive wheel designs. Our experiments show that individuated utility models substantially outperform universal utility models including foundation model baselines. Our results demonstrate that disagreement reflects meaningful preference heterogeneity rather than annotation noise. More broadly, our findings highlight the importance of collecting annotator attributes and learning individuated utility functions, enabling reward models that explicitly account for whose preferences they represent and faithfully capture human decision diversity.

0.17

Conditioning LLMs on Social Value Orientation improves behavioural alignment in a sequential social dilemma

cs.GT ·2026-10-01 ·Marco Saponara, Axel Abels, Ann Nowé, Tom Lenaerts
similarity 0.65
Abstract

Large language Models (LLMs) are increasingly used to simulate human decision-making, yet their outputs often under-represent human behavioural heterogeneity. We investigate whether conditioning LLMs on Social Value Orientation (SVO, a measure of how individuals value their own outcomes relative to others') can better reproduce human behaviour in a sequential social dilemma. Using experimental data from two variants of the Centipede Game (CG) as reference, we compare the default behaviour of eight LLMs with behaviour generated after conditioning them on SVO profiles drawn from the human sample. We find that the default strategies vary substantially across models, but are generally distant from the reference distributions. However, conditioning them on SVO profiles systematically steers their strategic behaviour, improving alignment with the human reference by up to 70% with respect to the default. Across models, higher induced SVO values decrease the probability of stopping the game, reproducing the relationship between prosociality and cooperation observed in human behaviour. Together with the sensitivity of LLMs' elicited SVO to prompt and order effects, these results suggest that the usefulness of SVO for behavioural simulation does not depend on LLMs possessing stable social preferences, but rather on their ability to map social preferences onto corresponding strategic choices.

0.45

A data-driven heuristic-ensemble method for behavioural inference in collective-risk dilemma experiments

cs.CY physics.soc-phphysics.data-an ·2026-10-09 ·Marc Sadurní, Martín F. Díaz, Josep Perelló, Miquel Montero
similarity 0.65
Abstract · 21 pages, 5 figures, 2 tables, methodological article

The collective-risk dilemma has become a paradigm for studying cooperation in a wide variety of social experiments, initiatives that generate increasingly rich datasets describing individual actions over successive decision-making processes. Beyond the purely descriptive analysis of such empirical datasets, a methodological framework is required to interpret and reproduce the observed dynamics, a scheme that may be skewed by exogenous biases. Here, we introduce a data-driven, non-parametric method for simulating individual decisions together with a validation protocol for assessing the fidelity level of the outcomes they generate. Our methodological contribution is based on an empirical heuristic ensemble: agents are represented by a finite set of behavioural heuristics, probabilities conditioned on different observable features of the game state, endowed with a hard-voting arbitration that determines their per-round final decisions. This reproducible generative framework allows for behavioural inference in different collective-risk dilemma setups: Behavioural heuristics are seeded directly from empirical data assessing conditional probabilities, without a priori assumptions; a plurality vote with uniform random tie-breaking resolves between competing heuristics each round, adding no free parameters; validation follows a pattern-oriented approach, comparing synthetic and empirical macro-indicators against single-heuristic and uniform-random baselines; synthetic data generated allow exploration of scenarios or experimental setups that have not been empirically realised; as a worked example, we instantiate our methodological approach on the xAire citizen science collective-risk dilemma experiment run in Barcelona.

0.31

Preference Reasoning under Indeterminacy in Large Language Models

cs.AI cs.GTcs.LG ·2026-08-19 ·Hadi Hosseini, Samarth Khanna, Xiyuan Wang
position similarity 0.65
Abstract · 55 pages, 14 figures

As large language models evolve into decision-making agents, the ability to reason over preferences becomes fundamental to alignment, coordination, and collective intelligence. Yet, unlike standard benchmarks, real-world preference reasoning is inherently indeterminate: information may be incomplete, and valid solutions may not exist. We argue that indeterminacy, rather than correctness alone, is a central challenge for AI reasoning. We formalize this challenge along two axes, (i) epistemic indeterminacy, arising from incomplete, partial, or expressive preferences, and (ii) structural indeterminacy, arising from the non-existence of solutions under standard social choice concepts. Across a hierarchy of tasks, we show that state-of-the-art language models systematically fail to distinguish between determined and undetermined instances, exhibiting miscalibrated reasoning even in verification settings.

0.24

Modeling Human Behavior with Type Vectors Using AI

cs.AI econ.TH ·2026-08-18 ·Matthew O. Jackson, Benjamin S. Manning, Yutong Xie, Walter Yuan, Qiaozhu Mei
similarity 0.64
Abstract

We introduce a general, easy-to-implement AI-based modeling technique for analyzing human behavior. A key feature of this approach, which contrasts with existing modeling techniques, is that it combines the flexibility and interpretability of natural language with a mathematical structure that can be fitted to data and easily analyzed. We assign a large language model a vector of trait intensities-a type vector-and then ask it to choose actions across settings in which we observe human choices. For instance, the type vector (2,4) could correspond to "You are a player characterized by the following profile: Altruism: 2 out of 5, Risk Aversion: 4 out of 5," after which it is asked to make choices. We can then vary the traits (e.g., Altruism, Fairness, Trust,...) and values (e.g., 1-5) to minimize distance to human choices. We illustrate the method by applying it to model 119,147 decisions made by 78,657 subjects from more than 35 countries across 10 classic economic game roles. We find that human behavior can be closely matched using three dimensions: Risk Aversion, Strategic Sophistication, and Trust. The type vectors needed to fit individuals across games cluster into fewer than a dozen groups, with substantial variation in fit across subjects. Moreover, the individual type vectors can predict behavior in held-out games with different rules and available actions. More broadly, this new modeling method is highly generalizable and interpretable: we can input any vector of traits and use them to model behavior across any setting

0.19