CS Paper Radar

AI Alignment through a Game-theoretic Lens: A Survey

Yanan Cai, Zhongrui Zhao, Zhigang Lu, Ickjai Lee, Wei Emma Zhang, Minhui Xue, Yihong Zhang, Shuchao Pang, Wei Xiang

cs.AI cs.CLcs.GT ·submitted 2026-08-28 04:31 UTC · v2
top venuesurvey

Abstract

As large language models and increasingly capable AI agents are deployed in high-risk settings, aligning them with complex human values has become a central challenge. Existing alignment methods, while effective in improving helpfulness, harmlessness, and controllability, often struggle to capture real-world preferences that are context-dependent, non-transitive, and shaped by dynamic multi-party interactions. This survey reviews AI alignment through a game-theoretic lens. Specifically, it organizes recent progress around key game-theoretic elements and synthesizes the literature along three challenges: preference diversity, alignment priority, and temporal dynamics. This perspective clarifies where current alignment methods genuinely benefit from game-theoretic analysis, where the framework is looser, and what challenges remain in building robust, adaptive, and verifiable AI systems.

Comment
This paper has been accepted by EMNLP-2026 as a main conference paper

Closest papers

Nearest neighbours by meaning across the whole index.

AI Alignment Is Neither Solvable nor Sufficient

cs.AI ·2026-11-09 ·Gerrit Großmann
position similarity 0.66
Abstract

Humans and AI systems increasingly act within the same social and economic environments. Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. We argue that this endeavor is misguided. First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable. Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.

0.18

SPEAR: Five Principles for Interactive Human-Agent Alignment

cs.HC cs.AIcs.MA ·2026-10-05 ·Tao Long, Lydia B. Chilton
positionhot direction similarity 0.64
Abstract · 3 pages. Best Talk Award at the ACM Conference on Human-AI Complementarity and Alignment (HCOMP 2026)

Recent AI alignment work often frames alignment as a pre-deployment optimization problem: collect human feedback, learn preferences or principles, finetune the model, and deploy an aligned system. This framing has produced major progress, but it under-specifies what happens once AI systems act as agents on users' behalf in situated, long-term, and social contexts. This position paper reframes human-agent alignment as an ongoing interaction design problem. We propose SPEAR, five pillars of interactive alignment: Specification (how people express intent and establish shared understanding), Process (how agents decide when to act, ask, defer, or pause), Evaluation (how people judge whether agents succeeded), Adaptation (how agents adapt to users over repeated use), and Recalibration (how people adapt their trust, expectations, and behavior in response to agents).

0.51

Culture in the Code: Anthropological Concepts Decoding AI's Hidden Assumptions

cs.CY ·2026-10-09 ·Sonja Schmer-Galunder
published similarity 0.64
Abstract

Contemporary AI development operates on foundational assumptions about intelligence, rationality, and human values that reflect narrow cultural perspectives, limiting alternative conceptions of how AI can be globally beneficial. This paper introduces seven anthropological concepts -- origin story, monoculture, thick description, personhood and community, cargo cult, structuralism, and liminality -- to expose hidden assumptions and offer alternative frameworks for understanding AI alignment. The paper examines how AI systems embody Western-centric notions of rationality as utility optimization, personhood as bounded individuals, and values as quantifiable preferences extracted from behavior. The concepts surface three assumptions that current alignment debates rarely examine: that values can be inferred from behavior, quantified, and formalized; that such quantified values capture what actually matters; and that values, once defined with sufficient precision, are appropriate optimization targets. Against "thin alignment" -- optimization for behavioral proxies stripped of context -- the paper proposes directions toward "thick alignment": recognizing that values emerge through social practice rather than existing as extractable properties; defining more appropriate metrics that capture not only end goals but the conditions that lead to long-term flourishing; and moving from models of fixed identity toward relational ontologies where personhood emerges through social participation. The paper concludes that if AI systems are to create conditions conducive to human flourishing rather than optimize metrics that flatten social experiences, three shifts are needed: redistributing interpretive sovereignty to affected communities, protecting what resists optimization, and extending accountability across generations.

0.63

Toward a Theory of Value in AI Alignment

cs.AI ·2026-08-10 ·Andrew Smart, Shazeda Ahmed, Jackie Kay, Jimmy Tobin, Kris Shrishak +1 more
similarity 0.64
Abstract · 15 pages, 2 figures

Can AI systems be aligned to human values? The popularization of large language models (LLMs) and multi-modal foundation models has seen a rise in harms spanning from toxic speech and hallucinations to AI agents executing unauthorized actions. Within the field of AI safety, these harmful instances are often framed as the alignment problem, or of models being misaligned with human values. Researchers have responded by pursuing applied and theoretical AI value alignment efforts, often without specifying what they mean by human values. How does the field of AI value alignment conceive of human values? How are these conceptions of values technically operationalized and evaluated? What does the emergent theory of value from this field signify for the future of AI? We annotated 94 value alignment research papers to discern their implicit theory of values in AI. The majority do not define values, relying heavily on preferences as a stand in that runs the risk of reducing complex culturally situated concepts down to binary choices. As researchers dispense with using human annotators for model training and evaluation, turning instead to synthetic data and autorater approaches to aligning and evaluating models, we identify the potential to close off alternative methods for contesting and enacting values in foundation models. In making AI value alignments philosophical commitments explicit, we seek to bring great specificity and under explored perspectives in the debate on whether and how AI can address human values.

0.44

When Numbers Start Talking: Numerical Signalling and Strategic Behaviour Among LLMs

cs.AI ·2026-10-02 ·Alessio Buscemi, Daniele Proverbio, Alessandro Di Stefano, The Anh Han, German Castignani +1 more
similarity 0.62
Abstract

Large language model (LLM)-based agents increasingly operate in multi-agent systems (MAS) characterised by strategic interaction. However, little is known about whether, and to what extent, different types of messages affect the outcomes of strategic games. By investigating AI agents based on four popular LLMs, playing four games with different cooperation equilibria, we study whether messages of different kinds (natural language, numerical signals, or random sequences) significantly modify the levels of cooperation in each game, also depending on the agents' assigned personalities. We observe that structured messages alter the final payoffs for most games and LLMs, but without a predictable pattern; this challenges the assumption that AI agents can converge to stable equilibria regardless of additional capabilities. Moreover, we observe that agent-generated numerical messages depart from randomness, most strongly and consistently when agents are explicitly instructed to communicate; however, they introduce an additional interpretability challenge, as their symbol distributions are mostly associated with the payoff structure and typically become more concentrated with repetition, but are overall difficult for humans to interpret. Monitoring for coordination of AI agents through restricted channels should thus prioritise message-level fingerprints, which generalise across models, over behavioural decisions, which do not.

0.12

Can You Trust Your Preference Models? Stackelberg Games for Alignment

cs.AI cs.GT ·2026-10-10 ·Riccardo Formenti, Aditya Agrawal, Vikas Garg
similarity 0.61
Abstract

Language models are frequently aligned to human preferences via imperfect evaluators whose errors can be exploited during optimization. We train policies to search for comparisons on which an evaluator may reverse the underlying preference, providing candidates for additional supervision. Since human preferences are unavailable during policy training, we use disagreement among an ensemble of preference models as a proxy for prediction error. We formulate this search as a zero-sum Stackelberg game between two opposing policies, a Leader which tries to minimize disagreement, and a Follower which maximizes it. For a general class of finite contextual Stackelberg games, we establish equilibrium existence and, under regularization, uniqueness and a closed-form characterization. We introduce Stackelberg Mirror Descent-Ascent (SMIDA), a two-timescale algorithm with linear last-iterate convergence in the tabular setting, and develop a neural policy-gradient analogue. Experiments with a synthetic oracle support disagreement as an informative proxy for prediction error. On held out data the trained policies outperform reference policies and increase the fraction of comparisons that exceed a disagreement threshold and reverse the oracle's preference.

0.09

Alignment Myopia in Generative AI

cs.CY stat.MLcs.LG ·2026-11-04 ·Remi Denton, Aida Davani, Cynthia L Bennett, Michael Madaio, Roma Patel +1 more
similarity 0.61
Abstract

Current generative AI alignment relies on a myopic paradigm that prioritizes immediate conversational satisfaction. By relying on short-horizon feedback, this approach ignores the profound, longitudinal externalities of persistent human-AI interaction, which can lead to adverse outcomes such as cognitive atrophy, social erosion, and diminished human agency. To address this, we propose a paradigm shift toward long-horizon alignment that accounts for how AI reshapes human experience over time. To this end, we map the tensions between near-term utility and long-term costs of AI use across four facets of human well-being to reveal outcomes that current reward models fail to capture. Ultimately, we advocate for moving beyond simple preference optimization toward alignment strategies that actively foster long-term human well-being by navigating these complex temporal scales.

0.28

Correlated Quantal Response Equilibrium: A New Equilibrium Notion for Human-AI Strategic Interactions

cs.GT ·2026-10-21 ·Zehao Zhao, Rahul Jain, Vijay Subramanian
similarity 0.60
Abstract

The modeling of the behavior of agents in strategic environments and the prediction of the resulting outcomes has long been a challenging problem; this has been the focus of game theory. The introduction of AI agents as human-aids or as sole decision-makers expands the underlying question: Can we model behavior and predict outcomes of strategic games that involve a mix of AI agents and human players? As an initial answer, we propose Correlated Quantal Response Equilibrium (CQRE) that combines and generalizes classical solution concepts such as Quantal Response Equilibrium and Correlated Equilibrium. We answer questions of existence and propose a method for computing the CQRE that is a best fit to a given joint action distribution. We demonstrate the empirical effectiveness of CQRE predictions using a diverse collection of complex general-sum games, including high-rank and large-action two-player games, challenging multiplayer games, and a large-scale extension of the Selten--Chmura setting to AI--AI and human--AI play.

0.20