CS Paper Radar

Acceptable Misalignment

Maud van Lier, Ludovica Marinucci, Vieri Giuliano Santucci, Raja Chatila, Stéphane Doncieux, Émiland Garrabé, Richard Duro

cs.AI cs.RO ·submitted 2026-10-12 14:42 UTC · v1
position

Abstract

There is a specific challenge designers face in developing autonomous robots capable of open-ended learning. This challenge is how one can develop such systems in an ethically sound way when there are many unknowns during design time about their manner of deployment, what they can and will learn, and how these systems will act on the basis of what they have learned. One response to this challenge is to align, as far as possible, the activities and goals of autonomous robots with human values, raising the question of how to make this alignment possible, a question known as the value alignment problem. We argue that the value alignment problem faces a number of challenges that show that full alignment is both theoretically and practically impossible, meaning that misalignment incidents will keep occurring. This undermines current alignment approaches, as the attempt to maximize alignment cannot properly deal with the possibility that the risk of misalignment that remains after alignment methods have been implemented might still be unacceptable to society. To overcome this difficulty, we argue that a shift in perspective is required from a focus on the maximization of alignment to the conditions under which misalignment incidents become acceptable. In this paper, we explore what this shift in perspective entails. We introduce the concept of acceptable misalignment, formulate normative requirements for alignment mechanisms aimed at bringing misalignment incidents to an acceptable level, and we conclude by presenting a new methodological framework that consists of alignment mechanisms that can meet these requirements.

Comment
16 pages, 1 figure

Closest papers

Nearest neighbours by meaning across the whole index.

AI Alignment Is Neither Solvable nor Sufficient

cs.AI ·2026-11-09 ·Gerrit Großmann
position similarity 0.67
Abstract

Humans and AI systems increasingly act within the same social and economic environments. Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. We argue that this endeavor is misguided. First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable. Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.

0.18

SPEAR: Five Principles for Interactive Human-Agent Alignment

cs.HC cs.AIcs.MA ·2026-10-05 ·Tao Long, Lydia B. Chilton
positionhot direction similarity 0.62
Abstract · 3 pages. Best Talk Award at the ACM Conference on Human-AI Complementarity and Alignment (HCOMP 2026)

Recent AI alignment work often frames alignment as a pre-deployment optimization problem: collect human feedback, learn preferences or principles, finetune the model, and deploy an aligned system. This framing has produced major progress, but it under-specifies what happens once AI systems act as agents on users' behalf in situated, long-term, and social contexts. This position paper reframes human-agent alignment as an ongoing interaction design problem. We propose SPEAR, five pillars of interactive alignment: Specification (how people express intent and establish shared understanding), Process (how agents decide when to act, ask, defer, or pause), Evaluation (how people judge whether agents succeeded), Adaptation (how agents adapt to users over repeated use), and Recalibration (how people adapt their trust, expectations, and behavior in response to agents).

0.51

Toward a Theory of Value in AI Alignment

cs.AI ·2026-08-10 ·Andrew Smart, Shazeda Ahmed, Jackie Kay, Jimmy Tobin, Kris Shrishak +1 more
similarity 0.62
Abstract · 15 pages, 2 figures

Can AI systems be aligned to human values? The popularization of large language models (LLMs) and multi-modal foundation models has seen a rise in harms spanning from toxic speech and hallucinations to AI agents executing unauthorized actions. Within the field of AI safety, these harmful instances are often framed as the alignment problem, or of models being misaligned with human values. Researchers have responded by pursuing applied and theoretical AI value alignment efforts, often without specifying what they mean by human values. How does the field of AI value alignment conceive of human values? How are these conceptions of values technically operationalized and evaluated? What does the emergent theory of value from this field signify for the future of AI? We annotated 94 value alignment research papers to discern their implicit theory of values in AI. The majority do not define values, relying heavily on preferences as a stand in that runs the risk of reducing complex culturally situated concepts down to binary choices. As researchers dispense with using human annotators for model training and evaluation, turning instead to synthetic data and autorater approaches to aligning and evaluating models, we identify the potential to close off alternative methods for contesting and enacting values in foundation models. In making AI value alignments philosophical commitments explicit, we seek to bring great specificity and under explored perspectives in the debate on whether and how AI can address human values.

0.44

Alignment Games: A Framework for Conceptual Repair in Human-AI Collaboration

cs.HC cs.AI ·2026-09-28 ·Hari Subramonyam, Maneesh Agrawala, Sean Follmer
similarity 0.61
Abstract

The meaning of a concept in use is shaped by the situation, task, goals, and prior knowledge. For example, a request to make a poster "visually appealing for a five-year-old" might evoke bright colors and cartoon imagery for one collaborator, but less text, bold shapes, and visual simplicity for another. We call such task-relevant differences conceptual misalignment. We introduce Alignment Games, a framework for making these differences visible and repairable during human-AI interaction. Drawing on theories of situated conceptualization, we characterize task-specific conceptual frames in terms of relevant attributes, values, relations, constraints, and priorities. We then define alignment moves that intervene on the situation, the reasoning used to interpret it, or the resulting frame. Through examples from educational content generation, creative coding, and argumentative writing, we show how these moves can be composed into repair sequences and derive design principles for supporting task-sufficient conceptual alignment at runtime.

0.34

From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes

cs.RO cs.AIcs.HC ·2026-10-26 ·Diana Cerdas-Vargas, Alejandro Romero, Richard J. Duro
similarity 0.59
Abstract

This work explores a multimodal approach to the alignment of the behavior of open-ended learning autonomous robots with human purposes through the use of large language models as cultural enablers. While autonomous robots can generate their own skills and goals through interaction with the environment, they often lack mechanisms to ensure that these remain relevant and aligned to those of human users. The proposed approach makes use of the general and cultural knowledge contained in LLMs and VLMs to establish a multimodal interactive alignment process in which human purposes, expressed in natural language, are transformed into motivational components of a robot architecture in the form of robot missions and drives that guide the behavior of the robot. The system is structured into three layers for semantic alignment, mission generation, and drive modeling. This design enables a bidirectional interaction loop: human inputs define high-level purposes, robots expand them into executable missions making use of multimodal information relayed by LLMs and VLMs, and feedback maintains alignment with human intent. Experiments in simulation and with a physical robot demonstrate that the process produces valid missions and behaviors, supporting effective human-agent alignment in interactive scenarios with autonomous robots.

0.12

Automated Researchers Can Mitigate Well-characterized Alignment Failures

cs.AI cs.CL ·2026-08-28 ·Chen Yueh-Han, Jiaxin Wen, Jan Hendrik Kirchner
benchmarkhighly cited similarity 0.59
Abstract

Automating alignment research may accelerate progress toward aligned AI, but whether it does is hard to measure. Luckily, many alignment failures, such as deception, sycophancy, and jailbreaks, are already measurable by public benchmarks. We study whether automated alignment researchers (AARs) can post-train to mitigate alignment failures by proposing training methods and data to simultaneously optimize multiple safety benchmarks, while largely preserving general capability. Across 10 alignment failures, the strongest AAR methods significantly reduce the targeted alignment failures and generalize to a held-out benchmark, multi-turn behavioral audits, and models up to 4.7x larger than the target model. As a human baseline, 28 experienced researchers receive up to eight hours to develop one-shot methods for the same benchmarks, but their methods underperform the best AAR methods. Using human ideas as the AARs' initial research direction does not improve performance, suggesting current AARs may not need guidance from experienced researchers. These results suggest that automating alignment research on well-characterized failures may be practical in the near term.

0.48

Thick Alignment: A Social Framework for Values and AI Systems

cs.CY ·2026-11-04 ·Alondra Nelson, Sola Kim
position similarity 0.58
Abstract

Computer science research on the "alignment" of artificial intelligence (AI) systems treats human values as content to be specified and optimized. Decades of social science scholarship describe values differently, as enacted in ordinary practice, interpreted in social context, and contested across plural orders of evaluation and worth. We argue that the dominant alignment techniques, which we call "thin alignment," convert situated moral or value judgments into one or a few metrics via devices of commensuration such as preference labeling, reward modeling, and consumer ratings. Drawing on philosophical and anthropological accounts of thickness, "thick alignment" instead examines how values take shape, inform the development of AI systems, and become subject to contestation over whose interests they serve. We describe two forms of misalignment in seemingly aligned algorithmic systems that highlight thin alignment: "Halcyon misalignment," which presumes a prior, just equilibrium of technology and values from which a system has departed, and "Mirrored misalignment," which names the risks and harms produced by systems that are aligned in a thin sense yet accurately reproduce an unequal social order. A thick account registers such systems as misaligned. We then trace how researchers have taken up and adapted the concept of thick alignment since it was first proposed in 2023, using these adaptations to refine the concept, and to distill three of its pillars: (1) contextual description that both reflects and constitutes values; (2) interrogation of values across the life cycle of a system; and (3) shared, accountable decision-making through which affected publics help determine which values guide the system. We close by considering implications for AI research and policy.

0.15

Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant Robots

cs.HC cs.AI ·2026-11-13 ·Mateusz Wozniak, Qi Gao, Agnieszka Wykowska
publishedbenchmark similarity 0.58
Abstract · 18 pages, 6 figures. Version accepted for publication in: Value Engineering in Artificial Intelligence: Fourth Inter-national Workshop, VALE…

Personal assistant robots powered by large language models (LLMs) are likely to enter everyday life within the next decade, but the values that should guide their behaviour remain largely unspecified. Current alignment practice steers a single model toward an aggregate of human values, ignoring that individual users may prefer a robot whose value profile differs from their own. We administer parallel self- and robot-targeted versions of three established psycho-metric instruments - the simplified Schwartz Values Survey (SVS), the Portrait Values Questionnaire (PVQ-21), and the Moral Foundations Questionnaire (MFQ) - to potential future users (N=22), capturing both their personal values and the values they would want an assistant robot to embody. We compare the three instruments as personalization tools using the UEQ-S and benchmark fifteen contemporary LLMs against the resulting profiles. Participants wanted their robot to score markedly lower than themselves on personal-focus values and higher on social-focus values; moral profiles were nearly identical. LLMs prompted to act as robots shifted only weakly toward the desired profile, indicating that default model behaviour and prompt engineering are no substitute for explicit value personalization.

0.44