CS Paper Radar

When the Environment Becomes the Interface: Multisensory Environmental Interfaces for Human-AI Interaction in Autonomous Vehicles

Keqi Chen, Runjia Tan, Xinyi Fu, Shanhe Lou, Kwan Min Lee, Chen Lv

cs.HC ·submitted 2026-09-26 13:23 UTC · v1

Abstract

As AI increasingly assumes operational control, human-computer interaction is shifting from operating systems through explicit interfaces to inhabiting intelligent environments. This raises a fundamental question: when users no longer directly manipulate a system, what mediates their relationship with intelligent technologies? We introduce environmental interfaces: designed environmental conditions that shape human-AI relationships through ambient, holistic, and evaluative pathways rather than explicit functional interaction. Using autonomous vehicle cabins as a revealing context, we conducted a within-subject experiment with 24 participants (216 observations), manipulating lighting and scent in a simulated autonomous driving environment. Three findings emerged. First, perceived atmosphere accounted for 66.5% of the variance in overall journey experience, showing that environmental conditions can function as an interface rather than merely a supporting design element. Second, multisensory processing followed a hierarchical architecture: individual sensory appraisals were initially independent, while cross-modal integration emerged during higher-order environmental evaluation. Third, olfactory stimuli influenced affective responses and experience evaluation more strongly than visual stimuli, challenging the visual dominance of automotive interaction design. Environmental quality and sensory congruency also predicted trust in the autonomous system, suggesting that passengers may use environmental cues as proxy signals when direct assessment of AI competence is difficult. These findings establish environmental interfaces as a distinct interaction modality and point to a broader transition from designing interfaces for operating intelligent systems to designing environments for inhabiting them.

Closest papers

Nearest neighbours by meaning across the whole index.

User Experience in Human-Machine Interaction: Insights from Field Studies in Autonomous Mobility

cs.HC ·2026-08-31 ·Helen Schneider, Svetlana Pavlitska, J. Marius Zöllner
published similarity 0.62
Abstract · Accepted for publication at ABAW Workshop at ECCV2026

Autonomous vehicles (AVs) promise safer, cleaner, and more inclusive mobility, yet large-scale adoption is hindered by user acceptance rather than by technical challenges. Prior studies on acceptance and user experience largely rely on surveys, simulators or Wizard-of-Oz setups, often over-representing technologically enthusiastic participants and focusing on drivers instead of passengers. We address this gap with real-world field studies with AVs in real traffic, totaling 144 participants. Using multi-modal sensing, we evaluated EGG, heartbeat, breathing, camera and voice signals for affect inference in combination with vehicle data. Our results show that breathing, camera and voice measurements are reliable and pratical in naturalistic passenger contexts. We further contribute a validated study protocol, a self-assessment app for real-time assessment during human-machine interaction, and a tailored questionnaire to capture participant attitudes towards AVs. By grounding UX evaluation in real-world contexts, this work lays a foundation for user-centered design of autonomous mobility systems and robotics in general. Our work bridges the gap between affective computing and technical implementation of autonomous vehicles.

0.14

Exploring the Effects of Olfactory Cues and Ventilation on Teleportation-based Navigation in VR

cs.HC ·2026-09-26 ·Dongyun Han, Heecheol Kim, Siyeon Bak, Chang-Guen Song, Sun-Jeong Kim +1 more
published similarity 0.61
Abstract

Spatial cognition supports how people interpret spatial relationships and navigate their surroundings. Although vision plays a dominant role, other sensory modalities, including olfaction, may also contribute under limited visual conditions. This paper investigates the role of olfactory cues in location recognition during virtual reality (VR) navigation. We conducted two formal studies using a wearable olfactory prototype. Study 1 examined whether olfactory cues and ventilation influenced users' recognition of encountered locations during \HDY{teleportation-} and dash-based navigation. Study 2 extended this investigation to repeated navigation and examined whether ventilation duration influenced recognition performance and user experience across successive movements. Results show that olfactory cues can support recognition of encountered locations during navigation, while ventilation was associated with reduced residual interference and improved usability. However, longer ventilation did not clearly improve recognition accuracy in repeated navigation. These findings suggest the potential of olfactory cues as contextual signals during VR navigation, while also highlighting the importance of scent management for maintaining perceptual clarity and user comfort.

0.45

OdorSpaces: Visual-Olfactory Embodied Navigation in 3D Environments

cs.RO cs.AIcs.CVcs.LG ·2026-11-03 ·Changyeon Lee, Kelin Yu, Zheng Wei, Shuna Ni, Ruohan Gao
publishedbenchmark similarity 0.60
Abstract · Accepted at the 10th Conference on Robot Learning (CoRL 2026). 20 pages, 4 figures, 6 tables

Olfaction provides critical information about airborne chemical cues, yet it remains largely underexplored in embodied AI, with limited prior machine olfaction work focusing primarily on semantic odor perception. We introduce OdorSpaces, a physics-based simulation platform and benchmark for visual-olfactory embodied navigation. OdorSpaces augments Habitat-Sim with 3D odor transport and dispersion simulated by Fire Dynamics Simulator (FDS), enabling agents to navigate photorealistic indoor environments using both visual and olfactory observations. The benchmark includes two representative source-seeking scenarios: invisible gas-leak localization and early-stage fire localization. We develop an initial multimodal reinforcement learning policy that integrates visual observations and olfactory measurements over time. Experiments show that the policy outperforms a diverse set of baselines, establishing a strong reference for future research on visual-olfactory embodied navigation.

0.33

Smelling the Way: Olfactory Modulation of Spatial Estimation and Path Integration in Virtual Reality

cs.HC ·2026-09-26 ·Siyeon Bak, Junho Kim, Dongyun Han, Sun-Jeong Kim, Isaac Cho
published similarity 0.57
Abstract

Spatial cognition enables individuals to perceive and interpret spatial relationships, estimate locations, and navigate within their surroundings. While vision plays a dominant role, other sensory modalities, particularly olfaction, can support spatial processing when visual cues are limited. This paper explores the effects of scent-delivery within virtual reality (VR) environments by using fan-mediated olfactory devices. We conducted two formal studies to evaluate the effects on spatial information acquisition. Study 1 (N=24) examined the participants' ability to localize a scent source while walking a linear path using two different olfactory devices. Study 2 (N=19) employed a triangle completion task, in which participants attempted to return to their starting point under varying rotation angles and olfactory conditions. Results indicate that scent-delivery feedback influenced spatial behavior in distinct ways across the two studies. In Study 1, visual-olfactory misalignment produced systematic directional biases without improving absolute localization accuracy. In Study 2, distance-modulated scent-delivery feedback reduced terminal homing error but also increased traveled distance and completion time, suggesting a more deliberate goal-confirmation strategy.

0.41

Towards Informed Autonomy Design: Understanding the Interplay of Function Characteristics and Autonomy Preferences in Automotive Agents

cs.HC ·2026-11-14 ·Sebastian Zepf, Annika Stampf, Markus Schuster, Enrico Rukzio, Mark Colley
similarity 0.56
Abstract

Proactive, autonomous interactions receive high interest in the field of intelligent assistants, including the automotive domain. A central challenge, however, is identifying appropriate levels of autonomy for different functions while maintaining user acceptance and trust. This work examines how function characteristics relate to suitable autonomy levels and whether these relationships generalize across functions. We conducted an expert workshop to define autonomy levels and identify relevant function characteristics. We then collected data from 126 participants, who rated 14 in-vehicle functions regarding autonomy preferences and function characteristics, from which we trained machine learning models to predict appropriate autonomy levels. Building on our findings, we developed our prototypical Autonomy Design Assistant that supports designers in making autonomy-related decisions, which we evaluated qualitatively with nine experts. Our results suggest that function characteristics can rank functions by their suitable level of autonomy and thereby guide more informed autonomy design decisions, whereas predictions for individual users remain uncertain.

0.24

Occlusion-induced risk and interventions in pedestrian-autonomous truck interactions on multi-lane roads: A virtual reality study

cs.HC ·2026-08-30 ·Yun Ye, Yuan Che, S. C. Wong, Stergios-Aristoteles Mitoulis, Haoyang Liang
similarity 0.56
Abstract

Autonomous trucks (ATs) may introduce distinct pedestrian-safety risks because of their large physical dimensions, constrained braking capability, limited driver-based communication cues, and potential to occlude surrounding traffic. This study employed a controlled virtual reality experiment with 54 participants to investigate pedestrian-AT interaction risk in an unsignalized multi-lane crossing scenario and to evaluate occlusion-targeted risk mitigation strategies. The experiment examined the effects of near-side vehicle type, weather condition, and far-side vehicle yielding strategy on pedestrian behavior, perceived risk, and objective safety. Based on a representative high-risk scenario, three targeted interventions were designed and tested: an environment-aware external human-machine interface (eHMI), a projected eHMI, and an auditory warning. The results showed that ATs increased perceived risk and encouraged more cautious crossing behavior, suggesting a risk-compensation effect. However, this compensation was weakened under rainy conditions, where braking-related safety margins were reduced. AT-induced occlusion further increased far-side interaction risk by limiting pedestrians' recognition of hidden vehicles. Among the three interventions, the projected eHMI showed the best overall performance, improving objective safety margins, enhancing risk awareness, and supporting behavioral adjustment. These findings highlight the need for AT-specific interface and warning strategies that address both intention communication and risk localization.

0.23

Revisiting Direct Control: A Field Evaluation of Sensor Fusion Efficacy in Teleoperating Full-Scale Autonomous Vehicles via Bird's Eye View

cs.RO cs.HC ·2026-10-26 ·Stanisław Kuczma, Sławomir K. Tadeja, Michał R. Nowicki
published similarity 0.56
Abstract · Accepted at 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

In autonomous driving, robot teleoperation serves as a critical safety fallback mechanism, requiring operators to assume direct control in the event of system failure. While recent research has pivoted toward high-level supervisory control or simulation-based studies, these approaches often fail to capture the physical dynamics and sensory demands of driving a full-scale vehicle. To address this gap, we evaluated the impact of spatial data representation on operator performance using a real car-sized vehicle in a field experiment with $N = 18$ participants. We compared three visualization interfaces: (i) a standard camera feed baseline, with two bird's-eye views (BEV) from (ii) a raw LiDAR point cloud, and (iii) a sensor fusion of LiDAR and camera data streams. Contrary to prior passive observation studies suggesting that sensor-fusion BEV may hinder awareness, our active-control results demonstrate that such representation significantly outperforms the standard LiDAR BEV in perceived usability ($p \leq 0.0001$) and cognitive task load ($p = 0.01$, NASA-TLX score: 34.58/100) while also being significantly better than the Camera-only view ($p = 0.043$). The fused interface also led to statistically significant reductions in task completion time ($p = 0.0461$) and superior usability (SUS: 77.9/100) compared to those of the Camera-only and LiDAR-only views. By synthesizing objective performance data with comprehensive subjective assessments, we demonstrate that BEV fusion maximizes safety margins while minimizing the operator's mental demand. These findings challenge existing teleoperation conventions and provide an empirically validated approach for designing robust remote piloting systems.

0.29

Bring Buttons Back: Physical Interfaces for the Age of Automation

cs.HC ·2026-09-13 ·David Goedicke, Donald Degraen, Tom Igoe, David Sirkin, Wendy Ju +1 more
similarity 0.55
Abstract · 6 Pages, 7 figures, article

As AI starts to permeate everyday life, deployment focuses on embedding intelligence in the background, abstracting away controls, and leaving automated decision-making opaque, with few obvious opportunities for human intervention. Where interfaces remain, design has drifted toward abstract, screen-based control, replacing material interaction with menu navigation. These shifts weaken the coupling between human action and system state, eroding operators' ability to understand, anticipate, and intervene in automated systems. We introduce Physically Stateful Interfaces (PSIs), a design concept that re-imagines traditional controls, such as buttons, switches, and knobs, as actuated, bidirectional interface elements. Three foundational behaviors, Self/Reset, Resist/Hide, and Assert/Unhide, unify affordance, feedforward, and feedback into a single interaction point. Rather than static inputs whose impact and consequences must be read elsewhere on a screen, PSIs position physical controls as shared mediators between automated systems and human operators. As automated systems become increasingly capable, their interfaces need to be deliberately designed to center human agency, supporting deliberate choice while keeping automation legible, contestable, and overrideable.

0.24