CS Paper Radar

RoboCafé in the Open: Interaction Continuity in Long-Term Public Human-Robot Interaction

Kaitlynn Taylor Pineda, Kush Kumar Kushwaha, Jie Wang, Jiaming Du, Anvii Mishra, Emilie Basu Suri, Angela Guo, Chien-Ming Huang

cs.RO ·submitted 2026-09-23 07:38 UTC · v1
hot direction

Abstract

As robots remain in public spaces over extended periods, they must maintain interaction continuity by preserving and correctly applying context as people, encounters, and circumstances change. To study interaction continuity in long-term public human-robot interactions, we developed RoboCafé, an autonomous conversational coffee robot designed to support repeated interactions through task-aware dialogue, real-time multimodal perception, and memory of prior encounters. We deployed RoboCafé for 12 days in a university building, where it received 148 orders. The deployment involved repeat customers, passersby, changing groups, and back-to-back orders that repeatedly crossed the boundaries assumed by the system's order-centered interaction model. We found that successful interaction continuity requires a robot to determine who is currently present, which prior context belongs to whom, where interactions begin and end, and whether its representation of an interaction matches what is occurring in the physical world. From these observations, we derive four system design requirements for maintaining interaction continuity in longitudinal public human-robot interactions: contextual interaction state, persistent person grounding, explicit interaction life-cycle management, and interaction observability.

Comment
8 pages, 3 figures. Kaitlynn Taylor Pineda and Kush Kumar Kushwaha contributed equally to this work. Submitted to IEEE International Conference on Robotics and Automation (ICRA 2027)

Closest papers

Nearest neighbours by meaning across the whole index.

Mind the Context: Continual Learning of Socially Appropriate Robot Actions via Environmental-Social Disentanglement

cs.RO ·2026-08-13 ·Rafal Robert Karpinski, Fethiye Irmak Dogan, Nikhil Churamani, Yiming Luo, Maartje M. A. de Graaf +2 more
publishedcode/data similarity 0.66
Abstract · Extended version of the paper accepted at the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)

Social robots are expected to operate across diverse environments, where similar arrangements can imply different socially appropriate actions, e.g., starting a conversation may be acceptable in a crowded home but disruptive in an office meeting. Because such norms and environments cannot all be anticipated in advance, robots require continual learning (CL) to adapt from sequential experience while retaining previously acquired knowledge. Prior work has studied CL for generating socially appropriate robot actions, but it has not addressed domain-incremental settings in which the robot incrementally encounters diverse contexts (e.g., living room, meeting room, office, hallway), where both environmental (e.g., whether the space is open or cluttered with furniture) and social cues (e.g., how people or other agents are positioned around the robot) jointly shape the appropriateness of robot actions. We address this gap with the Explicit Disentanglement Dual-Branch (EDD) framework. EDD explicitly separates environmental and social-agent related knowledge and uses replay-based rehearsal to mitigate forgetting while learning the appropriateness of robot actions (e.g., cleaning, serving, starting a conversation) across several indoor domains. Experiments show that EDD outperforms several state-of-the-art baselines, and ablation studies further evaluate different disentanglement strategies and the sensitivity to domain ordering. Our code is publicly available at https://github.com/Cambridge-AFAR/Mind-the-Context.git.

0.47

MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions

cs.RO ·2026-09-21 ·Lijian Lin, Ye Zhu, Fan Zhang, Yunfei Liu, Baofeng Li +3 more
similarity 0.65
Abstract

% !TEX root = ../main.tex Real-time embodied companion interaction requires a robot to infer user intent from streaming speech, generate timely responses, and execute expressive, interruptible motions. Existing systems typically decouple dialogue orchestration from gesture synthesis, relying on offline motion generation from complete audio. This separation leaves open how a deployed robot can dynamically synchronize response content, prosodic timing, and physical safety under incremental inputs and uncertain turn boundaries. We present MIRA, a unified framework for real-time full-duplex embodied companion interaction. Given streaming user speech, dialogue history, and vocal affect, MIRA predicts both the response text and an explicit embodiment cue that routes the response to the appropriate physical behavior. Discrete social behaviors (\eg listening and greeting) are mapped to validated robot trajectories, while speaking responses are accompanied by streaming, generative co-speech motion. For co-speech motion generation, we propose ROSCO, a prefix-conditioned diffusion model for streaming audio-to-joint motion generation. We further design RHPC, an inference scheme that maintains a sufficiently long temporal context for motion prediction while bounding physical commitment to a short, interruptible prefix. At the interaction level, we design CORTEX, a dual-timescale interaction policy that combines low-latency barge-in preemption and streaming response generation with deliberative turn decisions, backed by a robot-side execution layer that enforces physical safety constraints during execution. MIRA is deployed on an Astribot S1 humanoid robot. Quantitative evaluations demonstrate competitive audio-motion alignment relative to state-of-the-art motion-generation baselines, while real-robot deployment measurements characterize streaming responsiveness and interruption handling.

0.17

How to start a conversation proactively?

cs.HC cs.LGcs.RO ·2026-08-22 ·Mohit Nathani, Aaditya Gupta, Eshwar Dhande, Chayan Sarkar
publisheddatasetcode/data similarity 0.62
Abstract · The article is accepted in IEEE RO-MAN 2026. The dataset is available at https://github.com/mohit-nathani1/conversation-opener-dataset

Proactively initiating a conversation is a fundamental social competence and a critical capability for social robots. However, determining how and when to start a conversation in a given situation remains an open challenge, primarily due to the lack of a principled definition of conversational context. While recent large language models (LLMs) demonstrate impressive conversational fluency, they are predominantly turn-reactive and often produce generic or monotonous openers when tasked with initiating interaction, even when contextual information is available. Such limitations are particularly problematic for embodied social robots, where inappropriate or poorly timed initiations can negatively affect user engagement and rapport. In this work, we propose a schema of contextual variables that jointly characterize situations relevant for proactive conversation initiation. The schema is designed to structure perceptual, situational, and interactional cues such that they can be effectively exploited by language models. Furthermore, motivated by the absence of suitable training data, we introduce a curated dataset of contextually-aligned conversation openers (CCO) covering diverse everyday scenarios. Using this dataset, we fine-tune small language models (SLMs) to generate proactive conversation initiations. Through a comparative evaluation involving zero-shot LLMs, off-the-shelf SLMs, and our fine-tuned models, we demonstrate that explicit contextual structuring and targeted data curation substantially improve the quality and contextual appropriateness of conversation openers. Our results highlight the importance of context modeling for proactive social interaction and provide actionable building blocks for conversational robots.

0.41

InstAgent: A Recursively Self-Improving Agent Operating System for Heterogeneous Robot Collaboration

cs.RO ·2026-10-16 ·Haoyang Luo, Junhao Qin, Jingyan Cao, Yi Huang, Guangjie Yang +5 more
similarity 0.62
Abstract · 9 pages, 3 figures

Heterogeneous robot teams enable complex, long-horizon tasks by combining complementary sensing, mobility, and manipulation capabilities across multiple embodiments. Yet existing systems struggle to retain execution-time corrections as reusable experience, maintain task-relevant context memory across space and time for reliable handoffs, and explicitly represent the inter-robot dependencies required for structured collaboration. In order to address these challenges, we present InstAgent, a recursively self-improving (RSI) Agent Operating System for heterogeneous robot collaboration. InstAgent organizes team behaviors as agentic behavior trees (BTs) that explicitly model capability assignment, inter-robot dependencies and handoff checks. The RSI mechanism updates these collaboration programs from execution feedback and preserves successful revisions for reuse in future tasks, while shared multimodal spatio-temporal memory provides context for planning and handoff validation. Experiments on 20 real-world multi-robot tasks with three high-level models show consistent gains in task success, first-attempt completion, and execution efficiency.

0.46

Towards an Extensible Benchmark for Spoken Dialogue with Social Robots

cs.RO ·2026-10-06 ·Casey Kennington, Ross Mead, Saad Elbeleidy, Jesse Thomason
publishedbenchmark similarity 0.62
Abstract · Accepted to and presented at the IROS 2026 Workshop on Human-Robot Dialogue

Language models provide a plug-and-play interface between humans and robots, but important challenges remain when speech, dialogue, fast interaction, and collaboration are required. We propose a benchmark for the community to use as a way to explore common spoken dialogue artifacts between robots and humans, including requests for clarification, interruptions, embodied signals (e.g., head nods or facial cues), and time constraints. We also explain our vision to extend the benchmark for other aspects of human-robot interaction that are important to the larger research community. To facilitate the benchmark, we further propose using \textit{Retico}, a real-time communication framework that fulfills important technical requirements to enable robots to have spoken dialogue capabilities.

0.20

Visual Proactivity: Enhancing Human-Robot Collaboration Through Intent Communication

cs.RO ·2026-09-18 ·Valerio Bo, Edison Bejarano, Anaís Garrell, Alberto Sanfeliu
top venue similarity 0.61
Abstract · Accepted for publication at the 2026 IEEE International Conference on Robotics and Automation (ICRA)

As robots transition from performing repetitive tasks to collaborating with humans, understanding human intent becomes crucial to effective interaction. Anticipation enables robots to predict human actions, while proactivity allows them to take initiative and guide human behavior toward optimal outcomes. Although research has largely focused on how robots infer and respond to human intentions, less attention has been paid to how robots communicate their own intent. This paper introduces visual proactivity, a novel, simple yet effective approach that enables robots to communicate their intentions through visual feedback, influencing human behavior and enhancing transparency and fluency. We develop and evaluate proactive robotic behaviors in a human-to-robot handover scenario, where a user study validates human perception of reactive, anticipatory, and proactive behaviors. The results demonstrate that effective visual proactivity fosters better alignment and coordination, paving the way for more intuitive human-robot collaboration.

0.46

Building Rapport with Self-Disclosure in Human-Robot Interaction

cs.HC cs.RO ·2026-08-22 ·Haritha Anupindi, Chayan Sarkar, Tapas Nayak, Marynel Vazquez
publishedhot direction similarity 0.61
Abstract · Accepted in IEEE RO-MAN 2026

As social robots become increasingly present in everyday environments, establishing rapport with users is critical for enabling sustained and meaningful human-robot interaction. In human-human communication, self-disclosure is a well-established mechanism for building rapport; however, its role and impact in human-robot interaction (HRI) are not yet well understood. Although recent advances in large language models (LLMs) have improved the linguistic competence of conversational robots, such systems are typically deployed as reactive information providers, offering limited social reciprocity and relational depth. In this article, we investigate the role of robot self-disclosure in fostering rapport during face-to-face human-robot interaction. We propose a dialogue system, called RADIA that leverages retrieval-augmented generation (RAG) to enable a robot to retrieve and reference simulated autobiographical memories, allowing first-person self-disclosure to be incorporated naturally into ongoing conversations. We evaluate the proposed approach through an in-person user study, comparing interactions with a self-disclosing robot against a baseline non-self-disclosing robot. Our results show that robot self-disclosure significantly influences conversational dynamics and enhances users' perceived rapport with the robot, suggesting that self-disclosure is a valuable social behavior for long-term and socially engaging human-robot interaction.

0.38

SocialCoAct: Enabling and Benchmarking Egocentric Social Co-Action for VLM-Based Humanoid Robots

cs.RO ·2026-10-17 ·Jiancong Ma, Kun Li, Jin Jiang, Shengcai Liao
benchmarkcode/data similarity 0.61
Abstract

Enabling humanoid robots to participate effectively in social interactions is important for their integration into human-centered environments. Existing methods, however, typically map pre-generated speech directly to continuous motion trajectories but have limited awareness of complex social environments and lack explicit high-level semantic action planning. We introduce SocialCoAct, a unified agent-centered framework for VLM-based egocentric social co-action. We formulate egocentric social co-action as an end-to-end agent task and develop SocialCoAct-Agent, which transforms egocentric observations and dialogue history into verbal responses and atomic action sequences, aligns each action with its corresponding utterance span, and executes the resulting behaviors in MuJoCo and on a physical humanoid robot. To evaluate the VLM capabilities required by this task, we construct SocialCoAct-Bench from continuous egocentric recordings of natural multi-party interactions. The benchmark comprises 423 human-verified multiple-choice questions across three subtasks: Long Episode Understanding (LEU), Speaker Intention Understanding (SIU), and Coordinated Action Planning (CAP). Across a diverse set of closed-source and open-weight VLMs, including general and egocentric models, the highest overall accuracy is 77.54%, while accuracy on the hard subset reaches only 46.51%. Extended reasoning improves Qwen3-VL-2B and Qwen3-VL-4B by 19.62 and 17.02 percentage points, respectively, but reduces inference throughput by 53.6$\times$ and 20.2$\times$. Visual-input ablations further show that the benefits of visual context vary across models. SocialCoAct provides a foundation for developing and evaluating VLM-based humanoid agents that coordinate context-aware speech and action in complex social interactions. The codebase and benchmark will be made publicly available upon publication.

0.29