CS Paper Radar

EgoAsk: Egocentric Teaching of Personalized Object Knowledge for Household Robots

Yuanda Hu, Wenbin Zuo, Yiting Shen, Tianle Chen, Hector Fabio Calero Tobar, Yate Ge, Xiaohua Sun, Weiwei Guo

cs.HC ·submitted 2026-09-15 07:39 UTC · v1

Abstract

Unlike users, who know their own belongings and routines, household robots cannot easily acquire such personalized object knowledge automatically and depend on users to teach them. User-initiated teaching requires users to arrange dedicated teaching sessions and decide what to teach, even when they are unsure what the robot needs to learn. We introduce EgoAsk, a smart-glasses-based system that proactively embeds personalized object teaching into everyday activities. EgoAsk shares the user's first-person view with the robot, identifies gaps in personalized object knowledge, and analyzes ongoing activity to ask context-relevant questions that support future household assistance. To examine how teaching initiative and question timing affect users' teaching experiences, we conducted a within-subjects study with 18 participants and found lower reported knowledge-gap monitoring burden with robot-initiated questioning and less need for context reconstruction with EgoAsk. These findings characterize teaching burdens and timing preferences, offering design implications for egocentric robot-teaching systems.

Closest papers

Nearest neighbours by meaning across the whole index.

HEIR: Harness Egocentric Intent for Human-Robot Interactions

cs.RO ·2026-11-12 ·Yichi Zhang, Bo Wang, Zhenghan Zhu, Yutong Han, Jiayin Zou +9 more
code/data similarity 0.64
Abstract

For robots to assist, serve, and collaborate with humans, they must not only interpret human requests conveyed through language and multimodal intentional cues, but also execute long-horizon tasks that interleave navigation and manipulation. In this paper, we introduce HEIR, a resource for studying how to Harness Egocentric Intent for Human-Robot Interaction. HEIR pairs human speech, measured gaze, and egocentric video with robot observations and teleoperated navigation-manipulation trajectories, while each session preserves successive episodes, with each request fulfilled through multiple execution subtasks. The captured data, together with human-authored annotations, support the study of intent grounding, execution planning, and task completion within a unified interaction setting. To demonstrate the value of HEIR, we conduct extensive offline experiments to evaluate how models benefit from additional human-centric context across these tasks, together with real-robot experiments that assess execution task completion. HEIR thus provides a valuable resource for studying user-centric robot interaction in naturalistic household environments. Our demo and dataset are available online.

0.18

Longitudinal Robot Learning from Demonstration with Care Providers in a Home Environment

cs.RO cs.HC ·2026-08-25 ·Nina Moorman, Julianna Schalkwyk, Vriksha Srihari, Qingyu Xiao, Kamel Alrashedy +4 more
publishedcode/data similarity 0.62
Abstract · ICRA 2026 Workshop on Bridging the Gap between Robot Learning and Human-Robot Interaction

Learning from demonstration (LfD) methods enable non-expert end users to teach robots novel skills without explicit programming. However most evaluations of the usability of LfD with non-experts has been conducted in controlled laboratory environments with a robotics experimenter present. In this work we identify non-expert end users' key barriers when teaching robots via demonstration without live robotics expert feedback in a home environment. In our human subjects experiment we support the non-expert end users through two forms of demonstrator guidance developed in prior work: pre-training and adaptive feedback. Towards the ecological validity of the evaluation, we conduct this experimentation over multiple visits, with a population of care providers. Finally, we propose to open source the resulting LfD dataset of care providers teaching a robot assistive tasks over multiple visits to a home environment.

0.32

AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalization

cs.CV ·2026-09-17 ·Xu Yuan, Yi Wang, Zhuohang Jiang, Haohao Qu, Yujuan Ding +6 more
survey similarity 0.62
Abstract

Recent advances in artificial intelligence (AI) are reshaping smart glasses from egocentric capture and display devices into platforms for wearable intelligence. Smart glasses increasingly serve as wearable AI systems that connect first-person observation with real-time assistance under strict form-factor constraints. We frame this transition through the lens of \emph{AI smart glasses} and define them as a system-level concept in which egocentric sensing, resource-aware computing, intelligent reasoning, multimodal interaction, and real-world application constraints are co-designed for personalized assistance in the physical world. To systematically study this perspective, we organize the survey around four connected dimensions. First, we examine the hardware foundation that bounds sensing, computation, feedback delivery, and sustained deployment. Second, we study wearable intelligence, where egocentric signals are transformed into perceptual, contextual, and agentic capabilities. Third, we discuss interaction design, through which users request, receive, correct, and regulate assistance during ongoing activity. Fourth, we analyze application scenarios across healthcare, accessibility, situated learning, daily life assistance, cultural tourism, and industrial support, showing how domain requirements reshape system design and evaluation. We further identify five cross-cutting research challenges for future AI smart glasses: next-generation hardware, trustworthy egocentric intelligence, lifelong personalized memory, proactive intelligence, and embodied foundation models. By centering smart glasses as wearable-intelligence platforms, this survey provides a unified framework for organizing technologies, applications, and open challenges in this emerging area.

0.30

Bridging Teacher Expectations and Robot Learning via Coupling Dynamics

cs.RO cs.HC ·2026-08-25 ·Evan Dallas, Sean Dallas, Wing-Yue Geoffrey Louie
published similarity 0.62
Abstract · 8 pages, 1 figure

Human-robot teaching focuses on enabling nontechnical experts to customize robots according to their needs after deployment. With recent advances in machine learning, human-robot teaching is no longer confined to offline learning where the data gathering step from a human teacher is separated from when the robot learns. Instead, more recent approaches for human-robot teaching focus on coupling human teaching with robot learning. This coupling impacts the structure, timing, and content of the teaching and learning interaction. However, it is currently unclear how such coupling dynamics affect humanrobot teaching effectiveness and human perceptions towards the teaching process. Informed by human learning theories, in this paper we propose a new scale for classifying human-robot teaching interactions according to coupling dynamics present between the human teacher and robot learner. We apply this scale to a subset of the human-robot teaching literature to identify how coupling dynamics and human teacher mental model mismatches with the ground truth robot learning system affect teaching effectiveness and human perceptions towards the teaching process

0.23

Robots That Take Initiative: A Framework for Building and Evaluating Proactive Robots

cs.RO cs.AI ·2026-09-24 ·Maithili Patel, Sonia Chernova
similarity 0.62
Abstract

Effective robot assistance beyond narrow roles and repetitive tasks requires robots to be proactive - to decide what needs to be done rather than waiting to be told. While proactivity is increasingly explored, it lacks a unified formulation, and work in the domain is typically evaluated offline against static human models that cannot capture the effect of a robot's actions on the environment and the user's own behavior. We introduce a unified formalism for proactive robot assistance, organize it into three levels, and provide a framework to address the highest level of unprompted proactive assistance. We then show that offline evaluation overstates performance in this setting, and contribute a closed-loop evaluation with a human model that adapts to the robot. Finally, we present a method, GAP, that instantiates our framework, learning from passive observation to anticipate user goals and act. Under closed-loop evaluation, prior state-of-the-art methods collapse, in some cases adding more work than they save, while GAP remains robust and substantially outperforms them.

0.22

COOL: Curiosity-Driven Object Ownership Learning for Personalized Robotic Assistance

cs.RO ·2026-10-07 ·Samira Huber, Ruben Hammele, Sören Pirk
published similarity 0.62
Abstract · Accepted to CoRL 2026

Robots are increasingly expected to provide personalized services in everyday environments. To do so, they must ground natural-language commands such as "Where is my backpack?" or "Find my bottle" and execute them by reasoning about object instances, people, locations, and ownership. This is challenging because ownership is rarely labeled explicitly and must be inferred from long-term, behavioral evidence of human-object interactions. To address this, we present COOL, a novel robotic framework for autonomously learning object ownership from everyday observations and maintaining a long-term spatial memory of its environment. To keep its memory current, COOL uses an agent-based curiosity-driven data collection strategy that guides the robot toward the most promising locations to gain information and refresh stale observations. Offline experiments, ablation studies, and real-world evaluations show that COOL can infer ownership relations from real-world interactions and use this knowledge for ownership-conditioned navigation and task execution.

0.29

Report on The 1st Workshop on Human-Centered Proactive and Personalized Agents for Interactive Information Access at CHIIR 2026

cs.HC ·2026-08-19 ·Kirandeep Kaur, Vinayak Gupta, Tanya Roosta, Madhura Raju, Grace Hui Yang +1 more
top venue similarity 0.61
Abstract

Interactive information access is increasingly moving beyond reactive query-response paradigms toward agentic systems that can personalize interaction, retain context, infer latent needs, recommend next steps, and initiate support. This shift creates new opportunities for adaptive and context-aware assistance, while also raising important questions about autonomy, privacy, trust, transparency, user welfare, and evaluation. The First Workshop on Human-Centered Proactive and Personalized Agents for Interactive Information Access provided an interdisciplinary forum for examining these questions across information retrieval, human-computer interaction, dialogue systems, AI ethics, cognitive science, learning technologies, and human-centered AI. Through invited talks, paper presentations, and open discussion, the workshop engaged with topics including calibrated initiative, knowledge-gap navigation, long-term memory, value-sensitive design, implicit personalization, AI-mediated care, proactive dialogue, and evaluation beyond task accuracy. A central theme across the workshop was that proactivity should not be understood only as earlier action or improved prediction, but as a form of initiative that must be appropriately timed, transparent, contestable, and aligned with user goals. This report summarizes the workshop and synthesizes the research challenges it surfaced for designing proactive and personalized agents in interactive information access.

0.42

An Agentic Framework for Proactive Robots with Procedural Assistance Memory

cs.RO ·2026-10-15 ·Dohyun Lee, Minho Park, Jeonghoon Park, Dongyoon Hwang, Junha Hyung +3 more
code/data similarity 0.60
Abstract · Conference on Robot Learning (CoRL) 2026. Project page: https://davian-robotics.github.io/PAM/

Proactive robots should anticipate user needs and provide assistance before being explicitly asked. A straightforward approach is to infer human intentions from commonsense knowledge, but the same observed behavior can imply different needs across users, potentially leading to unwanted assistance. We observe that everyday routines in the real world exhibit recurring patterns, so learning these patterns can reduce ambiguity in proactive decisions. Based on this, we propose a memory-based agentic framework in which a VLM task predictor reads a Procedural Assistance Memory (PAM) alongside the current observation to decide when to assist and what task to perform, and a VLA executor performs the predicted task. PAM is structured as a finite-state machine over activity phases, per-state assistance rules, and event-driven transitions; a reflection VLM revises PAM from each day's interaction trace without weight updates or retraining. In our 5-day desk-assistance evaluation, memory-free baselines remain conservative, with limited recall on proactive tasks. Our system improves consistently with accumulated experience, achieving the highest precision, recall, and F1 and reducing first-fire delay from approximately 5 seconds on Day 1 to 1 second by Day 5. Our approach also adapts to changes in the user's routine through continued memory reflection, recovering from memory--routine misalignment after a single memory update. The project page is available at https://davian-robotics.github.io/PAM/

0.56