CS Paper Radar

FocusAdapt: Context-aware Adaptive Focus Assistance in Diminished Reality

Tianyu Zhang, Shutong Wu, Jiankun Yang, Zhen Bai, Yukang Yan

cs.HC ·submitted 2026-08-31 00:52 UTC · v1
published

Abstract

Diminished Reality (DR) can reduce visual clutter by removing irrelevant objects. However, removing all task-irrelevant objects may eliminate useful contextual information and reduce situational awareness. We present FocusAdapt, a context-aware DR system that predicts object-level distraction by integrating visual saliency, semantic relevance, and gaze behavior. Based on findings from a formative study, FocusAdapt selectively diminishes highly distracting objects while preserving useful context, enabling adaptive focus assistance during procedural tasks.

DOI
10.1145/3830397.3841867

Closest papers

Nearest neighbours by meaning across the whole index.

Dotting the Eye: An Intent-Driven Image Retouching Agent for Visual Focus Enhancement

cs.CV ·2026-09-01 ·Chujie Qin, Zilong Zhang, Zewei Chang, Chunle Guo, Ruixing Wang +3 more
top venuecode/data similarity 0.63
Abstract · Accepted to the European Conference on Computer Vision (ECCV) 2026

Image retouching is commonly formulated as enhancing overall visual quality through color adjustment, but in practice, it also serves to emphasize visual focus by guiding viewers' attention toward a specific subject or region. Achieving such focus-oriented retouching is inherently challenging, as it requires well-coordinated global and local adjustments to manipulate perceptual saliency while maintaining visual naturalness. This intricate process typically demands substantial professional expertise. In this study, we propose EyeControl, a MLLM-driven agent with a diffusion-based retouching executor that enables visual focus enhancement under weak user intent. With only a few clicks or coarse strokes, EyeControl directs visual attention to the intended region, effectively "dotting the eye" of the image. The core idea is to explicitly link the weak user intention with the target editing region and the corresponding tonal adjustment operations during retouching. To achieve this, the system first interprets the intent and image content to infer the visual focus and generate structured intent guidance for the retouching executor. Second, the retouching executor is encouraged to respond more strongly to the target region, explicitly aligning its attention map with a designed pseudo-intent map. We also introduce an operation-consistency constraint to improve coordination between global and local adjustments, achieving more natural and coherent retouching. Additionally, we contribute ControlArt-Bench, a high-quality evaluation dataset for visual focus enhancement. Extensive evaluations demonstrate that EyeControl yields perceptually appealing results with stronger intent alignment. Code can be found in https://github.com/DragonisCV/EyeControl.

0.58

MoGround: Measuring and Mitigating Modality Distraction in Vision-Language Models

cs.LG ·2026-09-27 ·Luca Zhou, Bo Zhao, Rose Yu, Emanuele Rodolà, Roberto Dessì
dataset similarity 0.61
Abstract · 9 pages of the main paper with 3 tables and 5 figures

We release MoGround, a vision-language dataset spanning four visual domains in which the answer to every question is guaranteed to be available from exactly one modality. This guarantee enables us to measure modality distraction, the failure in which a model answers a question correctly from one modality alone and then flips to a wrong answer once irrelevant content from the other modality is added. Existing probes rarely establish single-modality answerability this way, making it hard to isolate distraction in the first place. Across seven open-source VLMs, we find that modality distraction is not universal but model-dependent. The weaker-grounded modality is the more distracted one (r = +0.86), and distraction scales inversely with grounding strength (r = -0.90). The single-modality guarantee also enables a mitigation method that needs to distinguish between relevant and irrelevant context. Trained on one split of MoGround alone, a weight-space robustness vector reduces distraction on all seven models by 9% to 51%, at a cost of only 0.1 average points of accuracy on standard multimodal tasks.

0.26

Exploring Object Recall in an Object-Rich AR Office Scene

cs.HC ·2026-10-13 ·Sangita Kunapuli, Shane Dirksen, Radha Kumaran, Yanxiu Jin, You-Jin Kim +2 more
published similarity 0.58
Abstract · 11 pages, 9 figures. Accepted to IEEE ISMAR 2026 and published in IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 32, …

Augmented reality (AR) changes how users perceive and attend to their surroundings. Prior work suggests AR can reduce attention to the physical world, but it remains unclear whether this holds in environments dense with both virtual and physical objects, and which factors most affect object recall. This study examines recall in a mixed-reality office scene after an object-selection task, testing object-level factors (virtuality, size), context-level factors (semantic congruency, saliency), and user-level factors (cueing, dual tasking). Cueing informed participants of a later recall test, while dual tasking added cognitive load through a secondary audio task. Virtual objects were recalled more accurately than physical ones. Cueing produced a modest improvement in recall of present objects. Using playback software to reconstruct each participant's view throughout the trials, we computed object-level saliency and found a weak positive correlation with recall, slightly stronger than the effects of virtuality, but not reaching statistical significance. These findings have implications for AR interface design and for building mixed-reality environments that better support cognition during real-world tasks.

0.30

FocusDrive: Reasoning with Visual Focus for Autonomous Driving

cs.CV cs.RO ·2026-09-27 ·Zhiyuan Liu, Zehong Ke, Yuanxin Tian, Hao Cheng, Jinhao Li +5 more
similarity 0.56
Abstract

Driving decisions depend on both where to focus and how to act on what is seen. Effective driving reasoning must establish which objects matter, where they are, and how they inform the intended action. Text-based rationales can describe a driving response while leaving its correspondence to specific visual evidence implicit. Visual focus provides a concrete starting point for this connection by identifying what matters in the scene and where it is. We propose FocusDrive, a structured multimodal reasoning framework that organizes end-to-end planning around explicit visual focus. It pairs descriptions of decision-relevant objects with image-patch references, bringing explicit visual focus into the reasoning that generates driving plans and trajectories. We first assess this focus representation through driver gaze prediction, then investigate its role in planning reasoning using driving-focus annotations within existing NAVSIM training scenes. Experiments on W3DA and NAVSIM demonstrate competitive gaze prediction and end-to-end planning performance, with FocusDrive improving over text-based chain-of-thought. These results support visual focus as an effective link between scene understanding and driving action.

0.18

GazeAgent: VLM-Guided Implicit Gaze Calibration from Egocentric Multimodal Behavior for AI Glasses

cs.HC cs.CV ·2026-10-17 ·Yuhong Zhang, Khanh An Nguyen, Xinyu Tian, KungYi Chang, Abraham Padua +4 more
similarity 0.56
Abstract · 27 pages, 14 figures, 3 tables

AI glasses typically require explicit, user-specific eye-tracker calibration, and accuracy can drift when the device shifts during use. We present GazeAgent, a VLM-guided framework that continuously and adaptively improves gaze-calibration accuracy from naturally occurring behavior without asking users to interrupt their activities. GazeAgent identifies precise visual attention (PVA) events from multimodal streams of inertial measurement unit (IMU), gaze, and wearer-hand signals, validates their task relevance, and localizes functional targets as pseudo-calibration targets. In a 10-participant scripted study, the system detected all 820 predefined PVA opportunities and, under device disturbance, reduced median gaze error by 44.1% compared with explicit calibration. In a five-participant case study, autonomous calibration produced the lowest workload and interruption ratings, and all participants preferred it for routine use. By deriving task-critical targets from natural multimodal behavior, GazeAgent demonstrates the potential for continuous, adaptive, and low-burden gaze interaction on egocentric AI glasses.

0.08

VISTA: A Multimodal Egocentric Benchmark for Goal-Oriented Perception and Assistance to Blind and Low-Vision Users

cs.CV ·2026-10-16 ·Noé Mosca, Mohamed Rayan Barhdadi, Muhammad Hamza, Joshua Yeh, Suryodaya Bikram Shahi +9 more
benchmarkcode/data similarity 0.56
Abstract · Code and dataset are available at https://harvard-ai-and-robotics-lab.github.io/VISTA

Blindness and low vision limit many people's ability to complete daily tasks, motivating AI assistants that provide useful guidance. Yet, existing benchmarks do not adequately test whether vision models can provide effective everyday assistance. In particular, i) existing benchmarks primarily rely on visual and textual inputs, with limited support for other modalities that are increasingly available in egocentric wearable devices such as gaze, inertial measurements, spatial trajectories, and others; and ii) existing evaluations emphasize scene understanding and question answering, with limited coverage of goal-conditioned guidance for blind and low-vision users across everyday tasks. To bridge this gap, we introduce VISTA (Visually Impaired Scene and Task Annotations), a multimodal egocentric benchmark for goal-oriented assistance for blind and low-vision users. VISTA contains $\sim$1,000 egocentric recordings captured with Project Aria glasses; each recording provides synchronized sensor streams, six of which serve as model inputs, and three different types of annotation, covering ten task categories (TC), from object localization to hazard detection. We evaluate 14 representative vision-language models zero-shot on VISTA and design a lightweight multimodal model baseline that integrates all six modalities. Experimental results show that, although existing models can often describe egocentric scenes, they remain limited in generating goal-directed assistive guidance. VISTA thus provides a benchmark for evaluating multimodal models that could improve everyday accessibility.

0.25

Amplify What You Gaze At: Target Saliency Boosting in Text-to-Image Generation

cs.CV ·2026-09-25 ·Shengqi Dang, Zhengxi Yu, Feilin Han, Xingyu Lan, Nan Cao
dataset similarity 0.56
Abstract

Text-to-image generation has advanced in controlling what, where, and how objects appear, yet how visual attention is distributed among objects remains largely unexplored. In this paper, we introduce Target Saliency Boosting, a new task aimed at boosting the visual saliency of a specific object during text-to-image generation without requiring any visual priors. Our key insight is that visual saliency is inherently relative: boosting the saliency of a target object also depends on the global saliency distribution across all objects in the scene. Based on this insight, we propose GazeME, a lightweight framework that uses saliency-marked prompts, inserting learnable marker tokens around object descriptions to indicate which objects to visually emphasize or suppress. To learn these markers, we construct a saliency-semantics dataset that associates objects in image--prompt pairs with object-level saliency scores, and propose Saliency Prior Marker Activation (SPMA), a saliency-aware stochastic marker activation strategy that exploits relative saliency relationships for robust training. During inference, GazeME automatically inserts appropriate markers into the prompt, thereby directly enhancing the visual saliency of the target object. Extensive experiments demonstrate that GazeME effectively boosts target saliency while preserving both semantic alignment and image quality.

0.11

Training Vision Transformers to Focus: Emergence of Spatial Attention Head Specialization

cs.CV ·2026-11-09 ·Yuji Nozawa, Yu-Chieh Lin, Youyang Ng
top venuecode/data similarity 0.55
Abstract · Accepted at NeurIPS 2026. Project page: https://yuji-nozawa.github.io/ReFoCo/

Self-supervised learning in Vision Transformer models, such as DINO, enables robust visual perception without requiring labeled data by leveraging large-scale image datasets. However, these models lack a focusing mechanism during training, resulting in perceptual representations that are often ambiguous or overly similar. In this work, we introduce two visual focusing loss functions designed to establish correspondence between the model's attention and specific regions within images. Specifically, we leverage the multi-head attention mechanism within the model to selectively steer the model's focus, leading to the emergence of spatial attention head specialization, characterized by diverse and spatially focused perceptual capabilities without requiring annotations. Through both qualitative and quantitative evaluations, we demonstrate that our method substantially increases the spatial selectivity and diversity of attention heads, improving both interpretability and performance on dense prediction tasks such as zero-shot image segmentation.

0.61