CS Paper Radar

A11yLTLNav: Automatic Detection of Accessibility Navigation Failures

Chenming Ge, Kewen Peng, Chengyang Shi, Ben Greenman, Yue Jiang

cs.HC cs.SE ·submitted 2026-09-16 00:39 UTC · v1

Abstract

For blind and low-vision (BLV) screen-reader users, a website that appears accessible in a static snapshot can become difficult or impossible to navigate once interaction begins. Yet, most automated accessibility checkers miss failures involving focus, interface state, and accessible feedback across interactions. We present A11yLTLNav, a property-based approach for automatically detecting accessibility navigation failures. Through a structured review of prior research, we organize accessibility navigation failures into a failure taxonomy and formalize a browser-observable subset as executable Linear Temporal Logic properties over action-state traces. A11yLTLNav combines random keyboard exploration with runtime property monitoring to detect these failures during interactions. We evaluate A11yLTLNav on 31 generated websites based on real-world websites and tasks. It reported 309 accessibility failures, of which 274 were confirmed, achieving 88.7% precision and identifying more confirmed failures than the comparison checkers. Our results show that A11yLTLNav transforms accessibility knowledge into reusable checks of interface behavior over time.

Closest papers

Nearest neighbours by meaning across the whole index.

InterA11y: Interactive Web Accessibility Error Detection with LLM Agents

cs.HC ·2026-10-29 ·Mingyuan Zhong, Mengqi Shi, Davin Win Kyi, Ajit Mallavarapu, Xia Chen +2 more
code/data similarity 0.68
Abstract · We release our code and data at https://github.com/jszh/InterA11y

Large Language Models (LLMs) have shown potential to expand the coverage of web accessibility evaluation. However, current LLM-based checkers neglect much of the interactive and dynamic web contexts, resulting in limited coverage, high false discovery rates, and inconsistent performance across models. We present InterA11y, an agentic accessibility evaluation system that combines rubrics and interactive tools to support LLM agents in evaluating accessibility effectively. The rubrics decompose WCAG success criteria (SC) into testable aspects and guide agents in collecting and interpreting evidence. The interactive tools inspect or operate a webpage deterministically to collect evidence, such as focus appearances and error messages. We developed these rubrics and tools on 682 curated test cases grounded in established accessibility guidelines that target 10 commonly violated SC. We evaluated InterA11y on 197 expert-annotated cases sampled from 54 real webpages. Results show InterA11y achieves higher recall than the strongest evaluated LLM-driven baseline (79.0% vs. 44.4%), while maintaining comparable precision to best rule-based checkers (93.1% vs. 90.3-97.6%), with little variability between models. Further, when equipped with our interactive tools, general-purpose web agents detect more accessibility errors with fewer false positives and less token usage than without these tools.

0.33

Agentic Web Accessibility Auditing: A Criterion-Specific Framework for Translating WCAG Requirements into Assessments

cs.HC ·2026-09-08 ·Arjun Mishra, Pranav Karthik, Byungjun Bae, Dongwook Yoon
dataset similarity 0.65
Abstract · Preprint. 26 pages, 7 figures. Revised title, abstract, conclusion, and exposition; clarified methods, dataset construction, and interpretat…

Web accessibility auditing requires interpreting diverse requirements and examining interface behavior. Rule-based checks and noninteractive model assessments can miss barriers requiring contextual or interactive evidence. We present an agentic framework that assigns a vision-language agent to each accessibility requirement. Guided by tailored instructions, agents inspect webpages, operate controls, and record evidence supporting their findings. We implement the framework for 40 requirements from the Web Content Accessibility Guidelines (WCAG). To compare detection and cost, we construct a dataset of 250 page-criterion records derived from expert audits across 11 scholarly platforms. Agents recover 67 of 78 reported positive cases (86% recall), compared with 36% for axe-core, a rule-based checker, and 67% for an uncued, noninteractive vision-language model, at lower precision (56%). They recover nine of ten Keyboard and No Keyboard Trap cases missed by both baselines. Together, the framework, implementations, and dataset support automated accessibility auditing grounded in inspectable evidence.

0.16

Spatial Organization Is Accessibility Structure: Hierarchical Screen Reading of Mobile UIs

cs.HC ·2026-11-02 ·Yongxiang Hu, Xuan Wang, Qingyang Qiu, Yingjie Liu, Chun Kang +3 more
similarity 0.64
Abstract · 26 pages, 6 figures, 7 tables

Screen readers for blind and low-vision (BLV) users flatten mobile UIs into a linear sequence. Modern mobile UIs, however, contain dozens of elements grouped by spatial layout. Such linear access is ill-suited for BLV users, who must infer each element's meaning and which elements form a functional unit. Drawing on a quantitative TalkBack audit and formative interviews, we present Spatial Screen Reader (SeeR), which builds a function-aware hierarchy from accessibility metadata and provides unit summaries with hierarchical navigation. Compared with TalkBack, SeeR reduces non-informative focus stops by 82.1% and navigation steps by 52.7%. With ten BLV participants, summaries helped them connect controls to their functions and skip nonessential elements, especially on unfamiliar pages, though extra navigation levels added effort when the target was already known. These findings recast spatial organization as accessibility information: apps and platforms should expose it, rather than leaving screen readers to infer it post hoc.

0.17

Hybrid Computer Use: A Unified Programmatic, Semantic, and Visual Interface for General Agents

cs.HC cs.CL ·2026-10-30 ·Haoqing Wang, Xiaoyi Zhang, Sicheng Fan, Zhexin Zhang, Xinjie Zhang +3 more
position similarity 0.63
Abstract

Computer-use agents increasingly tackle tasks ranging from everyday desktop activities to long-horizon professional workflows. Yet interfaces centered on screenshots and mouse-and-keyboard actions can impose a human-interaction prior, requiring repeated screen interpretation while leaving the computer's semantic and programmatic capabilities underexposed. We argue for exposing these capabilities and letting models choose and combine them. We introduce Hybrid Computer Use (HCU), a unified interface for general agents that combines three complementary channels: programmatic interaction through code and command-line tools, semantic interaction through accessibility (AX), and visual interaction through graphical user interfaces (GUI). Our HCU supports efficient AX interaction and guards against stale-state actions during channel switching. A supplementary HCU skill guides agents in selecting and coordinating channels to use the unified interface effectively. Our designed AX tools return compact observations, execute actions in batches, and report their effects as AX diffs, so the model needs fewer round trips. On OSWorld 2.0, HCU achieves the highest Partial and Binary scores among the compared systems. Analyzing agent trajectories, we find that all evaluated models improve with HCU yet allocate work across the channels differently and consistently outperform compared baselines in terms of both efficiency and task success, highlighting the value of unified computer-use interfaces that allow flexible channel selection and combination.

0.13

Coral: Contextual Gists for Blind and Low Vision Screen Reader Users' Understanding of Dynamic User Interfaces

cs.HC ·2026-09-19 ·Ritesh Kanchi, Jianna So, Krzysztof Z. Gajos
hot direction similarity 0.63
Abstract

Blind and low vision (BLV) screen reader users construct mental models of user interfaces (UIs) through incremental screen reader interaction, a time-consuming and cognitively demanding process complicated by modern interfaces that may not convey dynamic content accessibly. We interviewed eleven BLV screen reader users about UI understanding and derived design objectives that informed Coral, a context-aware browser extension. Coral synthesizes interface and user context to generate screen reader-narrated gists which holistically notify users of interface states and changes relevant to their likely goals and ongoing interaction. In a comparative evaluation with eight BLV screen reader users, participants used Coral to form initial expectations and interpret interface changes, and reported spending less time and effort manually verifying interaction outcomes. Together, our findings provide deeper insight into how BLV screen reader users navigate gaps in their UI understanding, and how intelligent support that combines interface and user context can bridge these gaps.

0.24

A Large-Scale Evaluation of Cookie Notice Privacy and Accessibility for Blind and Low Vision Users

cs.CR ·2026-10-31 ·James M. Clarke, Maryam Mehrnezhad
similarity 0.60
Abstract

We present the first large-scale privacy and accessibility evaluation of cookie notices across 2,061 domains for blind and low vision (BLV) users. Using a Playwright-based crawler, a custom NVDA screen reader add-on, and the Lighthouse and WAVE accessibility tools, we measure UK GDPR/PECR compliance, tracker activity across pre-interaction, post-accept, and post-reject phases, and cookie notice accessibility for screen reader users. We find that 51 % of sites display a cookie notice, of which (at least) 50 % violate UK GDPR/PECR requirements. Accepting a notice produces a mean five-fold increase in tracker activity, and degrades page landmark structure on one in five sites. While 95 % of notices are readable by a screen reader, only 37 % carry a meaningful title, 39 % are announced immediately on page load, and 42 % use descriptive button labels. We reveal a systemic correlation: cookie notices failing legal privacy requirements disproportionately exhibit the weakest screen reader accessibility. This compounding privacy burden on BLV users demands urgent intervention from developers, regulators, and standards bodies.

0.31

Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications

cs.HC cs.AI ·2026-09-01 ·Satwik Ram Kodandaram, Monalika Padma Reddy, Xiaojun Bi, Jiawei Zhou, I. V. Ramakrishnan +1 more
top venue similarity 0.59
Abstract · Accepted to the EMNLP 2026 Main Conference

Computer-use agents are emerging as a paradigm for agentic human-AI interaction, combining language reasoning with multi-modal interface grounding to operate GUIs. Yet their effectiveness for blind screen-reader users in real-world desktop workflows remains unclear. We present a three-week diary study with 8 blind users using OLLA, a screen-reader-accessible CUA prototype, collecting 1,258 commands across 12 applications with screenshots, UI trees, model responses, and action traces. We evaluate GPT-5 during deployment and re-execute the same commands with four additional models. GPT-5 achieved the highest success rate at 52.5%. Trace analysis reveals grounding, planning, constraint-tracking, and termination failures, while interviews reveal beyond-automation needs.

0.60

InSight: A Mobile Assistant Helping Visually Impaired People Navigate Unfamiliar Environment

cs.HC ·2026-10-13 ·Junwei Zheng, Jiale Wei, Di Wen, Yufan Chen, Ruiping Liu +4 more
similarity 0.58
Abstract

Navigating unfamiliar outdoor environments is one of the most demanding everyday tasks for people with visual impairments (PVI). However, existing navigation systems cannot simultaneously satisfy the needs of informative perception and precise localization during navigation. This paper explores the core navigation demands of PVI through a preliminary study. Based on the results of the pre-study, we present the design of InSight, a mobile assistant that combines scene descriptions with precise localization to support PVI navigating unfamiliar outdoor environments. The offline mode of InSight runs entirely on device for real-time navigation, while the online mode streams camera frames to a self-hosted server for arbitrary object detection and accurate localization. Finally, a user study with 21 visually impaired participants validates that InSight could assist users in navigating unfamiliar outdoor environments with increased success rate and confidence compared to mainstream navigation applications like Apple Maps.

0.18