CS Paper Radar

MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places

Jason Armitage, Ioannis Tsochantaridis, Linda Mazzone, Chuqiao Yan, Srini Narayanan, Sarah Ebling

cs.AI ·submitted 2026-08-28 14:41 UTC · v1
benchmark

Abstract

We introduce MAP, the first benchmark to evaluate multimodal AI systems as assistants for users with accessibility requirements when planning visits to places in the real world. In our evaluation, systems are presented with requests to verify or recommend a point of interest meeting an accessibility requirement. MAP contains two novel assessments: Claim verification for accessibility planning assesses if information on places and stated accessibility features is supported and identifies places that satisfy requested accessibility features. Visual evidence retrieval for accessibility planning checks if a multimodal AI system can select visual evidence for the requested place and accessibility feature. Our methodology supports comparison of AI systems in a setting where place information and accessibility information can change over time by evaluating systems and refreshing ground truth data at scheduled times. The benchmark is based on automatic rating and human rating for a proportion of responses.

Closest papers

Nearest neighbours by meaning across the whole index.

National-Scale Multimodal Accessibility Fields from Open Geospatial Data

cs.CY ·2026-10-23 ·Till Zacher, Louis Stille-Hönig, Markus Lienkamp
similarity 0.67
Abstract

Large-area accessibility analysis remains difficult to reproduce because multimodal routing, heterogeneous opportunity data, and reusable spatial outputs are rarely integrated in a single transparent workflow. We present MAPPER (Multimodal Accessibility and Place Profiling Engine for Arbitrary Regions), an open-data pipeline for multimodal accessibility analytics. The workflow builds walk, bike, car, and public transport travel-time graphs from OpenStreetMap and GTFS data, harmonizes mapped POI- and proxy layers representing the land-use side of the accessibility relation, and computes continuous accessibility fields using network kernel density estimation. A low-overhead native compute core (contraction-hierarchy routing, C++/OpenMP accessibility accumulation, DAG-style orchestration eliminating redundant computation) is the enabler that makes many graphs, large extents, and a broad layer set jointly tractable on commodity hardware. Outputs are sampled to a uniform H3 grid for downstream urban and regional analysis. A Germany-wide production run yields an H3-indexed accessibility dataset with nearly 40 million cells and 972 mode-layer fields across nine network and intermodal modes. Routing comparisons against an empirical mobility panel show low mean bias for walking, cycling, and car, while public transport results are interpreted more cautiously as schedule-based supply estimates sensitive to waiting and transfer assumptions. A Bavaria case study operationalizes the travel-time targets of the state's central-place planning system on the resulting fields and reveals strong modal differences in accessibility to basic services. The contribution is a reproducible framework for large-area multimodal accessibility analysis, with an aligned output structure designed for reuse in urban and regional analysis, planning support, and scenario modeling.

0.22

Time-Aware Assistive Navigation

cs.CV cs.RO ·2026-09-04 ·Masaki Kuribayashi, Zhongkai Shangguan, Eshed Ohn-Bar
top venuebenchmarkcode/data similarity 0.64
Abstract · ICRA 2026

Can interactive vision-and-language agents learn not just what to say but also \textbf{\textit{when}} to say it? Current language models rarely plan over whether and when to realize a real-time response to a user. However, providing accurate and timely support for human decision-making, such as when guiding visually impaired individuals through urban environments, requires careful real-time responsiveness--poorly timed responses can distract users or add unnecessary cognitive load. As a machine intelligence challenge for Multimodal Large Language Model (MLLM)-based agents, we introduce a large-scale multimodal benchmark for an egocentric, assistive navigation task in complex outdoor environments. Using this benchmark, we uncover a fundamental limitation of off-the-shelf MLLMs in delivering safe and time-sensitive navigation instructions, even with model fine-tuning on substantial amounts of data. We then demonstrate that a simple yet effective modification of the model, including direct supervision to predict the underlying reason for each instruction, yields significant performance gains across open-loop, closed-loop, and sim-to-real generalization settings. However, our analysis highlights persistent challenges in temporal reasoning, safety-critical object awareness, and relational and distance understanding. To advance the development of scalable assistive agents, we will release our simulation, benchmark, and code (available at the project website: https://timeli-icra.github.io/).

0.46

"I didn't know how to read a map, but now I can": TouchingSpace, an Audio-Haptic Map for Blind and Low-Vision Readers

cs.HC ·2026-09-28 ·Li Liu, Yihe Wang, Jiaming Qu, Ashmita Dua, David T. Lee +1 more
similarity 0.64
Abstract

Accessible map systems either make a layout explorable by hand or convey information through speech; few combine both to support pre-travel spatial understanding for blind and low-vision (BLV) people. We present TouchingSpace: a system that retrieves map data for an outdoor place and renders its surroundings as bounded regions at fixed trackpad positions. During exploration, users receive audio and haptic feedback and can ask a conversational agent open-ended questions. We conducted a user study with fourteen BLV participants who explored a place using TouchingSpace and reflected on the experience. We found participants used sound and vibration to locate places and speech to identify and describe them; the bounded surface supported discovery, revision, and spatial checks by hand; they expected this awareness to support future travel. TouchingSpace demonstrates how a laptop trackpad can support self-directed spatial exploration. These findings suggest accessible AI maps should ground conversation on bounded, user-controlled spatial surfaces.

0.09

Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild

cs.AI ·2026-10-08 ·Yuxuan Cao, Junlong Li, Hao Li, Junxian He
benchmark similarity 0.64
Abstract

Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intelligence: exploring unfamiliar environments, updating spatial understanding through interaction, and adapting actions based on feedback to sustain progress toward a sequence of goals. Existing benchmarks cover only a limited range of spatial layouts, scales, and traversal requirements. We introduce Mine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real-world locations. It comprises 180 tasks covering 30 such locations across 20 countries and regions on five continents, including 20 outdoor and 10 indoor settings. These settings span diverse spatial scales, layouts, terrains, and connectivity patterns, from Midtown Manhattan and rural Entrup to Santa Lucía Hill and Buckingham Palace. We select meaningful waypoints, such as landmarks, buildings, and rooms, and manually verify their accessibility. Each task provides a natural-language instruction specifying which waypoints to visit and in what order. Completing these tasks requires agents to find accessible routes and entrances, open doors, and move between levels using stairs and ladders, while monitoring their progress and recovering from navigation errors. Across eight evaluated state-of-the-art models, GPT-6 Astra achieves the highest success rate of 85.6%. However, the second-best model, Claude Opus 5.5, completes 73.9% of tasks, while the strongest evaluated open-weight model, DeepSeek-V4.1-Flash, reaches 23.9%, highlighting substantial room for improvement in the agentic spatial intelligence of current models. Comprehensive analyses and ablation studies on Mine Odyssey reveal current models' limitations and provide insights for advancing agentic spatial intelligence.

0.20

NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation

cs.CV cs.RO ·2026-10-14 ·Hochul Hwang, Jahir Sadik Monon, Soowan Yang, Shiven Umeshbhai Patel, Anh Nhat Hong Nguyen +7 more
benchmarkdatasetcode/data similarity 0.61
Abstract

Reliable recognition of accessibility-critical objects (e.g., audible pedestrian signals, door-activation buttons, and handrails) is essential for assistive navigation technologies that support safe, independent mobility for blind and low-vision (BLV) users. Yet these categories are underrepresented in existing large-scale vision datasets, and the few datasets that include them suffer from limited class coverage, inconsistent annotations, and poor diversity. We introduce NavAble, a large-scale dataset spanning 11 accessibility-critical object classes, combining 41K curated open-source images with 8K newly collected, densely annotated real-world images. To extend scale and diversity, we develop a synthetic data generation pipeline that renders high-fidelity 3D assets across 37 environments under diverse styles and viewpoints, yielding 452K frames with rich ground-truth annotations from 565 assets. The pipeline minimizes manual effort by combining automated web image crawling, VLM-based filtering, SAM 3D-based asset generation, and convenient camera trajectory configuration, enabling large-scale data generation at a diversity unattainable through manual collection. Experiments show that fine-tuning existing segmentation models on the NavAble Dataset achieves reliable accessibility-object segmentation (>80% mIoU on our held-out test set). We also demonstrate the effectiveness of synthetic data augmentation, which yields up to +3.5 mIoU over real-world-only fine-tuning. Beyond benchmark evaluation, validation on egocentric data from two mobility-assistive robots demonstrates improved perception reliability in real-world navigation scenarios. Together, NavAble establishes a scalable foundation for BLV navigation perception, directly addressing the data scarcity and embodiment gaps. The dataset and pipeline are publicly available at https://huggingface.co/datasets/NavAble/NeurIPS_2026_BLV.

0.24

Choose your own adventure: How data source selection shapes urban accessibility analyses

cs.CY stat.APecon.GN ·2026-10-19 ·Geoff Boeing, Jiyoon Kim, Daniele Simon-Thompson, Jaehyun Ha, Ruoyu Chen
published similarity 0.61
Abstract

Urban park accessibility shapes mental health, physical well-being, and community cohesion. Measuring it is thus central to urban and landscape planning. Although researchers have debated the impacts of different accessibility measures on analysis results, scant attention has been paid to variability from data source selection. Do the results of accessibility analyses vary depending on the dataset used to define park locations and attributes? We compare four widely used geospatial data sources - OpenStreetMap, Esri, SafeGraph, and The Trust for Public Land - by validating their data and measuring park access across two cities. These datasets purport to cover the same spatial features, but our findings reveal wide discrepancies in accessibility results, driven by differences in dataset completeness, coverage, and accuracy between the sources. We provide guidelines for selecting and carefully validating data for accessibility analyses, as these decisions directly impact the accuracy and equity of downstream urban park planning and decision-making.

0.41

Semantic- and Density-Aware Planning for Accessibility-Preserving Multi-Object Placement

cs.RO ·2026-08-17 ·Benno Wingender, Nils Dengler, Nicolas Busch, Sicong Pan, Maren Bennewitz
similarity 0.61
Abstract

Long-term manipulation planning requires robots to reason not only about immediate task success but also about how current decisions affect future interactions with the environment. In this context, household service robots may need to organize groceries in partially occupied shelves while using limited storage space efficiently and preserving access for subsequent placements. In this paper, we consider an online multi-object shelf-placement setting in which future objects arrivals are unknown. Existing approaches do not jointly address semantic organization, dense space utilization, and manipulator accessibility during sequential shelf filling. To address this gap, we propose Semantic-Dense Placement Planning (SDPP), an accessibility-preserving approach that ranks candidate poses using a semantic-density score combining inter-object semantic similarity with spatial proximity. An Accessibility Map (AM) further filters candidates unlikely to be reachable before motion planning and penalizes placements that reduce the remaining accessible workspace. Simulation experiments show that SDPP significantly improves semantic placement quality over state-of-the-art baselines and achieves the highest average shelf density, while the AM substantially reduces the time required to identify feasible placement poses. A qualitative real-world experiment demonstrates the applicability of our pipeline in a domestic shelf-storage scenario.

0.31

SituRecBench: Evaluating Frontier Models as Situated Recommendation Assistants in 3D Environments

cs.AI cs.CLcs.CVcs.IR+1 ·2026-11-07 ·Jingtong Yang, Dongding Lin, Jian Wang, Wenjie Li
top venuebenchmark similarity 0.61
Abstract · Accepted by NeurIPS 2026 Evaluations and Datasets Track

Recommender systems are transitioning from static retrieval engines into interactive assistants grounded in physical environments. However, current studies largely neglect the combined complexities of spatial navigation, visual grounding, and multi-turn social interaction. To bridge this gap, we introduce SituRecBench, an interactive benchmark for evaluating frontier vision-language models (VLMs) as situated recommendation assistants within high-fidelity 3D environments. We reformulate recommendation as a situated assistance process in which an assistant interacts with a user while sharing the user's egocentric view. Our benchmark simulates real-world shopping environments (e.g., supermarkets and home furnishings stores), requiring assistants to elicit preferences, ground recommendations in the environment, and guide the user to relevant locations through dialogue. The benchmark evaluates dialogue quality, recommendation utility, and navigation effectiveness through a unified interactive protocol. Extensive experiments reveal persistent challenges in spatial grounding and preference elicitation, highlighting limitations in their ability to integrate global contextual information with local observations. SituRecBench provides a rigorous foundation for advancing the development of situated assistants in the real world.

0.34