CS Paper Radar

An Affordable AI-Integrated Smart Cane for Multimodal Mobility Assistance of Visually Impaired Users

Ali Akarma, Adeel Ahmad, Toqeer Ali Syed

cs.AI cs.CVcs.CYcs.RO ·submitted 2026-09-12 08:45 UTC · v1

Abstract

Visual impairment affects over 2.2 billion people worldwide, yet conventional white canes cannot detect elevated hazards or provide semantic environmental context. Existing AI-assisted navigation systems typically rely on expensive hardware or cloud connectivity, limiting accessibility in resource-constrained settings. This paper presents an affordable (\$88 USD), fully offline AI-integrated smart cane designed for multimodal mobility assistance on an ultra-low-power Raspberry Pi Zero 2W. The system fuses RGB vision sensing with Time-of-Flight (ToF) distance estimation, pairing an INT8-quantized SSD MobileNet V1 model with distance-aware vibrotactile feedback and real-time audio alerts. To ensure operational robustness on constrained hardware, a multiprocessing architecture isolates sensor acquisition, neural inference, and haptic feedback into independent processes with fail-safe sensing support. Experimental evaluation across indoor mobility scenarios demonstrates a macro-averaged F1-score of 0.82 (precision: 0.85, recall: 0.81), a mean end-to-end latency of 330\,ms, and a peak power draw of 2.8\,W. A preliminary usability study with 12 participants (SUS: 78.5, NASA-TLX) demonstrated positive user perception and enhanced obstacle awareness. The proposed prototype validates the feasibility of deploying privacy-preserving, edge-native assistive intelligence for cost-sensitive mobility assistance.

Closest papers

Nearest neighbours by meaning across the whole index.

From White Canes to Edge AI: A Survey of Assistive Navigation Technologies for Blind and Low-Vision Individuals

cs.HC cs.CV ·2026-10-31 ·Abtin Zandi, Arian Ashrafi, Sarvin Nami, Seyed Sina Negahban, Amirabbas Entezari +6 more
survey similarity 0.70
Abstract

Independent mobility is a foundational requirement for the autonomy of blind and low-vision (BLV) individuals, yet the assistive technologies that support it have historically evolved slowly. This survey systematically reviews 108 peer-reviewed studies published between 2008 and 2026, organizing the field into four technological eras: (i) a foundational era of passive mechanical aids, namely the white cane and the guide dog; (ii) a hardware era of electronic travel aids built on ultrasonic, infrared, and satellite positioning sensors that alert users to proximity but lack semantic understanding; (iii) a deep learning era in which convolutional and transformer-based detectors introduced context-aware scene understanding, initially tethered to cloud infrastructure; and (iv) a contemporary edge-native era defined by model compression, quantization, hardware acceleration, and multi-sensor fusion that relocate intelligence onto resource-constrained wearable and mobile devices. For each era we analyze representative systems, sensing modalities, computational architectures, and reported performance, and we consolidate the field's model-optimization techniques into a single technical reference spanning four detector families and twelve compression strategies. Across the corpus, we find a persistent mismatch between rapid algorithmic progress and slow real-world adoption, driven by cognitive overload in user interfaces, fragmented evaluation protocols, and the scarcity of studies involving BLV participants. We conclude with open challenges, including seamless indoor-outdoor transition, standardized benchmarks, privacy-preserving perception, and the emerging role of multimodal foundation models, and outline a roadmap for socially acceptable, user-centered assistive navigation.

0.23

Real-Time Obstacle Detection and Distance Estimation on Smartphones for Visually Impaired and Distracted Pedestrians

cs.CV cs.AIcs.PFcs.SE ·2026-10-07 ·Abtin Zandi, Sarvin Nami, Roza Ganjipour, Bardia Sabagh Kermani, Amirreza Tarabkhah +1 more
similarity 0.63
Abstract · 10 pages, 3 figures, 4 tables

According to the World Health Organization, at least 2.2 billion people worldwide live with visual impairment, including tens of millions with severe vision loss. Traditional aids such as white canes or guide dogs remain costly, limited in capability, and inaccessible to many. At the same time, smartphone distraction in public spaces has been linked to elevated risks of collisions with vehicles, trips, and falls. These dual challenges motivate lightweight, smartphone-based solutions that combine accessibility with low latency. We present a Kotlin-based Android application that uses a YOLOv8 detector, retrained to recognize eighteen common urban obstacle classes, to deliver real-time obstacle alerts for both user groups. The detector was optimized with pruning, quantization, and early-exit branches, and deployed through the Android Neural Networks API (NNAPI), which can dispatch inference to GPU, NPU, or DSP accelerators when available. On a mid-range smartphone, these optimizations reduced average inference latency from about 3.2 s for the off-the-shelf YOLOv8n to 0.39-0.48 s; the recommended configuration (INT8 quantization with early exit) runs in 0.42-0.48 s while retaining a recall of 0.81. The application combines the optimized detector with ARCore-based or monocular focal-length distance estimation, so it works with a single smartphone camera and requires no LiDAR, depth camera, or other specialized sensor. Multimodal feedback, including Text-to-Speech guidance for visually impaired users and vibration alerts for distracted pedestrians, supports use across different contexts.

0.23

InSight: A Mobile Assistant Helping Visually Impaired People Navigate Unfamiliar Environment

cs.HC ·2026-10-13 ·Junwei Zheng, Jiale Wei, Di Wen, Yufan Chen, Ruiping Liu +4 more
similarity 0.63
Abstract

Navigating unfamiliar outdoor environments is one of the most demanding everyday tasks for people with visual impairments (PVI). However, existing navigation systems cannot simultaneously satisfy the needs of informative perception and precise localization during navigation. This paper explores the core navigation demands of PVI through a preliminary study. Based on the results of the pre-study, we present the design of InSight, a mobile assistant that combines scene descriptions with precise localization to support PVI navigating unfamiliar outdoor environments. The offline mode of InSight runs entirely on device for real-time navigation, while the online mode streams camera frames to a self-hosted server for arbitrary object detection and accurate localization. Finally, a user study with 21 visually impaired participants validates that InSight could assist users in navigating unfamiliar outdoor environments with increased success rate and confidence compared to mainstream navigation applications like Apple Maps.

0.18

AVD: A Multi-Sensor Teleoperated Wheelchair Platform with Safety-Aware Control

cs.RO ·2026-11-16 ·Namai Chandra, Madhur Thareja, Boby George
similarity 0.61
Abstract

Electric wheelchairs predominantly rely on manual joystick interfaces, which present critical barriers to independent mobility for individuals with severe motor and visual impairments. This paper presents AVD (Autonomous Vehicle Drive), a dependable teleoperation architecture for assistive mobility that unifies perception, communication, embedded control, and remote supervision in a single low-cost platform built entirely from commercially available hardware. The central engineering contribution is a safety-oriented design framework: control authority follows a static hierarchy, a manual emergency stop and the occupant's joystick pre-empt remote commands, and the on-board sensing layer has stop-only authority. The compute design partitions primary and safety functions and specifies a heartbeat-driven safe-state transition for the fault cases considered. The sensing stack this framework depends on is characterized against measured data: per-channel acquisition latency, a 2,025-scan LiDAR degradation campaign spanning five target configurations and a control under water mist, airborne dust, and direct aperture occlusion, and attitude measurements across static inclines to 47 degrees. The lowest pooled environmental-condition mean retained 76.6% of the clear-baseline forward-sector yield, and no scan lost the sector completely. A logged vibration-induced inertial dropout occurred while LiDAR and camera logging continued, illustrating complementary channel behavior. The result is a documented prototype and an explicit validation path for a deployable wheelchair implementation.

0.24

Cyber-Physical Systems for Accessibility and Ability Augmentation: Bridging Diverse Communities

cs.HC ·2026-08-19 ·Shuchang Xu, Riku Arakawa, Mina Huh, Nandi Zhang, Tianyu Zhang +9 more
published similarity 0.61
Abstract · UIST 26 Workshop

The powerful convergence of wearables, robotics, extended reality, and smart environments is expanding the design space for cyber-physical systems (CPS) that support and augment human abilities in daily life. By sensing real-world contexts, modeling user needs, and providing situated assistance, these systems can improve accessibility for people with disabilities while enhancing broader human abilities such as perception, memory, learning, and mobility. However, realizing this potential requires addressing key challenges in context sensing, user modeling, adaptive interaction, privacy, and evaluation to ensure that CPS are reliable and effective in real-world contexts. This workshop will bring together researchers and practitioners across HCI, AI, wearables, robotics, XR, smart environments, accessibility, and ability augmentation to examine shared strategies and challenges for designing accessibility- and ability-centered CPS. Through panel discussions, interactive demos, and mixed-group design activities, participants will identify recurring design principles, technical challenges, and future directions for CPS that support and augment human abilities in real-world settings. For details, please visit: https://cps4all.github.io.

0.31

"I didn't know how to read a map, but now I can": TouchingSpace, an Audio-Haptic Map for Blind and Low-Vision Readers

cs.HC ·2026-09-28 ·Li Liu, Yihe Wang, Jiaming Qu, Ashmita Dua, David T. Lee +1 more
similarity 0.60
Abstract

Accessible map systems either make a layout explorable by hand or convey information through speech; few combine both to support pre-travel spatial understanding for blind and low-vision (BLV) people. We present TouchingSpace: a system that retrieves map data for an outdoor place and renders its surroundings as bounded regions at fixed trackpad positions. During exploration, users receive audio and haptic feedback and can ask a conversational agent open-ended questions. We conducted a user study with fourteen BLV participants who explored a place using TouchingSpace and reflected on the experience. We found participants used sound and vibration to locate places and speech to identify and describe them; the bounded surface supported discovery, revision, and spatial checks by hand; they expected this awareness to support future travel. TouchingSpace demonstrates how a laptop trackpad can support self-directed spatial exploration. These findings suggest accessible AI maps should ground conversation on bounded, user-controlled spatial surfaces.

0.09

Edge AI on Constrained Devices for Binary Sleep-Wake Classification in Dynamic Environments

cs.LG ·2026-09-24 ·Stefan Reitmann, Lena Oden
similarity 0.60
Abstract

This paper presents an Edge AI-based system for detecting sleep and wake states in non-stationary mobile environments using resource-constrained embedded hardware. Conventional approaches relying on accelerometer-based activity metrics are highly susceptible to motion and vibration artifacts and are limited by strict compute and energy budgets of wearable and IoT devices. To address these challenges, a multimodal pipeline is designed and implemented on an ESP32-S3 microcontroller. The system combines inertial sensing for head movement analysis and visual pose classification. A dual-core architecture with FreeRTOS enables parallel execution of real-time data acquisition and on-device inference. Sleep detection follows a two-stage strategy: low-movement detection over a temporal window, followed by visual validation of poses. Experimental results show accuracies of 96.5% for motion-based detection and 89% for pose classification, yielding robust binary sleep-wake classification. Field tests confirmed feasibility in representative mobile scenarios. The results demonstrate that privacy-preserving, local sleep detection is achievable on edge hardware through careful co-design, while highlighting limitations in sensing intrusiveness, dataset scale, and system integration.

0.26

Assisted Spatial Cognition Through Vision-Language Models

cs.AI ·2026-09-11 ·H. Riaz, J. B. Fernandez, I. Mills, D. Hickey, F. Cleary +1 more
similarity 0.59
Abstract

Multimodal AI, powered by Large Language Models (LLMs) and Vision-Language Models (VLMs), is transforming assistive technologies by enabling simultaneous processing of visual and textual data. This advancement holds significant promise for over 43 million visually impaired and neuro-divergent individuals worldwide who face persistent challenges in navigating indoor and outdoor environments due to limited spatial awareness and insufficient environmental cues. Existing navigation aids often lack comprehensive 3D scene understanding, relying on constrained route-based strategies that hinder user autonomy. In this paper, we introduce a novel end-to-end framework that integrates LLMs, VLMs and digital twin technologies to deliver a spatially cognitive navigation support for visually impaired and neuro-divergent users. Our system captures video input via standard mobile phone cameras, and employs SLAM3R to generate dense 3D point clouds from monocular RGB sequences in real-time. Our custom post-processing algorithm ensures accurate point cloud alignment across multiple viewpoints without requiring predefined reference points. This enhances the capabilities of SpatialLM to produce structured 3D representations, including architectural elements and oriented object bounding boxes. The enriched spatial data is then processed by a locally deployed LLM, which interprets 3D contexts to generate detailed scene descriptions and precise distance measurements between users and surrounding objects. We evaluated our approach across diverse video scenarios featuring various perspectives, looped walking views and captured in multiple environments. The evaluation results demonstrate consistent accuracy in 3D scene interpretation and object localisation, underscoring the potential of our system as a transformative assistive navigation solution that combines advanced visual perception with spatial reasoning

0.11