CS Paper Radar

AOI-Net: Structural Face AOI-Guided Eye-Gaze Track Representation Learning for Autism Spectrum Disorder Detection

Zhanpei Huang, Binbin Sun, Jialiang Chen, Yiou Wang, Taochen Chen, Yuzhu Ji, Yiqun Zhang, Yiu-Ming Cheung

cs.CV cs.AIcs.ETcs.MM ·submitted 2026-08-29 14:26 UTC · v1
publishedcode/data

Abstract

Eye-movement tracking has emerged as a promising non-invasive approach to Autism Spectrum Disorder (ASD) screening, with systematic differences in attentional allocation and revisit behaviors observed during socially interactive tasks. Existing computational methods typically characterize eye-movements using discrete gaze trajectories and fixation events, yielding representations dominated by short-range temporal dynamics and limiting models that primarily emphasize long-range dependencies. Meanwhile, gaze behavior is naturally organized across semantically meaningful Areas of Interest (AOIs), whose attention allocation and transitions provide important structural cues, yet their relationships are rarely modeled explicitly. To address these limitations, we propose a structural face AOI-guided Eye-Gaze Track Network (AOI-Net) that jointly models short-term temporal dynamics and AOI-level structural organization. A network gating mechanism adaptively integrates the complementary temporal and structural representations according to their contributions to gaze-behavior characterization. To mitigate the pronounced class imbalance commonly encountered between individuals with ASD and Typically Developing (TD) participants in clinical datasets, class-distribution-aware learning is further employed to facilitate discriminative embedding learning under skewed class distributions. Experiments on a unique and large-scale clinical eye-tracking database comprising eight stimulus subsets and more than 1,300 participants show that AOI-Net consistently outperforms state-of-the-art methods. The proposed framework also enables interpretable gaze-behavior modeling and provides a practical basis for scalable AI-driven ASD screening in real-world healthcare. The code is available at https://github.com/Zhanpei-ai/CIM-AOI-Net/tree/main/Code

Comment
14 pages, 8 figures
Journal ref
IEEE Computational Intelligence Magazine, 2026
DOI
10.1109/MCI.2026.3729548

Closest papers

Nearest neighbours by meaning across the whole index.

ATHENA: Hypergraph-Based Learning for Toddler Autism Classification from Eye-Tracking

cs.AI ·2026-11-07 ·Quang-Anh N. D., Duc Pham Minh, Thao Pham Phuong, Minh-Anh Nguyen, Thuy Linh Nguyen Thi +1 more
similarity 0.69
Abstract

In this work we proposed a architecture designed for the classification of ASD in toddlers using eye-tracking data, addressing the critical need for early and precise detection. The proposed model incorporates FMMix, a feature-level augmentation technique that bolsters the model's learning capacity, and in this work we proposed the ATHENA, which captures intricate feature relationships via an adaptive hypergraph framework. Evaluated on two datasets dataset of gaze coordinates-the model achieves state-of-the-art performance, with AUC scores of 99.80% and 99.86%, respectively. It also exhibits high accuracy of 98.45% and 98.56%, sensitivity 98.01% and 98.10%, and specificity 98.74% and 99.02%, respectively. Demonstrating its superior ability to identify subtle gaze patterns associated with ASD, these results significantly surpass existing methods, highlighting the model's potential for clinical ASD screening. Future efforts will focus on validation across broader populations and integration with multi-modal data to further refine diagnostic accuracy.

0.18

ASD-FEAT: A Multi-Modal Infant Video-Derived Dataset for Early ASD Risk Prediction

cs.CV ·2026-10-02 ·Sidrah Liaqat, Halil Helvaci, Sen-Ching Cheung, Chongruo Wu, Dongjie Chen +2 more
similarity 0.66
Abstract

Accurate early screening for Autism Spectrum Disorder (ASD) is a precursor to timely intervention, which is critical for improving cognitive and behavioral outcomes. We present ASD-FEAT (ASD - Feature Extraction And Tracking), a multimodal dataset derived from video recordings of infant-adult interaction sessions. The key contribution of ASD-FEAT is the combination of longitudinal coverage from infancy through 36 months, repeated interaction sessions, clinically validated developmental outcomes, expert frame-level behavioral annotations, and privacy-conscious multimodal feature representations. To the best of our knowledge, existing ASD behavioral datasets do not jointly provide these characteristics at comparable scale. To demonstrate the utility of ASD-FEAT, we use it to evaluate a computer-vision-based end-to-end pipeline relying on machine learning techniques to automatically identify ASD risk. ASD-FEAT integrates both expert-defined and deep-learned features, including face and eye landmarks, facial action units, gaze direction, head position, mel-spectrogram audio representations, and optical flow, to identify behavioral markers of social interaction. Our automated pipeline achieves an ASD classification accuracy of 76.2% and an Area Under the Receiver Operating Characteristic (AUROC) of 0.82, compared to classifiers trained on manually labeled behaviors, which yielded 81.3% accuracy and an AUROC of 0.88. We further introduce a within-visit partner contrast: a per-visit signal contrasting examiner-directed and parent-directed social behavior which, when added to the classifier, lifts the fully automated Look Face + Smile and Look Face + Vocal configurations to Matthews correlations of 0.49 and 0.45 respectively, exceeding the human-coded single-partner baseline of 0.42.

0.20

The Language of Looking: Motion Tokens Reveal Domain-Dependent Local Structure in Gaze Sequences

cs.LG ·2026-10-12 ·Virmarie Maquiling, Enkelejda Kasneci
top venue similarity 0.64
Abstract · 18 pages, 6 figures, EMNLP26

Human gaze is the primary high-bandwidth interface through which we parse the semantics of our environment. While often modeled as a sequence, the degree to which gaze exhibits transferable local structure across domains remains underexplored. Using motion tokenization as a diagnostic lens across four diverse datasets, we investigate the transferability of local sequence structure. We demonstrate that while gaze can share a reusable motion-token vocabulary, identical motion tokens are organized into different local transition patterns depending on the dataset and task context. Our results show the limitations of naive data-pooling under low-semantic motion-token representations and provide a framework for evaluating behavioral sequences through information-theoretic diagnostics.

0.35

UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

cs.CV cs.AI ·2026-09-25 ·Lei Xin, Zeheng Wang, Jiayin Zhu, Shihong Huang, Fanhu Zeng +4 more
top venue similarity 0.62
Abstract · Accepted by ACM'MM 2026

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term developmental outcomes. However, existing ASD recognition methods are often constrained by the scarcity of diagnostic text data, forcing them to rely mainly on visual analysis and limiting their ability to model clinically meaningful semantic reasoning. To address this challenge, we propose UniAR, a unified framework enhanced by multi-granularity prompt learning for robust ASD recognition under heterogeneous data variations. Specifically, UniAR leverages a large multimodal model to generate hierarchical diagnostic descriptions at the word, phrase, and sentence levels, compensating for the lack of paired clinical reports. To align the generated semantics with visual evidence, we further design a Mixture-of-Experts-based Multi-Scale Alignment Module, which dynamically matches vector-quantized visual prototypes with semantic representations at corresponding granularities. Extensive experiments on four benchmarks covering brain MRI and facial expression scenarios show that UniAR consistently outperforms existing state-of-the-art methods, achieving average accuracies of 75.9\% on MRI benchmarks and 91.6\% on facial benchmarks, while improving average Accuracy on MRI benchmarks by 1.5 percentage points and average Accuracy on facial benchmarks by 1.2 percentage points over baselines. These results demonstrate that UniAR offers a robust and interpretable framework for ASD screening under semantic scarcity.

0.40

From Portraits to Pathology: Probing Diagnostic Overreach in Vision-Language Models

cs.CV cs.AI ·2026-09-02 ·Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel, Hansa Meghwani, Jyotika Singh +6 more
similarity 0.61
Abstract

Many medical conditions require clinical assessment of symptoms, history, and functioning rather than judgments from appearance. We study whether vision-language models (VLMs) abstain from facial attribution of autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), major depressive disorder (MDD), and generalized anxiety disorder (GAD). To support this study, we construct PARITY (Paired Assessment with Reused Identity), a demographically balanced dataset for testing whether changing facial expressions alters diagnostic judgments, while holding identity and other visual factors as constant as possible. Our comprehensive evaluation of 27 open- and closed-weight models reveals inconsistent abstention across expressions, conditions, and task formats. Models turn facial expressions into unsupported medical judgments: this behavior is widespread among open-weight VLMs and extends to safety-trained frontier models that abstain on individual portraits yet make diagnostic judgments when comparing portraits of the same identity. Safeguard checks reveal further gaps: clinical instructions do not consistently prevent these judgments, and a guardrail model rarely flags paired requests as unsafe. Semantic analysis further shows that most baseline open-model refusals fail to explain both the insufficiency of facial evidence and the need for clinical assessment. Unsupported judgments and seemingly authoritative explanations can cause distress, reinforce stigma, and offer false reassurance, underscoring the need for stronger safety measures in model training and deployment.

0.32

CGM: Chaos-Guided Graph Attention and Expert Routing for Autism Stereotypy Detection

cs.CV cs.LG ·2026-11-19 ·Hafsa Aslam, Yue Li, Saba Aslam, Abdur Rasool, Qiao Pan +2 more
similarity 0.61
Abstract

Skeleton graph attention for autism video analysis weights joints by content alone, giving an erratic wrist the same standing as a still one. Stereotypy, however, is defined by how unpredictably movement unfolds rather than how much of it occurs: in our cohort, every amplitude-based kinematic descriptor falls in stereotyped clips, while windowed Sample Entropy of angular velocity alone rises (Hedges' g = +1.05). We compute this irregularity as a per-joint, per-frame tensor that structurally drives computation. The proposed chaos-guided graph attention and mixture-of-experts model (CGM) couples two mechanisms. First, a learned pairwise term added to the graph-attention logit concentrates attention on irregular joints and re-ranks the remaining edge weight toward jointly irregular pairs; its unconstrained coefficient neither vanishes nor inverts, and zeroing the tensor realigns attention in 94% of clips. Second, a gate conditioned on the same signal routes each frame to two of four experts, whose weights track frame-level irregularity. Under subject-grouped five-fold cross-validation on 116 clips, CGM reaches 0.930 balanced accuracy, exceeding the graph-attention baseline by +0.187; the component ordering replicates on the public MMASD dataset, and on held-out Panoptic Studio sessions, CGM keeps 0.86 specificity without retraining, against at most 0.35 for chaos-free baselines. In ablation, routing without irregularity yields no reliable gain, whereas chaos modulation alone contributes +0.132. A severity index from the same tensor reaches 0.87 split-half reliability, separates the cohorts with Hedges' g = +1.08, and stays consistent within children across therapy recordings, supporting video-only monitoring of stereotypy during intervention.

0.29

Eye, Robot: Learning Expressive Gaze Dynamics via a VLM-Guided Transformer-VAE

cs.RO cs.AI ·2026-10-14 ·Yuhang Hu, Junlin Lyu, Yuan Fang, Xiaoying Shen, Lechen Zhang +4 more
benchmark similarity 0.60
Abstract

Gaze coordinates attention in human interaction through coupled eye, head, and eyelid motion. A humanoid robot must choose where to look while generating a suitable movement toward that target; fixed tracking rules alone do not model the temporal structure of human gaze. We separate these tasks with an asynchronous vision-language model that selects a scene referent, a reactive perception module that maintains fixation, and a Realistic Gaze Transformer (RGT) that generates target-conditioned motion. RGT is a Transformer-VAE trained on markerless human recordings retargeted to the robot's 11-DoF joint space: self-supervised pretraining encodes recent full-head history, and autoregressive fine-tuning exposes the model to its own predictions during rollout. At inference, a four-dimensional binocular-eye trajectory conditions coordinated neck and eye motion; an auxiliary blink predictor triggers eyelid templates during playback. On a two-second benchmark in which all controllers receive the same ground-truth target, RGT-Base obtains the lowest reported overall seven-joint MSE, and physical demonstrations show the integrated targeting and control pipeline. The evidence establishes robot-space kinematic fidelity and real-time RGT inference; perceived naturalness and open-vocabulary target-selection accuracy remain questions for controlled evaluation.

0.14

EyeTAG: Eye Trajectory-Aware Gaze Estimation

cs.CV cs.LG ·2026-10-01 ·Jungmin Lee, Niamat Ullah, Yoseob Han
publishedcode/data similarity 0.60
Abstract · Accepted to BMVC 2026

Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.

0.38