CS Paper Radar

Diffusion models for eye-gaze trajectory generation using position and velocity representations

Laxman Basnet, Alexander Szorkovszky, Pedro G. Lind, Anis Yazidi, Shailendra Bhandari

cs.CV cs.AIcs.NE ·submitted 2026-09-01 07:53 UTC · v2

Abstract

Eye-tracking data are expensive to collect, requiring specialized hardware and controlled laboratory conditions, and difficult to share due to privacy regulations, which together limit the quantity and diversity of gaze data available for research. We investigate unconditional gaze generation using diffusion models in position and component-wise velocity representations. The models are trained and evaluated on eight-second visual-search trajectories from 28 participants, using a split of 23 training participants and five held-out validation participants, with three training seeds. Evaluation includes kinematic distributions, trajectory geometry, event statistics, and a temporal Wasserstein GAN with gradient penalty as a learned comparator. Across eight position-derived features, mean Jensen-Shannon divergence is $0.016 \pm 0.004$ for the position model and $0.037 \pm 0.002$ for the velocity model, with standard deviations calculated across training seeds. These scores do not establish that representation choice caused the difference. A separate comparison using the same evaluation method shows that the ordering of the models varies across training seeds. The position model also produces smaller feature discrepancies than a kinematic Markov baseline. Differences in saccade counts between real and generated data depend strongly on the velocity threshold used to detect events. The results provide quantitative reference measurements for unconditional gaze generation on this visual-search task and show how evaluation choices affect conclusions about generated gaze.

Comment
13 pages, 9 figures

Closest papers

Nearest neighbours by meaning across the whole index.

DiffSSMEye: Conditional High-Frequency Eye Movement Synthesis with Structured State-Space Diffusion

cs.HC ·2026-11-12 ·Kamrul Hasan, Mehedi Hasan Raju, Oleg V. Komogortsev
similarity 0.72
Abstract · 14 pages, 5 tables, 2 figures

Synthesizing high-frequency eye movements requires preserving long-range temporal dependencies and subject-specific oculomotor dynamics. However, methods based on generative adversarial networks (GANs) and variational autoencoders (VAEs) can struggle to preserve these characteristics simultaneously. We introduce DiffSSMEye, a conditional denoising diffusion probabilistic model with bidirectional Structured State Space Sequence (S4) layers for generating five-second gaze-velocity sequences at 1,000 Hz. Generation is conditioned on a coarse gaze reference, user identity, and task, where identity and task embeddings are incorporated through feature-wise linear modulation. Identity-preservation, multi-resolution spectral, and integrated-drift losses complement noise prediction to constrain biometric consistency, frequency structure, and displacement error. Subject-disjoint evaluation on GazeBase across seven tasks yields mean real--synthetic embedding cosine similarities of 0.91--0.95. DiffSSMEye also achieves the lowest median spatial errors among these baselines for horizontal and random saccade tasks. These results demonstrate improved spatial accuracy and biometric feature consistency for subject-specific gaze synthesis.

0.12

GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction

cs.CV cs.LG ·2026-09-29 ·Sheng Zhao, Weikai Lin, Yuhao Zhu
top venue similarity 0.70
Abstract · Accepted at NeurIPS 2026. 25 pages

Egocentric gaze prediction enables many downstream applications but remains challenging, as human gaze is inherently stochastic. This stochasticity is constrained by structured temporal dynamics alternating between fixations and saccades, top-down influences from tasks, and bottom-up visual saliency. Based on this observation, we introduce GazeFlow, a framework that directly models gaze as a joint distribution of temporal gaze positions conditioned upon both top-down and bottom-up information. In particular, GazeFlow uses conditional flow matching (CFM): a learned velocity field iteratively transports a Gaussian noise sample into a plausible gaze trajectory drawn from this joint distribution. The velocity field is conditioned on bottom-up visual features extracted by a video encoder and on top-down task information obtained by globally querying these features. On standard datasets, GazeFlow achieves state-of-the-art performance on per-frame metrics, and the generated trajectories align better with human gaze temporal dynamics.

0.69

GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioning

cs.CV ·2026-09-15 ·Dongze Wu, David Colmenares, Fengting Yang, Jogendra Nath Kundu, Yao Xie +2 more
similarity 0.69
Abstract

Diffusion models are increasingly used to generate synthetic training data, but precise label control remains difficult when the conditioning signal is low-dimensional and coarse. Text-conditioned images are judged by broad prompt consistency, whereas supervised training requires precise correspondence between each image and its numerical label. This is challenging in eye tracking, where a 4D binocular gaze is expressed through subtle, spatially localized pupil and iris geometry. We introduce GazeDiT, a diffusion model that generates images for a requested 4D gaze through an internally constructed spatial condition that grounds the global gaze label in this local geometry. During training, a frozen SegFormer extracts pupil/iris geometry from diverse real images, allowing the model to learn realistic appearance conditioned on that geometry. At inference, a physical eye renderer samples gaze-consistent geometries by varying anatomy and camera state, enabling diverse synthesis without a source image. GazeDiT achieves substantially lower tail gaze-label error than other diffusion baselines, approaching the error of the same frozen gaze estimator on real images. Its generated data also improves the downstream eye tracker, reducing gaze error on difficult cases from 3.05° to 2.80° in the smallest cohort.

0.20

Motion-Based Tokenization for Cross-Dataset Egocentric Gaze Modeling

cs.CV ·2026-08-24 ·Virmarie Maquiling, Zhuojiang Cai, Enkelejda Kasneci
benchmark similarity 0.67
Abstract · 8 pages, 1 figure, 1 table

Gaze is increasingly used as an input signal for vision and multimodal models, yet no consensus exists on how to represent it across datasets. Raw traces preserve detail but are noisy and device-dependent, while coarse event labels are easy to model but can discard local motion structure. We formulate event-aligned, fixed-horizon angular displacement as an interpretable, event-conditioned motion vocabulary and compare it with event-only, spatial, absolute-angle, learned vector-quantized, and continuous representations. To assess transfer alongside target predictability and token collapse, our evaluation combines next-token prediction with target-domain regret, low-order target references, paired bootstrap, order sensitivity, motif overlap, and frozen structural probes. In an event-aligned headset benchmark, angular-motion tokens have lower target-domain regret than frozen-codebook VQ tokens in one transfer direction, while the reverse direction is inconclusive. The probes reveal complementary representation properties, and event-only tokens show that low perplexity can retain little motion information. On a third egocentric dataset, a matched comparison of I-VT, native, and frame-span interfaces shows that event construction materially changes transfer: native events have the lowest regret into EGTEA, while frame-span events have zero motif overlap and fail severely as a source. Motion-based tokenization therefore provides a compact representation for event-aligned egocentric gaze streams, while the evaluation identifies how target predictability and event construction shape cross-dataset conclusions.

0.51

EyeTAG: Eye Trajectory-Aware Gaze Estimation

cs.CV cs.LG ·2026-10-01 ·Jungmin Lee, Niamat Ullah, Yoseob Han
publishedcode/data similarity 0.67
Abstract · Accepted to BMVC 2026

Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.

0.38

EyeMakeYou: Identity-, Task-, and Subjective-State-Conditioned Diffusion for High-Frequency Gaze Synthesis

cs.HC cs.CV ·2026-09-03 ·Kamrul Hasan, Mehedi Hasan Raju, Oleg V. Komogortsev
similarity 0.66
Abstract · 20 pages, 3 tables, 3 figures

Eye movement biometrics (EMB) is an emerging behavioral modality for user authentication, particularly in virtual- and augmented-reality systems, where gaze dynamics contain distinctive subject-specific features. However, robust EMB systems require diverse, high-quality gaze recordings that are expensive to collect and often unavailable at the scale needed for model development. Generative models can mitigate data scarcity, but existing methods either synthesize generic gaze behavior or personalize signals primarily by identity, without jointly representing the user's task and subjective state. Consequently, generated signals may appear visually realistic while failing to retain the behavioral properties required for biometric applications. To address this limitation, we propose EyeMakeYou, a multi-conditional denoising diffusion framework for subject-specific, high-frequency gaze synthesis. EyeMakeYou generates 5-s, 1000-Hz bivariate gaze-velocity sequences from an identity-removed reference trajectory and conditions the denoising process on an identity embedding, a task embedding, and self-reported ratings of overall difficulty, mental tiredness, and eye tiredness. Its objective combines diffusion noise prediction and identity preservation with multi-resolution spectral, drift-consistency, and event-weighted local-smoothness losses. Experiments on GazeBase show that EyeMakeYou achieves higher median spatial accuracy and greater real--synthetic similarity in the embedding feature space than the existing generative approaches, while retaining selected task-dependent associations between subjective reports and oculomotor features. These findings support conditional diffusion as a practical approach for augmenting gaze datasets for biometric and interactive applications.

0.48

An Eye-Tracking Dataset for Viewing Distance Categories in Real-World Scenarios

cs.HC ·2026-08-29 ·Dohwa Kim, Yejin Choi, Seungbok Lee, Chi Yoon Jeong, Eunji Park
dataset similarity 0.65
Abstract · 22 pages, 6 figures, Manuscript submitted to Scientific Data

Estimating viewing distance from gaze behavior is essential for understanding user intent and enabling distance-aware interactive systems. However, most existing eye-tracking datasets have been collected in constrained settings, such as laboratory environments or static tasks. Consequently, they only partially capture viewing behaviors in real-world situations where viewing distance changes with natural head and body movements. We introduce GazeDepth, an eye-tracking dataset collected from 19 participants using a wearable tracker during tasks reflecting real-world scenarios. GazeDepth includes fixed-distance viewing scenarios with constant observer-target distances at near (33 cm), middle (50 cm), and far (300 cm), as well as variable-distance viewing scenarios in which participants shift gaze among targets at different depths in indoor and outdoor environments. The dataset provides synchronized gaze data, pupil size, 3D eye-vectors, and head-motion signals, along with distance labels. Statistical analyses showed that distance-related gaze features, such as vergence angle and estimated viewing distance, differed consistently across viewing-distance categories. In addition, classification models trained on GazeDepth further demonstrated that the dataset captures gaze characteristics that distinguish the three viewing-distance categories, supporting gaze-based distance inference and distance-aware interaction in realistic scenarios.

0.26

Drift Calibration in Geometric Eye Tracking Systems

cs.CV ·2026-08-30 ·Jiaqi Liu, Zixuan Wang, Yuhong Zhang, Dingkang Liang, Jane Hanqi Li +2 more
benchmarkdataset similarity 0.65
Abstract · 11 pages, 6 figures, 3 tables

Geometric eye trackers can provide the spatial accuracy required for gaze-based interaction and multimodal studies, but their measurements remain sensitive to residual session-specific calibration error. Research on correcting this error is difficult to compare because methods are typically evaluated with different devices, target layouts, and error definitions. We present a calibration-focused dataset containing 163 trials from 12 participants, with separate 18-point fitting and 32-point test grids, and use it to evaluate global, local, and composite correction functions under a common spatial-extrapolation protocol. We further introduce a lightweight neural refiner that combines ranked predictions from complementary calibrators. On this controlled dataset, post-vendor correction reduces the mean angular error from $1.53^\circ$ to $1.03^\circ$ with the strongest classical composite and to $0.96^\circ$ with the refiner. In a closed-loop gaze task, lower residual error is associated with higher performance across four online correction conditions. These results provide a reproducible data-quality benchmark for using gaze as a behavioral signal in interactive modeling.

0.21