CS Paper Radar

Measuring Browser Webcam Gaze Honestly: A Capture-Clock Methodology and Open Reference Implementation

Chi-Sheng Chen, Gabriel A. Brat

cs.HC cs.CVeess.SP ·submitted 2026-08-12 02:14 UTC · v2
publishedbenchmarktheory

Abstract

Browser-based webcam gaze trackers are increasingly used for crowd-scale data collection and in clinical settings where lab eye trackers are impractical, but the reported latency numbers may not represent real world functionality. The common practice of timestamping each gaze sample when it is emitted, rather than when its source frame was captured, makes the measured inference latency read about $0\,$ms no matter how slow the engine really is. We show how to measure it honestly, recovering a per-frame capture clock from the browser's \texttt{re\-quest\-Video\-Frame\-Call\-back} (rVFC) API (\texttt{captureTime} where the browser exposes it for local camera streams, else \texttt{presentationTime}, in which case every recovered latency is a verifiable lower bound): exact source-frame pairing through a per-frame queue for engines that expose their inference pipeline, and a further lower bound for engines that do not, such as WebGazer. We release an open TypeScript implementation and benchmark harness, demonstrated on two interchangeable engines: WebGazer and a new FaceMesh+KRR pipeline.

Comment
Accepted at DEMI 2026 (MICCAI 2026 Workshop on Data Engineering in Medical Imaging). Final version to appear in Springer LNCS

Closest papers

Nearest neighbours by meaning across the whole index.

OpenSAL360: Open-Source Crowdsourcing Platform for Omnidirectional Video Saliency Collection

cs.CV ·2026-09-18 ·Alexey Bryncev, Andrey Moskalenko, Kira Shilovskaya, Ivan Kosmynin, Dmitriy Vatolin
publisheddatasetcode/data similarity 0.61
Abstract · Accepted by ACM MM 2026

Omnidirectional video saliency prediction plays an important role in many immersive multimedia applications, including viewport-adaptive streaming and compression, foveated rendering, mesh simplification, perceptual quality assessment. Yet progress in this area remains constrained by the cost and complexity of collecting eye-tracking data with VR headsets, which makes large-scale dataset creation difficult to extend. We present OpenSAL360, the first open-source platform for scalable, low-cost 360° video saliency collection. Unlike conventional VR-based protocols, it requires only a standard screen, mouse, and internet connection, enabling parallel saliency data collection from common crowdsourcing assessors without specialized hardware. We validate our collection protocol against seven well-established VR eye-tracking datasets and conduct ablation studies on key interface, pre-, and post-processing parameters. To demonstrate the effectiveness and scalability of the proposed methodology, we collect and publicly release a saliency dataset covering 500 omnidirectional videos annotated by 2,000+ crowdsourcing assessors, making it, to the best of our knowledge, the largest dataset in this field. We make OpenSAL360 publicly available at https://github.com/msu-video-group/OpenSAL360.

0.51

PROWBench: Do Video Models Render What the Program Specifies?

cs.CV ·2026-10-01 ·Zheng-Hui Huang, Guixu Lin, Yu-Ju Tsai, Jian-Kai Zhu, Fengbo Lan +4 more
benchmarkcode/datahot directionhighly cited similarity 0.60
Abstract · Project page: https://alaya-lab.github.io/PROWBench

Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D, and bounding boxes. The benchmark covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. Grounded in these records, PROWBench evaluates entity control, long-horizon memory, and, with two VLM-based metrics, Logic-Render Alignment and Interaction Success Rate, adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.

0.57

ASTRA: Low-Overhead Runtime Architecture for STReam Adaptation in Video Analytics

cs.NI ·2026-09-07 ·Mahshid Ghasemi, Zoran Kostic, Javad Ghaderi, Gil Zussman
similarity 0.60
Abstract

Real-time video analytics is crucial for smart city applications and cloud-connected vehicle control. To improve analytics accuracy, it is desirable to process the video at the highest resolution and frame rate. However, due to limited resources, streaming and processing video at the highest resolution and frame rate from all cameras is not feasible and adversely affects the analytics latency. Intelligent adaptation of cameras' resolutions and frame rates based on network conditions and the video content is crucial in order to optimize the performance. In this paper, we present ASTRA, a low-overhead runtime architecture for online adaptation of live camera analytics at the edge. ASTRA can execute various online algorithms as a black box. We deployed ASTRA in the realistic NSF COSMOS testbed and uniquely assessed its real-time performance using COSMOS' street-level cameras. We further evaluated ASTRA with up to eight emulated cameras by streaming a comprehensive video dataset under real-world network conditions. We used ASTRA's architecture to evaluate the practical performance of several classes of adaptation algorithms, including theoretical and empirical methods. The results indicate that ASTRA can provide system reliability (i.e., the probability of meeting accuracy and latency requirements) of more than 90% while maintaining performance within a deviation of less than 10% from optimal offline performance with average GPU utilization overhead of around 2% per camera.

0.13

TimeLineXR: An Interactive Exploration of Time in Vision-Language Models

cs.CV ·2026-10-27 ·Nidham Tekaya, Markus Passecker, Matthias Zeppelzauer
code/data similarity 0.60
Abstract

Recently, it has been shown that vision-language models (VLMs) have a certain intrinsic understanding of time. They can estimate the temporal origin of objects and encode time along a low-dimensional, non-linear manifold inside their embedding space~\cite{tekaya2025matter}. That result is quantitative and reproducible, but the structure itself stays out of reach for end users: they cannot see it, query it with their own images, or inspect where a query lands relative to other years on the time axis. We present TimeLineXR, an interactive demonstration that turns this time axis into an explorable 3D scene. A web demo lets users upload or select images and see each placed on a color-coded 3D time axis; a Three.js WebXR scene renders the same time axis as a room-scale representation that can be grabbed, scaled, inspected, and queried by throwing image cards into it. Both clients run the same prediction pipeline against the same cached temporal index: encode the query with CLIP, project it through cosine-kernel KPCA, and snap it to the Bézier time axis. The system turns each image into a point on the time axis, so users can compare where different objects land and read off their estimated first year of appearance.Code source and reproduction steps can be found in: https://github.com/sterngefeuert/timeline-xr.

0.32

EyeTAG: Eye Trajectory-Aware Gaze Estimation

cs.CV cs.LG ·2026-10-01 ·Jungmin Lee, Niamat Ullah, Yoseob Han
publishedcode/data similarity 0.59
Abstract · Accepted to BMVC 2026

Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.

0.38

EgoHRV: Continuous Heart Rate Variability Estimation from Egocentric Systems for Autonomic Response and Skill Assessment

cs.CV eess.SP ·2026-08-19 ·Berken Utku Demirel, Christian Holz
top venuecode/data similarity 0.58
Abstract · Accepted to the European Conference on Computer Vision (ECCV) 2026

Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used noninvasive marker of autonomic regulation under stress. HRV reflects small timing differences between successive heartbeats and has so far been out of reach for egocentric platforms, where motion and noise in gaze video mask exactly this fine-grained timing. We propose EgoHRV, a method that estimates HRV as well as heart rate (HR) from the gaze cameras that are already integrated into egocentric headsets. Our pipeline combines a 3D backbone with a novel low--high decomposition module that extracts the blood volume pulse (BVP) signal from gaze video. Our cross-domain pretraining aligns the frequency-domain representations of contact-based and camera-derived signals. This alignment gives EgoHRV the temporal precision to recover HRV from the subtle fluctuations in gaze video. EgoHRV achieves state-of-the-art accuracy for HR and HRV estimation from egocentric video, and its uncertainty-aware design improves downstream behavioral modeling. Integrating our HRV estimates and confidence measures into EgoExo4D's proficiency estimator raises accuracy by 17.8%. Beyond skill, continuous HRV estimation also opens egocentric systems to stress- and arousal-aware estimation tasks. Code: https://github.com/eth-siplab/EgoHRV

0.48

PhotoFinish: Portable Timerless Probing of Microarchitectural Latencies

cs.CR ·2026-11-06 ·Tristan Hornetz, Eduard Ebert, Lorenz Hetterich, Lukas Gerlach, Michael Schwarz
similarity 0.58
Abstract

Timing measurements are a fundamental building block of microarchitectural attacks, enabling software to convert otherwise invisible latency differences into architectural observations. Thus, restricting high-resolution timers is a common defense, especially in constrained environments such as web browsers. Timerless probing primitives provide an alternative approach. Instead of reading or constructing a timer, they observe architectural events that correlate with the microarchitectural state. However, such primitives typically rely on platform-specific behavior, and none generalizes across the wider computing landscape. In this paper, we present PhotoFinish, a portable timerless probing primitive that converts latency differences directly into an architectural value. PhotoFinish races an operation under test against a fixed reference in two parallel threads that then write unique values to the same shared location, resulting in a latency-dependent architectural value. PhotoFinish only requires parallel execution and shared memory, without constructing or reading a clock. We evaluate PhotoFinish on 38 CPUs spanning four ISAs and demonstrate reliable cache state classification across widely different microarchitectures, with comparable fidelity when compiled to WebAssembly and executed inside a browser. We further use PhotoFinish in AES T-table and Evict+Reload attacks, show that it applies beyond caches to TLB and execution-engine latencies, and demonstrate sufficient precision to identify the LLC slice of an address. Our results demonstrate shared-memory races as a generic and precise replacement for explicit timing, showing that restricting timers alone does not prevent software from observing fine-grained microarchitectural latency differences.

0.25

How Well Does Webcam Eye Tracking Work "in the Wild"? A Large-Scale Validation of Chiasm and a Comparison with WebGazer

cs.CV ·2026-11-03 ·Artem V. Belopolsky, Florian Perdreau
code/data similarity 0.58
Abstract · 37 pages, 21 figures, 2 tables

Webcam eye tracking can greatly accelerate data collection in basic and applied research by reaching large, diverse samples of participants online. However, its spatial performance is considered well below that of hardware-based systems, limiting its use. We evaluated the first-generation Chiasm webcam eye tracker in a large online study ($N = 505$) spanning diverse participants, devices, and recording conditions. Chiasm achieved a mean spatial accuracy of 2.39 cm ($\approx 2.28^\circ$ at 60 cm), spatial precision (SD) of 1.05 cm ($\approx 1.00^\circ$ at 60 cm), and sample-to-sample precision (RMS-S2S) of 1.29 cm ($\approx 1.23^\circ$ at 60 cm). For the 91% of participants who met a 3 cm calibration criterion, spatial accuracy was 2.15 cm ($\approx 2.05^\circ$ at 60 cm). After three minutes of a task that encouraged head movement, spatial accuracy declined by 0.59 cm, while the gaze center at each target shifted by a mean distance of only 0.37 cm. On the validation block, the offset was 0.30 cm larger for men than for women. Age, region of residence, face luminance, and Michelson contrast showed no reliable relationship with spatial accuracy. A higher eye aspect ratio, a more open eye, was associated with a smaller offset. Wearing glasses decreased spatial accuracy on every validation block. In a comparison between a random subsample ($N = 101$) and a separate sample tested with the open-source tracker WebGazer ($N = 101$) using an identical protocol, Chiasm showed better spatial accuracy (2.18 vs. 4.51 cm) and spatial precision (SD; 1.01 vs. 1.55 cm). WebGazer showed better sample-to-sample precision (RMS-S2S; 0.79 vs. 1.26 cm), which likely reflects conservative, over-smoothed estimates rather than superior stability. These results suggest that Chiasm is narrowing the gap between webcam and hardware-based eye tracking, making large-scale gaze research "in the wild" a reality.

0.34